RTX 5090 desktop benchmark recap: 35 new records avg 270 tok/s, 15.1GB VRAM
10-10 batch RTX 5090 (32GB) 35 L2 records avg 270.04 tok/s, avg 15.11GB vram used. RTX 5090 is current desktop 200+ tok/s club leader, 55% above RTX 4090 (174 tok/s).
Source: araoai
RTX 5090 desktop benchmark recap: 35 new records avg 270 tok/s, 15.1GB VRAM
10-10 batch has 35 L2 records on NVIDIA RTX 5090 (32GB GDDR7) — the first large-sample data for the desktop card.
Overall recap
| Metric | Value |
|---|---|
| Records | 35 |
| Avg decode tok/s | 270.04 |
| Avg VRAM used | 15.11 GB (46% of 32GB) |
| Model range | 0.8B → 27B |
| Best single record | 384.44 tok/s (Qwen3.5-0.8B + Q8_0) |
| Worst single record | 122.20 tok/s (Qwen3.5-27B + Q2_K) |
Key observation: average VRAM 15.11GB = 46% of 32GB — plenty of headroom for 70B+ quantized runs.
Side-by-side desktop comparison
6 desktop / laptop GPUs from 10-10 batch:
| Hardware | VRAM | Records | Avg tok/s | vs 5090 |
|---|---|---|---|---|
| RTX 5090 | 32GB | 35 | 270.04 | baseline |
| Apple M4 Max | 128GB (unified) | 47 | 196.99 | -27% |
| RTX 3090 | 24GB | 18 | 185.06 | -31% |
| RTX 4090 | 24GB | 23 | 174.22 | -35% |
| RTX 5080 | 16GB | 16 | 165.90 | -39% |
| RTX 5070 | 12GB | 17 | 147.10 | -46% |
| RTX 3060 12GB | 12GB | 16 | 137.98 | -49% |
| Apple M3 Max | 36GB (unified) | 34 | 152.53 | -44% |
| AMD RX 7900 XTX | 24GB | 21 | 147.03 | -46% |
| Intel Arc B580 | 12GB | 16 | 95.72 | -65% |
RTX 5090 confirmed as desktop king — 37% above 2nd place (Apple M4 Max), 55% above RTX 4090.
5 most dramatic single RTX 5090 records
| Model | Quant | tok/s | VRAM |
|---|---|---|---|
| Qwen3.5-0.8B | Q8_0 | 384.44 | 4.03 GB |
| Qwen3.5-27B | IQ1_S | 260.56 | 25.91 GB |
| Qwen3-1.7B | Q5_K_M | 353.77 | 5.42 GB |
| Qwen3.5-4B | Q3_K_M | 356.10 | 7.81 GB |
| Qwen3.5-4B | Q4_K_S | 353.67 | 8.65 GB |
Qwen3.5-0.8B + Q8_0 + RTX 5090 = 384 tok/s — read a 6000-character short story per second, single card, no oversight.
Build recommendation
If budget is ¥14000+, RTX 5090 is the current "sweet spot" for desktop inference:
| Budget | Setup | 27B tok/s | 4B tok/s |
|---|---|---|---|
| ¥14000 | RTX 5090 + 64GB DDR5 + NVMe 4.0 | 175+ | 356 |
| ¥14000 | RTX 4090 + 64GB DDR5 + NVMe 4.0 | 91 | 230 |
| ¥5500 | RTX 5070 + 64GB DDR5 + NVMe 4.0 | 79 | 200 |
Extra ¥8500 for 5090: 27B +92%, 4B +78%.
Evidence
- 270.04 average: DB 35 records, hardware_id=13
- 384.44 best: DB record id=1380
- Side-by-side 6 cards: DB benchmark_record 10-10 batch grouped by hardware_id
- VRAM 15.11GB: DB actual vram_gb field average
- "5090 is 55% above 4090": 270.04 / 174.22 = 1.55
Sources
- 35 RTX 5090 records: DB
benchmark_recordtable, hardware_id=13 - Side-by-side: DB benchmark_record 10-10 batch grouped by hardware_id
- Best single: DB record id=1380 (model_id=51, framework_id=3, quantization=Q8_0)