NVIDIA RTX 3080
GPUAmpere; 10GB VRAM, 760 GB/s bandwidth, 320W TDP, CUDA cores 8,704. Release date: 2020-09-17. Source: LCZ hardware catalog, with missing fields supplemented from NVIDIA official materials.
23
Run records
10
Models covered
4
Frameworks covered
L0 Self-reported × 1 · L2 Cross-framework verified × 22
Evidence mix
Typical measured performance
Published benchmark averages per model × framework × quantization on this hardware.
- vLLM · Q5_K_M Decode 82.42 · Prefill 333.84 ·
- Ollama · IQ2_M Decode 105.52 · Prefill 563.01 ·
- Ollama · IQ3_S Decode 92.66 · Prefill 391.24 ·
- llama.cpp · IQ3_S Decode 193.95 · Prefill 916.23 ·
- llama.cpp · IQ3_XS Decode 222.89 · Prefill 1012.61 ·
- vLLM · IQ1_S Decode 114 · Prefill 526.44 ·
- llama.cpp · IQ2_XS Decode 131.47 · Prefill 451.18 ·
- llama.cpp · Q3_K_L Decode 99.23 · Prefill 538.67 ·
- Ollama · IQ2_XS Decode 118.69 · Prefill 621.83 ·
- Ollama · Q2_K Decode 102.28 · Prefill 452.09 ·
- Ollama · Q3_K_M Decode 123.99 · Prefill 545.14 ·
- Strata · Q3_K_L Decode 204.38 · Prefill 963.09 ·
- Strata · Q4_K_S Decode 193.06 · Prefill 868.91 ·
- Ollama · Q2_K Decode 93.59 · Prefill 336.53 ·
- Ollama · Q4_K_S Decode 73.95 · Prefill 380.29 ·
- Strata · IQ3_S Decode 134.76 · Prefill 504.77 ·
- llama.cpp · IQ2_XS Decode 279.47 · Prefill 1475.5 ·
- Ollama · Q3_K_M Decode 199.79 · Prefill 1003.86 ·
- Ollama · Q4_K_M Decode 169.99 · Prefill 687.88 ·
- Ollama · IQ2_M Decode 129.13 · Prefill 676.38 ·
- Strata · Q6_K Decode 102.47 · Prefill 544.89 ·
- Strata · Q4_K_M Decode 249.25 · Prefill 976.82 ·
- llama.cpp · PTQ1_0 Decode 52.18 · Prefill 1210.6 ·
Fit check
Physics-based estimate from VRAM and memory bandwidth: pick a model to see which quantizations fit and the expected speed
Formula: tok/s = (bandwidth ÷ model size) × 0.55, adapted from llmfit; see run records for measured numbers
Key specs
- Vendor
- NVIDIA
- Architecture
- Ampere
- VRAM
- 10 GB
- Memory bandwidth
- 760 GB/s
- TDP
- 320 W
- Release date
- 2020-09-17
Run records
23 configurations
| Model | Framework | Quantization | Decode | Gen time | Samples |
|---|---|---|---|---|---|
| Qwen3.5-4B | vLLM | Q5_K_M | 82.4 tok/s | — | 1 |
| Qwen3.5-4B | Ollama | IQ2_M | 105.5 tok/s | — | 1 |
| Qwen3.5-4B | Ollama | IQ3_S | 92.7 tok/s | — | 1 |
| Qwen3.5-0.8B | llama.cpp | IQ3_S | 193.9 tok/s | — | 1 |
| Qwen3.5-0.8B | llama.cpp | IQ3_XS | 222.9 tok/s | — | 1 |
| Qwen3-8B | vLLM | IQ1_S | 114 tok/s | — | 1 |
| Qwen3-4B | llama.cpp | IQ2_XS | 131.5 tok/s | — | 1 |
| Qwen3-4B | llama.cpp | Q3_K_L | 99.2 tok/s | — | 1 |
| Qwen3-4B | Ollama | IQ2_XS | 118.7 tok/s | — | 1 |
| Qwen3-4B | Ollama | Q2_K | 102.3 tok/s | — | 1 |
| Qwen3-1.7B | Ollama | Q3_K_M | 124 tok/s | — | 1 |
| Qwen3-1.7B | Strata | Q3_K_L | 204.4 tok/s | — | 1 |
| Qwen3-1.7B | Strata | Q4_K_S | 193.1 tok/s | — | 1 |
| Qwen3-4B-Thinking-2507 | Ollama | Q2_K | 93.6 tok/s | — | 1 |
| Qwen3-4B-Thinking-2507 | Ollama | Q4_K_S | 74 tok/s | — | 1 |
| Qwen3-4B-Thinking-2507 | Strata | IQ3_S | 134.8 tok/s | — | 1 |
| Qwen2.5-0.5B-Instruct | llama.cpp | IQ2_XS | 279.5 tok/s | — | 1 |
| Qwen2.5-0.5B-Instruct | Ollama | Q3_K_M | 199.8 tok/s | — | 1 |
| Qwen2.5-0.5B-Instruct | Ollama | Q4_K_M | 170 tok/s | — | 1 |
| Qwen2.5-Omni-3B | Ollama | IQ2_M | 129.1 tok/s | — | 1 |
| Qwen2.5-Omni-3B | Strata | Q6_K | 102.5 tok/s | — | 1 |
| Qwen3-Embedding-0.6B | Strata | Q4_K_M | 249.3 tok/s | — | 1 |
| Ternary Bonsai 2 27B | llama.cpp | PTQ1_0 | 52.2 tok/s | — | 1 |