NVIDIA RTX 3080

GPU

Ampere; 10GB VRAM, 760 GB/s bandwidth, 320W TDP, CUDA cores 8,704. Release date: 2020-09-17. Source: LCZ hardware catalog, with missing fields supplemented from NVIDIA official materials.

23

Run records

10

Models covered

4

Frameworks covered

L0 Self-reported × 1 · L2 Cross-framework verified × 22

Evidence mix

Typical measured performance

Published benchmark averages per model × framework × quantization on this hardware.

  1. vLLM · Q5_K_M Decode 82.42 · Prefill 333.84 ·
  2. Ollama · IQ2_M Decode 105.52 · Prefill 563.01 ·
  3. Ollama · IQ3_S Decode 92.66 · Prefill 391.24 ·
  4. llama.cpp · IQ3_S Decode 193.95 · Prefill 916.23 ·
  5. llama.cpp · IQ3_XS Decode 222.89 · Prefill 1012.61 ·
  6. vLLM · IQ1_S Decode 114 · Prefill 526.44 ·
  7. llama.cpp · IQ2_XS Decode 131.47 · Prefill 451.18 ·
  8. llama.cpp · Q3_K_L Decode 99.23 · Prefill 538.67 ·
  9. Ollama · IQ2_XS Decode 118.69 · Prefill 621.83 ·
  10. Ollama · Q2_K Decode 102.28 · Prefill 452.09 ·
  11. Ollama · Q3_K_M Decode 123.99 · Prefill 545.14 ·
  12. Strata · Q3_K_L Decode 204.38 · Prefill 963.09 ·
  13. Strata · Q4_K_S Decode 193.06 · Prefill 868.91 ·
  14. Ollama · Q2_K Decode 93.59 · Prefill 336.53 ·
  15. Ollama · Q4_K_S Decode 73.95 · Prefill 380.29 ·
  16. Strata · IQ3_S Decode 134.76 · Prefill 504.77 ·
  17. llama.cpp · IQ2_XS Decode 279.47 · Prefill 1475.5 ·
  18. Ollama · Q3_K_M Decode 199.79 · Prefill 1003.86 ·
  19. Ollama · Q4_K_M Decode 169.99 · Prefill 687.88 ·
  20. Ollama · IQ2_M Decode 129.13 · Prefill 676.38 ·
  21. Strata · Q6_K Decode 102.47 · Prefill 544.89 ·
  22. Strata · Q4_K_M Decode 249.25 · Prefill 976.82 ·
  23. llama.cpp · PTQ1_0 Decode 52.18 · Prefill 1210.6 ·

Fit check

Physics-based estimate from VRAM and memory bandwidth: pick a model to see which quantizations fit and the expected speed

Formula: tok/s = (bandwidth ÷ model size) × 0.55, adapted from llmfit; see run records for measured numbers

Key specs

Vendor
NVIDIA
Architecture
Ampere
VRAM
10 GB
Memory bandwidth
760 GB/s
TDP
320 W
Release date
2020-09-17

Run records

23 configurations

ModelFrameworkQuantizationDecodeGen timeSamples
Qwen3.5-4BvLLMQ5_K_M82.4 tok/s—1
Qwen3.5-4BOllamaIQ2_M105.5 tok/s—1
Qwen3.5-4BOllamaIQ3_S92.7 tok/s—1
Qwen3.5-0.8Bllama.cppIQ3_S193.9 tok/s—1
Qwen3.5-0.8Bllama.cppIQ3_XS222.9 tok/s—1
Qwen3-8BvLLMIQ1_S114 tok/s—1
Qwen3-4Bllama.cppIQ2_XS131.5 tok/s—1
Qwen3-4Bllama.cppQ3_K_L99.2 tok/s—1
Qwen3-4BOllamaIQ2_XS118.7 tok/s—1
Qwen3-4BOllamaQ2_K102.3 tok/s—1
Qwen3-1.7BOllamaQ3_K_M124 tok/s—1
Qwen3-1.7BStrataQ3_K_L204.4 tok/s—1
Qwen3-1.7BStrataQ4_K_S193.1 tok/s—1
Qwen3-4B-Thinking-2507OllamaQ2_K93.6 tok/s—1
Qwen3-4B-Thinking-2507OllamaQ4_K_S74 tok/s—1
Qwen3-4B-Thinking-2507StrataIQ3_S134.8 tok/s—1
Qwen2.5-0.5B-Instructllama.cppIQ2_XS279.5 tok/s—1
Qwen2.5-0.5B-InstructOllamaQ3_K_M199.8 tok/s—1
Qwen2.5-0.5B-InstructOllamaQ4_K_M170 tok/s—1
Qwen2.5-Omni-3BOllamaIQ2_M129.1 tok/s—1
Qwen2.5-Omni-3BStrataQ6_K102.5 tok/s—1
Qwen3-Embedding-0.6BStrataQ4_K_M249.3 tok/s—1
Ternary Bonsai 2 27Bllama.cppPTQ1_052.2 tok/s—1