NVIDIA RTX 4080 Super

GPU

NVIDIA Ada Lovelace (AD103): 16 GB GDDR6X on a 256-bit bus, 736 GB/s of memory bandwidth, 320 W board power.

80

Run records

20

Models covered

5

Frameworks covered

L0 Self-reported × 30 · L2 Cross-framework verified × 50

Evidence mix

Typical measured performance

Published benchmark averages per model × framework × quantization on this hardware.

  1. Ollama · Q2_K Decode 222.5 · Prefill 1136.51 ·
  2. vLLM · IQ3_XS Decode 194.23 · Prefill 865.43 ·
  3. Ollama · Q3_K_S Decode 155.17 · Prefill 518.65 ·
  4. llama.cpp · Q8_0 Decode 237.08 · Prefill 981.54 ·
  5. Strata · Q4_K_M Decode 305.26 · Prefill 1292.84 ·
  6. llama.cpp · Q4_K_M Decode 162.68 · Prefill 521.64 ·
  7. Ollama · Q3_K_L Decode 145.8 · Prefill 516.93 ·
  8. Strata · IQ3_XS Decode 244.72 · Prefill 1076.88 ·
  9. vLLM · Q4_K_S Decode 190.65 · Prefill 787.91 ·
  10. Ollama · IQ2_M Decode 242.65 · Prefill 957.57 ·
  11. Ollama · Q2_K Decode 231.86 · Prefill 790.98 ·
  12. vLLM · Q3_K_S Decode 302.38 · Prefill 993.94 ·
  13. vLLM · Q4_K_S Decode 370.71 · Prefill 1429.25 ·
  14. Ollama · IQ3_S Decode 352.12 · Prefill 1849.99 ·
  15. Ollama · IQ3_XS Decode 393.56 · Prefill 2131.15 ·
  16. vLLM · IQ2_M Decode 181.18 · Prefill 718.16 ·
  17. Strata · Q3_K_M Decode 213.25 · Prefill 900.93 ·
  18. Strata · Q3_K_S Decode 210.74 · Prefill 935.82 ·
  19. llama.cpp · Q3_K_S Decode 168.36 · Prefill 655.5 ·
  20. vLLM · IQ2_M Decode 214.44 · Prefill 870.17 ·
  21. llama.cpp · Q3_K_M Decode 201.4 · Prefill 1061.69 ·
  22. Strata · IQ2_XS Decode 307.75 · Prefill 1104.97 ·
  23. Strata · Q5_K_M Decode 179.72 · Prefill 584.63 ·
  24. Ollama · IQ3_XS Decode 190.92 · Prefill 657.75 ·
  25. Strata · IQ2_XS Decode 334.52 · Prefill 1091.81 ·
  26. llama.cpp · IQ2_XS Decode 372.98 · Prefill 1597.99 ·
  27. llama.cpp · IQ4_XS Decode 289.96 · Prefill 1071.96 ·
  28. Strata · FP16 Decode 277.08 · Prefill 1141.1 ·
  29. llama.cpp · IQ1_S Decode 208.08 · Prefill 731.51 ·
  30. vLLM · Q5_K_M Decode 152.06 · Prefill 772.68 ·
  31. Ollama · Q3_K_L Decode 202.33 · Prefill 955.67 ·
  32. llama.cpp · Q4_K_M Decode 344.73 · Prefill 1737.55 ·
  33. vLLM · IQ1_S Decode 314.79 · Prefill 1409.84 ·
  34. llama.cpp · IQ1_S Decode 204.16 · Prefill 1111.21 ·
  35. Ollama · IQ2_XS Decode 174.28 · Prefill 597.11 ·
  36. Ollama · Q3_K_M Decode 141.22 · Prefill 455.03 ·
  37. llama.cpp · IQ2_XS Decode 195.22 · Prefill 907.33 ·
  38. Ollama · Q3_K_S Decode 129.2 · Prefill 551.46 ·
  39. vLLM · IQ2_M Decode 224.52 · Prefill 1147.46 ·
  40. vLLM · Q3_K_L Decode 177.01 · Prefill 908.47 ·
  41. Ollama · IQ2_M Decode 204.79 · Prefill 1109.54 ·
  42. vLLM · Q3_K_S Decode 189.4 · Prefill 632.63 ·
  43. Ollama · IQ1_S Decode 212.31 · Prefill 982.11 ·
  44. vLLM · IQ3_S Decode 334.5 · Prefill 1159.32 ·
  45. Strata · Q3_K_L Decode 358.87 · Prefill 1331.79 ·
  46. Strata · IQ2_M Decode 252.15 · Prefill 1022.42 ·
  47. llama.cpp · IQ2_XS Decode 184.95 · Prefill 620.72 ·
  48. Strata · IQ1_S Decode 255.29 · Prefill 866.27 ·
  49. llama.cpp · Q6_K Decode 18.2 · Prefill 136.5 ·
  50. llama.cpp · Q5_K_S Decode 23.6 · Prefill 177 ·

Fit check

Physics-based estimate from VRAM and memory bandwidth: pick a model to see which quantizations fit and the expected speed

Formula: tok/s = (bandwidth ÷ model size) × 0.55, adapted from llmfit; see run records for measured numbers

Key specs

Vendor
NVIDIA
Architecture
Ada Lovelace
VRAM
16 GB
Memory bandwidth
736 GB/s
TDP
320 W
Release date
2024-01-31

Run records

50 configurations

ModelFrameworkQuantizationDecodeGen timeSamples
gemma-4-E4B-itOllamaQ2_K222.5 tok/s—1
Ministral-3-8B-Instruct-2512vLLMIQ3_XS194.2 tok/s—1
Ministral-3-8B-Instruct-2512OllamaQ3_K_S155.2 tok/s—1
iFlytek Spark-X2.5-4Bllama.cppQ8_0237.1 tok/s—1
iFlytek Spark-X2.5-4BStrataQ4_K_M305.3 tok/s—1
NVIDIA-Nemotron-3.5-Bolt-E4B-4GBllama.cppQ4_K_M162.7 tok/s—1
elephant-1OllamaQ3_K_L145.8 tok/s—1
elephant-1StrataIQ3_XS244.7 tok/s—1
Qwen3.5-4BvLLMQ4_K_S190.7 tok/s—1
Qwen3.5-4BOllamaIQ2_M242.7 tok/s—1
Qwen3.5-4BOllamaQ2_K231.9 tok/s—1
Qwen3.5-0.8BvLLMQ3_K_S302.4 tok/s—1
Qwen3.5-0.8BvLLMQ4_K_S370.7 tok/s—1
Qwen3.5-0.8BOllamaIQ3_S352.1 tok/s—1
Qwen3.5-0.8BOllamaIQ3_XS393.6 tok/s—1
Qwen3.8-9BvLLMIQ2_M181.2 tok/s—1
Qwen3.8-9BStrataQ3_K_M213.3 tok/s—1
Qwen3.8-9BStrataQ3_K_S210.7 tok/s—1
Qwen3-8Bllama.cppQ3_K_S168.4 tok/s—1
Qwen3-8BvLLMIQ2_M214.4 tok/s—2
Llama-3.1-8B-Instructllama.cppQ3_K_M201.4 tok/s—1
Llama-3.1-8B-InstructStrataIQ2_XS307.8 tok/s—1
Llama-3.1-8B-InstructStrataQ5_K_M179.7 tok/s—1
Llama-3-8B-InstructOllamaIQ3_XS190.9 tok/s—2
Llama-3-8B-InstructStrataIQ2_XS334.5 tok/s—1
Llama-3.2-3B-Instructllama.cppIQ2_XS373 tok/s—1
Llama-3.2-3B-Instructllama.cppIQ4_XS290 tok/s—1
Llama-3.2-3B-InstructStrataFP16277.1 tok/s—1
DeepSeek-R1-Distill-Qwen-14Bllama.cppIQ1_S208.1 tok/s—1
Qwen3.5-4BvLLMQ5_K_M152.1 tok/s—1
Qwen3.5-4BOllamaQ3_K_L202.3 tok/s—1
Qwen3.5-0.8Bllama.cppQ4_K_M344.7 tok/s—1
Qwen3.5-0.8BvLLMIQ1_S314.8 tok/s—1
Qwen3.8-9Bllama.cppIQ1_S204.2 tok/s—1
Qwen3.8-9BOllamaIQ2_XS174.3 tok/s—1
Qwen3.8-9BOllamaQ3_K_M141.2 tok/s—1
Qwen3-8Bllama.cppIQ2_XS195.2 tok/s—1
gemma-4-12B-itOllamaQ3_K_S129.2 tok/s—1
Llama-3.1-8B-InstructvLLMIQ2_M224.5 tok/s—1
Llama-3.1-8B-InstructvLLMQ3_K_L177 tok/s—1
Llama-3.1-8B-InstructOllamaIQ2_M204.8 tok/s—1
Llama-3-8B-InstructvLLMQ3_K_S189.4 tok/s—1
Llama-3-8B-InstructOllamaIQ1_S212.3 tok/s—1
Llama-3.2-3B-InstructvLLMIQ3_S334.5 tok/s—1
Llama-3.2-3B-InstructStrataQ3_K_L358.9 tok/s—1
DeepSeek-R1-Distill-Qwen-14BStrataIQ2_M252.2 tok/s—1
Kimi-K3llama.cppIQ2_XS184.9 tok/s—1
Kimi-K3StrataIQ1_S255.3 tok/s—1
Qwen3.5-35B-A3Bllama.cppQ6_K18.2 tok/s—1
Qwen3.5-35B-A3Bllama.cppQ5_K_S23.6 tok/s—1