NVIDIA RTX 5070

GPU

NVIDIA Blackwell (GB205): 12 GB GDDR7 on a 192-bit bus, 672 GB/s of memory bandwidth, 250 W board power. Bandwidth is up about a third over the RTX 4070, though the 12 GB capacity bounds model size.

110

Run records

24

Models covered

5

Frameworks covered

L0 Self-reported × 58 · L1 Reproduced × 1 · L2 Cross-framework verified × 51

Evidence mix

Typical measured performance

Published benchmark averages per model × framework × quantization on this hardware.

  1. Ollama · Q3_K_S Decode 115.6 · Prefill 548.24 ·
  2. Ollama · Q5_K_S Decode 83.31 · Prefill 418.84 ·
  3. llama.cpp · Q4_K_M Decode 186.5 · Prefill 627.46 ·
  4. Ollama · FP16 Decode 84.08 · Prefill 290.65 ·
  5. Ollama · IQ1_S Decode 239.86 · Prefill 1174.67 ·
  6. vLLM · IQ1_S Decode 119.3 · Prefill 428.02 ·
  7. Strata · IQ1_S Decode 153.3 · Prefill 756.44 ·
  8. Strata · IQ2_XS Decode 135.12 · Prefill 577.01 ·
  9. llama.cpp · IQ1_S Decode 116.33 · Prefill 578.18 ·
  10. llama.cpp · Q3_K_M Decode 92.37 · Prefill 403.56 ·
  11. vLLM · IQ1_S Decode 135.82 · Prefill 714.11 ·
  12. vLLM · Q2_K Decode 123.84 · Prefill 585.88 ·
  13. llama.cpp · IQ4_XS Decode 238.16 · Prefill 793.59 ·
  14. llama.cpp · Q5_K_S Decode 193.78 · Prefill 821.91 ·
  15. Ollama · FP16 Decode 100.33 · Prefill 407.38 ·
  16. Ollama · Q3_K_L Decode 203.4 · Prefill 838.81 ·
  17. vLLM · Q3_K_L Decode 76.49 · Prefill 291.62 ·
  18. Strata · Q5_K_M Decode 76.48 · Prefill 397.87 ·
  19. vLLM · Q3_K_M Decode 112.72 · Prefill 436.5 ·
  20. vLLM · Q4_K_M Decode 84.51 · Prefill 408.98 ·
  21. Strata · IQ3_S Decode 154.26 · Prefill 563.04 ·
  22. vLLM · IQ3_XS Decode 223.3 · Prefill 1047.53 ·
  23. vLLM · IQ4_XS Decode 188.25 · Prefill 873.82 ·
  24. Ollama · Q3_K_S Decode 197.53 · Prefill 685.98 ·
  25. Ollama · Q5_K_S Decode 124.72 · Prefill 677.51 ·
  26. Strata · IQ4_XS Decode 233.02 · Prefill 915.89 ·
  27. Strata · Q6_K Decode 189.01 · Prefill 746.35 ·
  28. Ollama · IQ1_S Decode 93.91 · Prefill 396.52 ·
  29. llama.cpp · IQ2_XS Decode 93.24 · Prefill 441.9 ·
  30. llama.cpp · Q3_K_S Decode 83.47 · Prefill 327.27 ·
  31. Strata · IQ3_XS Decode 128.34 · Prefill 480.77 ·
  32. vLLM · IQ2_M Decode 185.93 · Prefill 658.25 ·
  33. Ollama · Q3_K_L Decode 127.9 · Prefill 682.85 ·
  34. Ollama · Q5_K_M Decode 93.03 · Prefill 365.14 ·
  35. Strata · Q3_K_L Decode 187.58 · Prefill 667.52 ·
  36. llama.cpp · IQ4_XS Decode 54 · Prefill 640 ·
  37. llama.cpp · Q4_K_M Decode 50 · Prefill 586 ·
  38. llama.cpp · Q5_K_M Decode 52 · Prefill 454 ·
  39. llama.cpp · Q8_0 Decode 36 · Prefill 394 ·
  40. vLLM · IQ4_XS Decode 51 · Prefill 486 ·
  41. vLLM · Q4_K_M Decode 47 · Prefill 429 ·
  42. vLLM · Q5_K_M Decode 42 · Prefill 447 ·
  43. vLLM · Q8_0 Decode 28 · Prefill 334 ·
  44. Ollama · IQ4_XS Decode 55 · Prefill 549 ·
  45. Ollama · Q4_K_M Decode 50 · Prefill 516 ·
  46. Ollama · Q5_K_M Decode 42 · Prefill 504 ·
  47. Ollama · Q8_0 Decode 36 · Prefill 356 ·
  48. Strata · IQ4_XS Decode 70 · Prefill 837 ·
  49. Strata · Q4_K_M Decode 64 · Prefill 768 ·
  50. Strata · Q5_K_M Decode 72 · Prefill 739 ·

Fit check

Physics-based estimate from VRAM and memory bandwidth: pick a model to see which quantizations fit and the expected speed

Formula: tok/s = (bandwidth ÷ model size) × 0.55, adapted from llmfit; see run records for measured numbers

Key specs

Vendor
NVIDIA
Architecture
Blackwell
VRAM
12 GB
Memory bandwidth
672 GB/s
TDP
250 W
Release date
2025-03-05

Run records

50 configurations

ModelFrameworkQuantizationDecodeGen timeSamples
Qwen3.5-4BOllamaQ3_K_S115.6 tok/s—1
Qwen3.5-4BOllamaQ5_K_S83.3 tok/s—1
Qwen3.5-0.8Bllama.cppQ4_K_M186.5 tok/s—1
Qwen3.5-0.8BOllamaFP1684.1 tok/s—1
Qwen3.5-0.8BOllamaIQ1_S239.9 tok/s—1
Qwen3.8-9BvLLMIQ1_S119.3 tok/s—1
Qwen3.8-9BStrataIQ1_S153.3 tok/s—1
Qwen3.8-9BStrataIQ2_XS135.1 tok/s—1
Qwen3-8Bllama.cppIQ1_S116.3 tok/s—1
Qwen3-8Bllama.cppQ3_K_M92.4 tok/s—1
Llama-3.1-8B-InstructvLLMIQ1_S135.8 tok/s—1
Llama-3-8B-InstructvLLMQ2_K123.8 tok/s—1
Llama-3.2-3B-Instructllama.cppIQ4_XS238.2 tok/s—1
Llama-3.2-3B-Instructllama.cppQ5_K_S193.8 tok/s—1
Llama-3.2-3B-InstructOllamaFP16100.3 tok/s—1
Llama-3.2-3B-InstructOllamaQ3_K_L203.4 tok/s—1
Kimi-K3vLLMQ3_K_L76.5 tok/s—1
Kimi-K3StrataQ5_K_M76.5 tok/s—1
Qwen3.5-4BvLLMQ3_K_M112.7 tok/s—1
Qwen3.5-4BvLLMQ4_K_M84.5 tok/s—1
Qwen3.5-4BStrataIQ3_S154.3 tok/s—1
Qwen3.5-0.8BvLLMIQ3_XS223.3 tok/s—1
Qwen3.5-0.8BvLLMIQ4_XS188.3 tok/s—1
Qwen3.5-0.8BOllamaQ3_K_S197.5 tok/s—1
Qwen3.5-0.8BOllamaQ5_K_S124.7 tok/s—1
Qwen3.5-0.8BStrataIQ4_XS233 tok/s—1
Qwen3.5-0.8BStrataQ6_K189 tok/s—1
Qwen3.8-9BOllamaIQ1_S93.9 tok/s—1
Qwen3-8Bllama.cppIQ2_XS93.2 tok/s—1
Qwen3-8Bllama.cppQ3_K_S83.5 tok/s—1
Qwen3-8BStrataIQ3_XS128.3 tok/s—1
Qwen3-1.7BvLLMIQ2_M185.9 tok/s—1
Qwen3-1.7BOllamaQ3_K_L127.9 tok/s—1
Qwen3-1.7BOllamaQ5_K_M93 tok/s—1
Qwen3-1.7BStrataQ3_K_L187.6 tok/s—1
Qwen3.5-27Bllama.cppIQ4_XS54 tok/s—1
Qwen3.5-27Bllama.cppQ4_K_M50 tok/s—1
Qwen3.5-27Bllama.cppQ5_K_M52 tok/s—1
Qwen3.5-27Bllama.cppQ8_036 tok/s—1
Qwen3.5-27BvLLMIQ4_XS51 tok/s—1
Qwen3.5-27BvLLMQ4_K_M47 tok/s—1
Qwen3.5-27BvLLMQ5_K_M42 tok/s—1
Qwen3.5-27BvLLMQ8_028 tok/s—1
Qwen3.5-27BOllamaIQ4_XS55 tok/s—1
Qwen3.5-27BOllamaQ4_K_M50 tok/s—1
Qwen3.5-27BOllamaQ5_K_M42 tok/s—1
Qwen3.5-27BOllamaQ8_036 tok/s—1
Qwen3.5-27BStrataIQ4_XS70 tok/s—1
Qwen3.5-27BStrataQ4_K_M64 tok/s—1
Qwen3.5-27BStrataQ5_K_M72 tok/s—1