NVIDIA DGX Spark

GPU

Blackwell (GB10); 128GB VRAM, 273 GB/s bandwidth, 140W TDP, CUDA cores unknown. Release date: 2025-01-07. Source: LCZ hardware catalog, with missing fields supplemented from NVIDIA official materials.

85

Run records

22

Models covered

6

Frameworks covered

L0 Self-reported × 9 · L2 Cross-framework verified × 76

Evidence mix

Typical measured performance

Published benchmark averages per model × framework × quantization on this hardware.

  1. vLLM · IQ1_S Decode 306.37 · Prefill 1243.41 ·
  2. Ollama · IQ2_XS Decode 258.67 · Prefill 1172.95 ·
  3. llama.cpp · Q6_K Decode 110.08 · Prefill 496.59 ·
  4. Strata · Q3_K_M Decode 209.28 · Prefill 1036.76 ·
  5. Ollama · IQ3_S Decode 119.95 · Prefill 607.06 ·
  6. Strata · Q4_K_S Decode 169.69 · Prefill 928.54 ·
  7. vLLM · IQ2_XS Decode 310.29 · Prefill 1000.44 ·
  8. Ollama · IQ2_XS Decode 348.98 · Prefill 1689.57 ·
  9. Ollama · Q3_K_M Decode 387.92 · Prefill 1612.07 ·
  10. vLLM · IQ3_S Decode 178.22 · Prefill 972.1 ·
  11. vLLM · Q5_K_M Decode 122.43 · Prefill 399.31 ·
  12. Strata · Q3_K_S Decode 245.76 · Prefill 803.01 ·
  13. vLLM · Q3_K_M Decode 167.2 · Prefill 861.79 ·
  14. Strata · Q5_K_S Decode 197.21 · Prefill 856.13 ·
  15. llama.cpp · Q3_K_M Decode 206.5 · Prefill 1076.24 ·
  16. Strata · Q3_K_S Decode 338.04 · Prefill 1544.38 ·
  17. Strata · Q5_K_M Decode 221.01 · Prefill 864.66 ·
  18. vLLM · Q2_K Decode 374.76 · Prefill 1672.78 ·
  19. vLLM · Q5_K_M Decode 240.21 · Prefill 1012.49 ·
  20. Strata · IQ3_XS Decode 315.95 · Prefill 1231.57 ·
  21. Strata · Q4_K_S Decode 375.7 · Prefill 1600.4 ·
  22. vLLM · IQ1_S Decode 291.5 · Prefill 1360.04 ·
  23. vLLM · Q5_K_M Decode 151.71 · Prefill 832.95 ·
  24. Strata · Q5_K_M Decode 220.61 · Prefill 780.86 ·
  25. Ollama · Q3_K_L Decode 131.46 · Prefill 705.73 ·
  26. Ollama · Q4_K_S Decode 128.78 · Prefill 450.1 ·
  27. Strata · IQ2_M Decode 261.64 · Prefill 916.28 ·
  28. Strata · IQ2_XS Decode 259.54 · Prefill 1190.55 ·
  29. llama.cpp · IQ2_XS Decode 344.66 · Prefill 1710.46 ·
  30. Strata · IQ2_XS Decode 295.77 · Prefill 1437.09 ·
  31. Strata · Q5_K_M Decode 294.11 · Prefill 1222.22 ·
  32. vLLM · Q3_K_M Decode 258.63 · Prefill 1322.45 ·
  33. vLLM · Q5_K_M Decode 183.75 · Prefill 675.77 ·
  34. Ollama · IQ2_M Decode 292.67 · Prefill 1302.54 ·
  35. vLLM · IQ1_S Decode 353.22 · Prefill 1765.39 ·
  36. vLLM · Q5_K_S Decode 364.71 · Prefill 1715.93 ·
  37. Ollama · IQ2_M Decode 338.92 · Prefill 1553.22 ·
  38. vLLM · IQ2_M Decode 380.91 · Prefill 2066.12 ·
  39. vLLM · IQ3_XS Decode 360.01 · Prefill 1436.84 ·
  40. Strata · Q3_K_L Decode 389.31 · Prefill 1363.67 ·
  41. Strata · Q4_K_M Decode 326.71 · Prefill 1358.52 ·
  42. llama.cpp · Q2_K Decode 258.21 · Prefill 920.61 ·
  43. vLLM · FP16 Decode 107.3 · Prefill 444.7 ·
  44. vLLM · Q2_K Decode 266.26 · Prefill 1108.22 ·
  45. vLLM · Q3_K_L Decode 219.42 · Prefill 756.56 ·
  46. llama.cpp · IQ2_M Decode 181.94 · Prefill 684.54 ·
  47. vLLM · IQ2_M Decode 210.69 · Prefill 1114.84 ·
  48. vLLM · Q8_0 Decode 101.73 · Prefill 523.44 ·
  49. Ollama · IQ3_XS Decode 391.28 · Prefill 1950.82 ·
  50. Ollama · IQ4_XS Decode 338.64 · Prefill 1719.5 ·

Fit check

Physics-based estimate from VRAM and memory bandwidth: pick a model to see which quantizations fit and the expected speed

Formula: tok/s = (bandwidth ÷ model size) × 0.55, adapted from llmfit; see run records for measured numbers

Key specs

Vendor
NVIDIA
Architecture
Blackwell (GB10)
VRAM
128 GB
Memory bandwidth
273 GB/s
TDP
140 W
Release date
2025-01-07

Run records

50 configurations

ModelFrameworkQuantizationDecodeGen timeSamples
Qwen3.5-4BvLLMIQ1_S306.4 tok/s—1
Qwen3.5-4BOllamaIQ2_XS258.7 tok/s—1
Qwen3.5-9Bllama.cppQ6_K110.1 tok/s—1
Qwen3.5-9BStrataQ3_K_M209.3 tok/s—1
Qwen3.5-27BOllamaIQ3_S120 tok/s—1
Qwen3.5-27BStrataQ4_K_S169.7 tok/s—1
Qwen3.5-0.8BvLLMIQ2_XS310.3 tok/s—1
Qwen3.5-0.8BOllamaIQ2_XS349 tok/s—1
Qwen3.5-0.8BOllamaQ3_K_M387.9 tok/s—1
Qwen3.8-9BvLLMIQ3_S178.2 tok/s—1
Qwen3.8-9BvLLMQ5_K_M122.4 tok/s—1
Qwen3.8-9BStrataQ3_K_S245.8 tok/s—1
Qwen3-8BvLLMQ3_K_M167.2 tok/s—1
Qwen3-8BStrataQ5_K_S197.2 tok/s—1
Qwen3-4Bllama.cppQ3_K_M206.5 tok/s—1
Qwen3-4BStrataQ3_K_S338 tok/s—1
Qwen3-4BStrataQ5_K_M221 tok/s—2
Qwen3-1.7BvLLMQ2_K374.8 tok/s—1
Qwen3-1.7BvLLMQ5_K_M240.2 tok/s—1
Qwen3-1.7BStrataIQ3_XS315.9 tok/s—1
Qwen3-1.7BStrataQ4_K_S375.7 tok/s—1
Qwen3-4B-Thinking-2507vLLMIQ1_S291.5 tok/s—1
Qwen3-4B-Thinking-2507vLLMQ5_K_M151.7 tok/s—1
Qwen3-4B-Thinking-2507StrataQ5_K_M220.6 tok/s—1
Qwen3-Coder-NextOllamaQ3_K_L131.5 tok/s—1
Qwen3-Coder-NextOllamaQ4_K_S128.8 tok/s—1
Qwen3-Coder-NextStrataIQ2_M261.6 tok/s—1
Qwen3-Coder-NextStrataIQ2_XS259.5 tok/s—1
Qwen2.5-0.5B-Instructllama.cppIQ2_XS344.7 tok/s—1
Qwen2.5-0.5B-InstructStrataIQ2_XS295.8 tok/s—1
Qwen2.5-0.5B-InstructStrataQ5_K_M294.1 tok/s—1
Qwen2.5-Omni-3BvLLMQ3_K_M258.6 tok/s—1
Qwen2.5-Omni-3BvLLMQ5_K_M183.8 tok/s—1
Qwen2.5-Omni-3BOllamaIQ2_M292.7 tok/s—1
Qwen3-Embedding-0.6BvLLMIQ1_S353.2 tok/s—1
Qwen3-Embedding-0.6BvLLMQ5_K_S364.7 tok/s—1
Qwen3-Embedding-0.6BOllamaIQ2_M338.9 tok/s—1
gemma-4-E2B-itvLLMIQ2_M380.9 tok/s—1
gemma-4-E2B-itvLLMIQ3_XS360 tok/s—1
gemma-4-E2B-itStrataQ3_K_L389.3 tok/s—1
gemma-4-E2B-itStrataQ4_K_M326.7 tok/s—1
gemma-4-E4B-itllama.cppQ2_K258.2 tok/s—1
gemma-4-E4B-itvLLMFP16107.3 tok/s—1
Qwen3.5-4BvLLMQ2_K266.3 tok/s—1
Qwen3.5-4BvLLMQ3_K_L219.4 tok/s—1
Qwen3.5-9Bllama.cppIQ2_M181.9 tok/s—1
Qwen3.5-9BvLLMIQ2_M210.7 tok/s—1
Qwen3.5-9BvLLMQ8_0101.7 tok/s—1
Qwen3.5-0.8BOllamaIQ3_XS391.3 tok/s—1
Qwen3.5-0.8BOllamaIQ4_XS338.6 tok/s—1