2x AMD Strix Halo

Other

45

Run records

15

Models covered

4

Frameworks covered

L2 Cross-framework verified × 45

Evidence mix

Typical measured performance

Published benchmark averages per model × framework × quantization on this hardware.

  1. llama.cpp · FP16 Decode 94.9 · Prefill 326.17 ·
  2. Strata · Q5_K_M Decode 206.52 · Prefill 718.15 ·
  3. Ollama · FP16 Decode 58.36 · Prefill 233.8 ·
  4. Strata · IQ3_XS Decode 234.55 · Prefill 886.87 ·
  5. Ollama · IQ2_M Decode 146.37 · Prefill 764.74 ·
  6. Strata · IQ2_XS Decode 213.76 · Prefill 822.5 ·
  7. vLLM · FP16 Decode 163.9 · Prefill 573.99 ·
  8. vLLM · IQ3_XS Decode 352.1 · Prefill 1673.75 ·
  9. Strata · Q2_K Decode 284.75 · Prefill 1118.17 ·
  10. Strata · Q5_K_S Decode 358.45 · Prefill 1286.06 ·
  11. llama.cpp · IQ1_S Decode 185.41 · Prefill 664.26 ·
  12. llama.cpp · IQ4_XS Decode 145.51 · Prefill 650.25 ·
  13. Strata · IQ2_M Decode 243.64 · Prefill 1152.92 ·
  14. Strata · IQ3_S Decode 220.16 · Prefill 1160.56 ·
  15. vLLM · Q3_K_L Decode 159.39 · Prefill 715.55 ·
  16. vLLM · Q4_K_S Decode 137.45 · Prefill 721.98 ·
  17. Strata · IQ1_S Decode 293.93 · Prefill 1381.03 ·
  18. Strata · Q5_K_M Decode 157.71 · Prefill 787.63 ·
  19. llama.cpp · FP16 Decode 87.02 · Prefill 443.84 ·
  20. Ollama · IQ3_S Decode 197.15 · Prefill 955.03 ·
  21. Ollama · Q4_K_M Decode 165.67 · Prefill 892.04 ·
  22. llama.cpp · Q5_K_S Decode 236.18 · Prefill 1075.25 ·
  23. Strata · Q5_K_M Decode 284.45 · Prefill 1280.23 ·
  24. Strata · Q5_K_S Decode 326.25 · Prefill 1416.72 ·
  25. llama.cpp · Q3_K_M Decode 181.31 · Prefill 944.53 ·
  26. llama.cpp · Q4_K_M Decode 153.87 · Prefill 676.15 ·
  27. Strata · IQ2_M Decode 312.05 · Prefill 1568.66 ·
  28. vLLM · Q3_K_S Decode 149.19 · Prefill 617.1 ·
  29. vLLM · Q8_0 Decode 87.69 · Prefill 408.22 ·
  30. Strata · Q3_K_M Decode 208.39 · Prefill 864.96 ·
  31. Ollama · Q3_K_S Decode 347.47 · Prefill 1631.19 ·
  32. Ollama · Q8_0 Decode 256.99 · Prefill 1220.11 ·
  33. Strata · Q4_K_M Decode 303.04 · Prefill 1091.32 ·
  34. llama.cpp · IQ2_XS Decode 273.09 · Prefill 1000.97 ·
  35. llama.cpp · IQ3_S Decode 239.07 · Prefill 794.3 ·
  36. vLLM · Q5_K_M Decode 183.47 · Prefill 864.96 ·
  37. vLLM · Q6_K Decode 171.22 · Prefill 756.33 ·
  38. Ollama · FP16 Decode 181.91 · Prefill 736.26 ·
  39. Ollama · IQ4_XS Decode 379.81 · Prefill 1812.54 ·
  40. Strata · IQ4_XS Decode 288.7 · Prefill 1574.67 ·
  41. vLLM · Q3_K_S Decode 277.77 · Prefill 1024.78 ·
  42. Strata · IQ3_XS Decode 321.59 · Prefill 1311.64 ·
  43. vLLM · IQ3_S Decode 197.35 · Prefill 823.81 ·
  44. vLLM · Q5_K_M Decode 149.23 · Prefill 534.97 ·
  45. Strata · Q2_K Decode 311.82 · Prefill 1554.79 ·

Fit check

Physics-based estimate from VRAM and memory bandwidth: pick a model to see which quantizations fit and the expected speed

VRAM is not recorded for this hardware; fit cannot be computed

Formula: tok/s = (bandwidth ÷ model size) × 0.55, adapted from llmfit; see run records for measured numbers

Key specs

Vendor
AMD
Memory bandwidth
512 GB/s

Run records

45 configurations

ModelFrameworkQuantizationDecodeGen timeSamples
Qwen3.5-4Bllama.cppFP1694.9 tok/s—1
Qwen3.5-4BStrataQ5_K_M206.5 tok/s—1
Qwen3.5-9BOllamaFP1658.4 tok/s—1
Qwen3.5-9BStrataIQ3_XS234.6 tok/s—1
Qwen3.5-27BOllamaIQ2_M146.4 tok/s—1
Qwen3.5-27BStrataIQ2_XS213.8 tok/s—1
Qwen3.5-0.8BvLLMFP16163.9 tok/s—1
Qwen3.5-0.8BvLLMIQ3_XS352.1 tok/s—1
Qwen3.5-0.8BStrataQ2_K284.8 tok/s—1
Qwen3.5-0.8BStrataQ5_K_S358.4 tok/s—1
Qwen3.8-9Bllama.cppIQ1_S185.4 tok/s—1
Qwen3.8-9Bllama.cppIQ4_XS145.5 tok/s—1
Qwen3.8-9BStrataIQ2_M243.6 tok/s—1
Qwen3.8-9BStrataIQ3_S220.2 tok/s—1
Qwen3-8BvLLMQ3_K_L159.4 tok/s—1
Qwen3-8BvLLMQ4_K_S137.4 tok/s—1
Qwen3-8BStrataIQ1_S293.9 tok/s—1
Qwen3-8BStrataQ5_K_M157.7 tok/s—1
Qwen3-4Bllama.cppFP1687 tok/s—1
Qwen3-4BOllamaIQ3_S197.2 tok/s—1
Qwen3-4BOllamaQ4_K_M165.7 tok/s—1
Qwen3-1.7Bllama.cppQ5_K_S236.2 tok/s—1
Qwen3-1.7BStrataQ5_K_M284.4 tok/s—1
Qwen3-1.7BStrataQ5_K_S326.3 tok/s—1
Qwen3-4B-Thinking-2507llama.cppQ3_K_M181.3 tok/s—1
Qwen3-4B-Thinking-2507llama.cppQ4_K_M153.9 tok/s—1
Qwen3-4B-Thinking-2507StrataIQ2_M312.1 tok/s—1
Qwen3-Coder-NextvLLMQ3_K_S149.2 tok/s—1
Qwen3-Coder-NextvLLMQ8_087.7 tok/s—1
Qwen3-Coder-NextStrataQ3_K_M208.4 tok/s—1
Qwen2.5-0.5B-InstructOllamaQ3_K_S347.5 tok/s—1
Qwen2.5-0.5B-InstructOllamaQ8_0257 tok/s—1
Qwen2.5-0.5B-InstructStrataQ4_K_M303 tok/s—1
Qwen2.5-Omni-3Bllama.cppIQ2_XS273.1 tok/s—1
Qwen2.5-Omni-3Bllama.cppIQ3_S239.1 tok/s—1
Qwen2.5-Omni-3BvLLMQ5_K_M183.5 tok/s—1
Qwen2.5-Omni-3BvLLMQ6_K171.2 tok/s—1
Qwen3-Embedding-0.6BOllamaFP16181.9 tok/s—1
Qwen3-Embedding-0.6BOllamaIQ4_XS379.8 tok/s—1
Qwen3-Embedding-0.6BStrataIQ4_XS288.7 tok/s—1
gemma-4-E2B-itvLLMQ3_K_S277.8 tok/s—1
gemma-4-E2B-itStrataIQ3_XS321.6 tok/s—1
gemma-4-E4B-itvLLMIQ3_S197.3 tok/s—1
gemma-4-E4B-itvLLMQ5_K_M149.2 tok/s—1
gemma-4-E4B-itStrataQ2_K311.8 tok/s—1