Qwen3-4B-Thinking-2507 · Q2_K
L2 Cross-framework verifiedNVIDIA RTX 5090 · 32GB · llama.cpp
Other
2026-10-15
392.48 tok/s
Decode
1,604.11 tok/s
Prefill
0.17 s
TTFT
6.72 GB
VRAM
Config (latest run)
Sources: Other · 1 runs · 1 independent sources
Each row is a run configuration (model + hardware + framework + quantization); repeated runs of the same setup are merged — raw counts appear in each result’s source line.
NVIDIA RTX 5090 · 32GB · llama.cpp
Other
2026-10-15
392.48 tok/s
Decode
1,604.11 tok/s
Prefill
0.17 s
TTFT
6.72 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · llama.cpp
Other
2026-10-15
184.55 tok/s
Decode
803.14 tok/s
Prefill
1.75 s
TTFT
11.76 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · vLLM
Other
2026-10-15
386.87 tok/s
Decode
1,842.03 tok/s
Prefill
2.34 s
TTFT
6.47 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA RTX 5090 · 32GB · vLLM
Other
2026-10-15
337.16 tok/s
Decode
1,247.18 tok/s
Prefill
0.59 s
TTFT
8.15 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Ryzen AI Max+ 395 · 128GB · llama.cpp
Other
2026-10-15
107.36 tok/s
Decode
478 tok/s
Prefill
0.7 s
TTFT
9.91 GB
VRAM
Sources: Other · 2 runs · 1 independent sources
AMD Ryzen AI Max+ 395 · 128GB · llama.cpp
Other
2026-10-15
78.29 tok/s
Decode
263.52 tok/s
Prefill
0.96 s
TTFT
11.76 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Ryzen AI Max+ 395 · 128GB · Ollama
Other
2026-10-15
145.01 tok/s
Decode
581.64 tok/s
Prefill
2 s
TTFT
7.56 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
Apple M5 Max · 128GB · Ollama
Other
2026-10-15
192.3 tok/s
Decode
1,031.35 tok/s
Prefill
0.19 s
TTFT
9.66 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
Apple M5 Max · 128GB · Strata
Other
2026-10-15
281.24 tok/s
Decode
1,191.19 tok/s
Prefill
0.52 s
TTFT
9.91 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA DGX Spark · 128GB · vLLM
Other
2026-10-15
291.5 tok/s
Decode
1,360.04 tok/s
Prefill
1.03 s
TTFT
5.63 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA DGX Spark · 128GB · vLLM
Other
2026-10-15
151.71 tok/s
Decode
832.95 tok/s
Prefill
2.16 s
TTFT
9.91 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA DGX Spark · 128GB · Strata
Other
2026-10-15
220.61 tok/s
Decode
780.86 tok/s
Prefill
1.84 s
TTFT
9.91 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA Jetson AGX Orin 64GB · 64GB · vLLM
Other
2026-10-15
89.8 tok/s
Decode
485.57 tok/s
Prefill
1.3 s
TTFT
7.56 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA Jetson AGX Orin 64GB · 64GB · vLLM
Other
2026-10-15
57.15 tok/s
Decode
278.01 tok/s
Prefill
1.97 s
TTFT
9.66 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
NVIDIA Jetson AGX Orin 64GB · 64GB · Ollama
Other
2026-10-15
77.04 tok/s
Decode
283.31 tok/s
Prefill
1.72 s
TTFT
6.72 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Strix Halo (Radeon 8060S) · llama.cpp
Other
2026-10-15
117.04 tok/s
Decode
404.22 tok/s
Prefill
1.25 s
TTFT
7.56 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Strix Halo (Radeon 8060S) · llama.cpp
Other
2026-10-15
77.99 tok/s
Decode
423.17 tok/s
Prefill
0.35 s
TTFT
9.91 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Strix Halo (Radeon 8060S) · Strata
Other
2026-10-15
206.18 tok/s
Decode
1,005.37 tok/s
Prefill
1.09 s
TTFT
6.13 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
AMD Strix Halo (Radeon 8060S) · Strata
Other
2026-10-15
166.26 tok/s
Decode
669.87 tok/s
Prefill
0.77 s
TTFT
7.64 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
2x AMD Strix Halo · llama.cpp
Other
2026-10-15
181.31 tok/s
Decode
944.53 tok/s
Prefill
2.23 s
TTFT
7.81 GB
VRAM
Sources: Other · 1 runs · 1 independent sources
102 results