Arao AI

Benchmark search

Try these searches

Found 50 run configurations

Each row is a run configuration (model + hardware + framework + quantization); repeated runs of the same setup are merged — raw counts appear in each result’s source line.

Modelgemma-4-E4B-it

gemma-4-E4B-it · Q3_K_L

L2 Cross-framework verified

NVIDIA RTX 5090 · 32GB · vLLM

Other

3 days ago

336.48 tok/s

Decode

1,309.15 tok/s

Prefill

0.79 s

TTFT

8.15 GB

VRAM

Config (latest run)
Ubuntu 22.04 CUDA 12.6 Q3_K_L vLLM v0.47 Flash Attention

Sources: Other · 1 runs · 1 independent sources

View details

gemma-4-E4B-it · Q3_K_S

L2 Cross-framework verified

NVIDIA RTX 5090 · 32GB · vLLM

Other

3 days ago

310.95 tok/s

Decode

1,691.37 tok/s

Prefill

1.24 s

TTFT

7.56 GB

VRAM

Config (latest run)
Ubuntu 22.04 CUDA 12.6 Q3_K_S vLLM v0.45 Flash Attention

Sources: Other · 1 runs · 1 independent sources

View details

gemma-4-E4B-it · Q3_K_S

L2 Cross-framework verified

NVIDIA RTX 5090 · 32GB · Strata

Other

3 days ago

326.07 tok/s

Decode

1,598 tok/s

Prefill

0.35 s

TTFT

7.56 GB

VRAM

Config (latest run)
Windows 11 23H2 CUDA 12.8 Q3_K_S Strata v0.30

Sources: Other · 1 runs · 1 independent sources

View details

gemma-4-E4B-it · Q5_K_M

L2 Cross-framework verified

NVIDIA RTX 5090 · 32GB · Strata

Other

3 days ago

361.26 tok/s

Decode

1,894.18 tok/s

Prefill

0.61 s

TTFT

9.91 GB

VRAM

Config (latest run)
Ubuntu 24.04 CUDA 12.6 Q5_K_M Strata v0.31 Flash Attention

Sources: Other · 1 runs · 1 independent sources

View details

gemma-4-E4B-it · IQ4_XS

L2 Cross-framework verified

AMD Ryzen AI Max+ 395 · 128GB · vLLM

Other

3 days ago

142.31 tok/s

Decode

714.99 tok/s

Prefill

0.35 s

TTFT

8.32 GB

VRAM

Config (latest run)
macOS 15.0 CUDA N/A IQ4_XS vLLM v0.37 Flash Attention

Sources: Other · 1 runs · 1 independent sources

View details

gemma-4-E4B-it · Q3_K_S

L2 Cross-framework verified

AMD Ryzen AI Max+ 395 · 128GB · vLLM

Other

3 days ago

163.73 tok/s

Decode

675.44 tok/s

Prefill

0.46 s

TTFT

7.56 GB

VRAM

Config (latest run)
Windows 11 23H2 CUDA N/A Q3_K_S vLLM v0.35 Flash Attention

Sources: Other · 1 runs · 1 independent sources

View details

gemma-4-E4B-it · IQ2_XS

L2 Cross-framework verified

AMD Ryzen AI Max+ 395 · 128GB · Strata

Other

3 days ago

273.72 tok/s

Decode

1,347.27 tok/s

Prefill

0.96 s

TTFT

6.13 GB

VRAM

Config (latest run)
Windows 11 23H2 CUDA N/A IQ2_XS Strata v0.41

Sources: Other · 1 runs · 1 independent sources

View details

gemma-4-E4B-it · IQ3_XS

L2 Cross-framework verified

Apple M5 Max · 128GB · Ollama

Other

3 days ago

264.68 tok/s

Decode

1,192.68 tok/s

Prefill

1.24 s

TTFT

7.31 GB

VRAM

Config (latest run)
Windows 11 23H2 CUDA 12.6 IQ3_XS Ollama v0.25

Sources: Other · 1 runs · 1 independent sources

View details

gemma-4-E4B-it · Q2_K

L2 Cross-framework verified

Apple M5 Max · 128GB · Ollama

Other

3 days ago

303.01 tok/s

Decode

1,437.6 tok/s

Prefill

1.76 s

TTFT

6.72 GB

VRAM

Config (latest run)
Ubuntu 24.04 CUDA 12.4 Q2_K Ollama v0.41 Flash Attention

Sources: Other · 1 runs · 1 independent sources

View details

gemma-4-E4B-it · FP16

L2 Cross-framework verified

Apple M5 Max · 128GB · Strata

Other

3 days ago

155.09 tok/s

Decode

679.29 tok/s

Prefill

0.3 s

TTFT

17.64 GB

VRAM

Config (latest run)
Ubuntu 22.04 CUDA 12.4 FP16 Strata v0.53

Sources: Other · 1 runs · 1 independent sources

View details

gemma-4-E4B-it · Q3_K_L

L2 Cross-framework verified

Apple M5 Max · 128GB · Strata

Other

3 days ago

374.77 tok/s

Decode

1,948.71 tok/s

Prefill

0.83 s

TTFT

8.15 GB

VRAM

Config (latest run)
Ubuntu 22.04 CUDA 12.4 Q3_K_L Strata v0.22 Flash Attention

Sources: Other · 1 runs · 1 independent sources

View details

gemma-4-E4B-it · Q2_K

L2 Cross-framework verified

NVIDIA DGX Spark · 128GB · llama.cpp

Other

3 days ago

258.21 tok/s

Decode

920.61 tok/s

Prefill

0.49 s

TTFT

6.72 GB

VRAM

Config (latest run)
Ubuntu 24.04 CUDA 12.8 Q2_K llama.cpp v0.37 Flash Attention

Sources: Other · 1 runs · 1 independent sources

View details

gemma-4-E4B-it · FP16

L2 Cross-framework verified

NVIDIA DGX Spark · 128GB · vLLM

Other

3 days ago

107.3 tok/s

Decode

444.7 tok/s

Prefill

1.39 s

TTFT

17.64 GB

VRAM

Config (latest run)
Windows 11 23H2 CUDA 12.4 FP16 vLLM v0.22 Flash Attention

Sources: Other · 1 runs · 1 independent sources

View details

gemma-4-E4B-it · IQ3_S

L2 Cross-framework verified

NVIDIA Jetson AGX Orin 64GB · 64GB · vLLM

Other

3 days ago

89.49 tok/s

Decode

443.95 tok/s

Prefill

2.24 s

TTFT

7.64 GB

VRAM

Config (latest run)
JetPack 6.0 CUDA 12.6 IQ3_S vLLM v0.40

Sources: Other · 1 runs · 1 independent sources

View details

gemma-4-E4B-it · Q8_0

L2 Cross-framework verified

NVIDIA Jetson AGX Orin 64GB · 64GB · vLLM

Other

3 days ago

46.76 tok/s

Decode

225.72 tok/s

Prefill

0.23 s

TTFT

11.76 GB

VRAM

Config (latest run)
JetPack 6.0 CUDA 12.6 Q8_0 vLLM v0.48 Flash Attention

Sources: Other · 1 runs · 1 independent sources

View details

gemma-4-E4B-it · IQ2_M

L2 Cross-framework verified

NVIDIA Jetson AGX Orin 64GB · 64GB · Ollama

Other

3 days ago

83.45 tok/s

Decode

389.24 tok/s

Prefill

0.17 s

TTFT

6.47 GB

VRAM

Config (latest run)
JetPack 6.0 CUDA 12.6 IQ2_M Ollama v0.38 Flash Attention

Sources: Other · 1 runs · 1 independent sources

View details

gemma-4-E4B-it · IQ3_S

L2 Cross-framework verified

NVIDIA Jetson AGX Orin 64GB · 64GB · Ollama

Other

3 days ago

81.99 tok/s

Decode

290.18 tok/s

Prefill

2.23 s

TTFT

7.64 GB

VRAM

Config (latest run)
JetPack 6.0 CUDA 12.6 IQ3_S Ollama v0.34 Flash Attention

Sources: Other · 1 runs · 1 independent sources

View details

gemma-4-E4B-it · IQ2_M

L2 Cross-framework verified

AMD Strix Halo (Radeon 8060S) · llama.cpp

Other

3 days ago

142.24 tok/s

Decode

506.53 tok/s

Prefill

0.92 s

TTFT

6.47 GB

VRAM

Config (latest run)
Ubuntu 22.04 CUDA N/A IQ2_M llama.cpp v0.26 Flash Attention

Sources: Other · 1 runs · 1 independent sources

View details

gemma-4-E4B-it · Q8_0

L2 Cross-framework verified

AMD Strix Halo (Radeon 8060S) · Strata

Other

3 days ago

92.35 tok/s

Decode

312.7 tok/s

Prefill

1.41 s

TTFT

11.76 GB

VRAM

Config (latest run)
Ubuntu 22.04 CUDA N/A Q8_0 Strata v0.25

Sources: Other · 1 runs · 1 independent sources

View details

gemma-4-E4B-it · IQ3_S

L2 Cross-framework verified

2x AMD Strix Halo · vLLM

Other

3 days ago

197.35 tok/s

Decode

823.81 tok/s

Prefill

1.44 s

TTFT

7.64 GB

VRAM

Config (latest run)
macOS 15.0 CUDA N/A IQ3_S vLLM v0.52

Sources: Other · 1 runs · 1 independent sources

View details

50 results

Related models
Related hardware
Related frameworks
Related quantizations