Arao AI

Benchmark search

Try these searches

Found 83 run configurations

Each row is a run configuration (model + hardware + framework + quantization); repeated runs of the same setup are merged — raw counts appear in each result’s source line.

ModelQwen3-Embedding-0.6B

Qwen3-Embedding-0.6B · IQ2_XS

L2 Cross-framework verified

NVIDIA RTX 5090 · 32GB · vLLM

Other

3 days ago

304.51 tok/s

Decode

1,432.77 tok/s

Prefill

0.21 s

TTFT

2.71 GB

VRAM

Config (latest run)
Windows 11 23H2 CUDA 12.8 IQ2_XS vLLM v0.48

Sources: Other · 1 runs · 1 independent sources

View details

Qwen3-Embedding-0.6B · Q3_K_S

L2 Cross-framework verified

NVIDIA RTX 5090 · 32GB · vLLM

Other

3 days ago

322.2 tok/s

Decode

1,743.15 tok/s

Prefill

2.04 s

TTFT

2.92 GB

VRAM

Config (latest run)
Ubuntu 22.04 CUDA 12.4 Q3_K_S vLLM v0.26 Flash Attention

Sources: Other · 1 runs · 1 independent sources

View details

Qwen3-Embedding-0.6B · Q2_K

L2 Cross-framework verified

NVIDIA RTX 5090 · 32GB · Strata

Other

3 days ago

350.56 tok/s

Decode

1,533.82 tok/s

Prefill

1.55 s

TTFT

2.79 GB

VRAM

Config (latest run)
macOS 15.0 CUDA 12.4 Q2_K Strata v0.24 Flash Attention

Sources: Other · 1 runs · 1 independent sources

View details

Qwen3-Embedding-0.6B · Q5_K_M

L2 Cross-framework verified

NVIDIA RTX 5090 · 32GB · Strata

Other

3 days ago

291.92 tok/s

Decode

1,133.9 tok/s

Prefill

2.01 s

TTFT

3.28 GB

VRAM

Config (latest run)
macOS 15.0 CUDA 12.6 Q5_K_M Strata v0.48

Sources: Other · 1 runs · 1 independent sources

View details

Qwen3-Embedding-0.6B · IQ1_S

L2 Cross-framework verified

AMD Ryzen AI Max+ 395 · 128GB · llama.cpp

Other

3 days ago

285.55 tok/s

Decode

1,529.78 tok/s

Prefill

1.12 s

TTFT

2.62 GB

VRAM

Config (latest run)
macOS 15.0 CUDA N/A IQ1_S llama.cpp v0.36 Flash Attention

Sources: Other · 1 runs · 1 independent sources

View details

Qwen3-Embedding-0.6B · Q5_K_S

L2 Cross-framework verified

AMD Ryzen AI Max+ 395 · 128GB · Ollama

Other

3 days ago

220.14 tok/s

Decode

748.53 tok/s

Prefill

1.23 s

TTFT

3.23 GB

VRAM

Config (latest run)
Windows 11 23H2 CUDA N/A Q5_K_S Ollama v0.34

Sources: Other · 1 runs · 1 independent sources

View details

Qwen3-Embedding-0.6B · Q6_K

L2 Cross-framework verified

Apple M5 Max · 128GB · llama.cpp

Other

3 days ago

297.37 tok/s

Decode

1,423.38 tok/s

Prefill

1.96 s

TTFT

3.36 GB

VRAM

Config (latest run)
Ubuntu 22.04 CUDA 12.4 Q6_K llama.cpp v0.50 Flash Attention

Sources: Other · 1 runs · 1 independent sources

View details

Qwen3-Embedding-0.6B · FP16

L2 Cross-framework verified

Apple M5 Max · 128GB · vLLM

Other

3 days ago

257.66 tok/s

Decode

966.33 tok/s

Prefill

1.43 s

TTFT

4.43 GB

VRAM

Config (latest run)
Ubuntu 22.04 CUDA 12.4 FP16 vLLM v0.50 Flash Attention

Sources: Other · 1 runs · 1 independent sources

View details

Qwen3-Embedding-0.6B · IQ2_M

L2 Cross-framework verified

Apple M5 Max · 128GB · vLLM

Other

3 days ago

302.89 tok/s

Decode

1,094.91 tok/s

Prefill

2 s

TTFT

2.75 GB

VRAM

Config (latest run)
macOS 15.0 CUDA 12.8 IQ2_M vLLM v0.32

Sources: Other · 1 runs · 1 independent sources

View details

Qwen3-Embedding-0.6B · IQ1_S

L2 Cross-framework verified

NVIDIA DGX Spark · 128GB · vLLM

Other

3 days ago

353.22 tok/s

Decode

1,765.39 tok/s

Prefill

0.67 s

TTFT

2.62 GB

VRAM

Config (latest run)
Windows 11 23H2 CUDA 12.8 IQ1_S vLLM v0.45 Flash Attention

Sources: Other · 1 runs · 1 independent sources

View details

Qwen3-Embedding-0.6B · Q5_K_S

L2 Cross-framework verified

NVIDIA DGX Spark · 128GB · vLLM

Other

3 days ago

364.71 tok/s

Decode

1,715.93 tok/s

Prefill

1.71 s

TTFT

3.23 GB

VRAM

Config (latest run)
Ubuntu 24.04 CUDA 12.6 Q5_K_S vLLM v0.47 Flash Attention

Sources: Other · 1 runs · 1 independent sources

View details

Qwen3-Embedding-0.6B · IQ2_M

L2 Cross-framework verified

NVIDIA DGX Spark · 128GB · Ollama

Other

3 days ago

338.92 tok/s

Decode

1,553.22 tok/s

Prefill

1.7 s

TTFT

2.75 GB

VRAM

Config (latest run)
Windows 11 23H2 CUDA 12.6 IQ2_M Ollama v0.31

Sources: Other · 1 runs · 1 independent sources

View details

Qwen3-Embedding-0.6B · IQ1_S

L2 Cross-framework verified

NVIDIA Jetson AGX Orin 64GB · 64GB · llama.cpp

Other

3 days ago

232.92 tok/s

Decode

851.04 tok/s

Prefill

1.8 s

TTFT

2.62 GB

VRAM

Config (latest run)
JetPack 6.0 CUDA 12.6 IQ1_S llama.cpp v0.45 Flash Attention

Sources: Other · 1 runs · 1 independent sources

View details

Qwen3-Embedding-0.6B · Q5_K_S

L2 Cross-framework verified

NVIDIA Jetson AGX Orin 64GB · 64GB · llama.cpp

Other

3 days ago

140.38 tok/s

Decode

456.07 tok/s

Prefill

0.39 s

TTFT

3.23 GB

VRAM

Config (latest run)
JetPack 6.0 CUDA 12.6 Q5_K_S llama.cpp v0.24 Flash Attention

Sources: Other · 1 runs · 1 independent sources

View details

Qwen3-Embedding-0.6B · Q5_K_S

L2 Cross-framework verified

NVIDIA Jetson AGX Orin 64GB · 64GB · Ollama

Other

3 days ago

120.47 tok/s

Decode

591.73 tok/s

Prefill

1.27 s

TTFT

3.23 GB

VRAM

Config (latest run)
JetPack 6.0 CUDA 12.6 Q5_K_S Ollama v0.50

Sources: Other · 1 runs · 1 independent sources

View details

Qwen3-Embedding-0.6B · IQ2_XS

L2 Cross-framework verified

AMD Strix Halo (Radeon 8060S) · vLLM

Other

3 days ago

355.91 tok/s

Decode

1,759.4 tok/s

Prefill

0.94 s

TTFT

2.71 GB

VRAM

Config (latest run)
macOS 15.0 CUDA N/A IQ2_XS vLLM v0.36 Flash Attention

Sources: Other · 1 runs · 1 independent sources

View details

Qwen3-Embedding-0.6B · IQ3_S

L2 Cross-framework verified

AMD Strix Halo (Radeon 8060S) · vLLM

Other

3 days ago

267.01 tok/s

Decode

1,239.81 tok/s

Prefill

0.37 s

TTFT

2.93 GB

VRAM

Config (latest run)
Windows 11 23H2 CUDA N/A IQ3_S vLLM v0.38

Sources: Other · 1 runs · 1 independent sources

View details

Qwen3-Embedding-0.6B · IQ3_S

L2 Cross-framework verified

AMD Strix Halo (Radeon 8060S) · Strata

Other

3 days ago

374.22 tok/s

Decode

1,937.23 tok/s

Prefill

0.42 s

TTFT

2.93 GB

VRAM

Config (latest run)
Ubuntu 22.04 CUDA N/A IQ3_S Strata v0.29 Flash Attention

Sources: Other · 1 runs · 1 independent sources

View details

Qwen3-Embedding-0.6B · FP16

L2 Cross-framework verified

2x AMD Strix Halo · Ollama

Other

3 days ago

181.91 tok/s

Decode

736.26 tok/s

Prefill

2 s

TTFT

4.43 GB

VRAM

Config (latest run)
Ubuntu 24.04 CUDA N/A FP16 Ollama v0.20 Flash Attention

Sources: Other · 1 runs · 1 independent sources

View details

Qwen3-Embedding-0.6B · IQ4_XS

L2 Cross-framework verified

2x AMD Strix Halo · Ollama

Other

3 days ago

379.81 tok/s

Decode

1,812.54 tok/s

Prefill

1.53 s

TTFT

3.03 GB

VRAM

Config (latest run)
macOS 15.0 CUDA N/A IQ4_XS Ollama v0.46

Sources: Other · 1 runs · 1 independent sources

View details

83 results

Related models
Related hardware
Related frameworks
Related quantizations