Qwen2.5-Omni-3B

Name: Qwen2.5-Omni-3B
Author: Qwen

Quantized builds

Benchmark records

Quantized builds

Community and official quantized releases; "linked records" counts only benchmarks tied to that exact release.

No quantized builds cataloged yet.

18 configurations

Hardware	Framework	Quant	Decode	VRAM used	Samples	Evidence
NVIDIA RTX 5060 SUPER	llama.cpp	Q8_0	109.2 tok/s	3.7 GB	1	L0 Self-reported
NVIDIA RTX 5060 SUPER	llama.cpp	Q5_K_M	158.8 tok/s	2.7 GB	1	L0 Self-reported
NVIDIA RTX 5060 SUPER	llama.cpp	Q4_K_M	194.1 tok/s	2.3 GB	1	L0 Self-reported
NVIDIA RTX 5070 SUPER	llama.cpp	Q8_0	218.4 tok/s	3.7 GB	1	L0 Self-reported
NVIDIA RTX 5070 SUPER	llama.cpp	Q5_K_M	317.7 tok/s	2.7 GB	1	L0 Self-reported
NVIDIA RTX 5070 SUPER	llama.cpp	Q4_K_M	388.3 tok/s	2.3 GB	1	L0 Self-reported
NVIDIA RTX 3090	llama.cpp	Q8_0	228.2 tok/s	3.7 GB	1	L0 Self-reported
NVIDIA RTX 3090	llama.cpp	Q5_K_M	331.9 tok/s	2.7 GB	1	L0 Self-reported
NVIDIA RTX 3090	llama.cpp	Q4_K_M	405.6 tok/s	2.3 GB	1	L0 Self-reported
NVIDIA RTX 5070	llama.cpp	Q8_0	163.8 tok/s	3.7 GB	1	L0 Self-reported
NVIDIA RTX 5070	llama.cpp	Q5_K_M	238.3 tok/s	2.7 GB	1	L0 Self-reported
NVIDIA RTX 5070	llama.cpp	Q4_K_M	291.2 tok/s	2.3 GB	1	L0 Self-reported
NVIDIA RTX 5090	llama.cpp	Q8_0	436.8 tok/s	3.7 GB	1	L0 Self-reported
NVIDIA RTX 5090	llama.cpp	Q5_K_M	635.3 tok/s	2.7 GB	1	L0 Self-reported
NVIDIA RTX 5090	llama.cpp	Q4_K_M	776.5 tok/s	2.3 GB	1	L0 Self-reported
NVIDIA RTX 4090	llama.cpp	Q8_0	245.7 tok/s	3.7 GB	1	L0 Self-reported
NVIDIA RTX 4090	llama.cpp	Q5_K_M	357.4 tok/s	2.7 GB	1	L0 Self-reported
NVIDIA RTX 4090	llama.cpp	Q4_K_M	436.8 tok/s	2.3 GB	1	L0 Self-reported

18 of these records are not linked to a specific quantized build; the full set is on the search page.

No discussions yet — start the first one.