DeepSeek-R1-Distill-Qwen-14B + Qwen3.5-9B first DB entries: distilled model hits 130 tok/s on 12GB card
10-13 batch: 27 Qwen3.5-9B records avg 109.44 tok/s, 18 DeepSeek-R1-Distill-Qwen-14B records avg 108.01 tok/s. 9B and 14B distilled models are the sweet spot for 12-16GB cards.
Source: araoai
DeepSeek-R1-Distill-Qwen-14B + Qwen3.5-9B first DB entries: distilled model hits 130 tok/s on 12GB card
10-13 batch covers two models for the first time: Qwen3.5-9B (id=49) and DeepSeek-R1-Distill-Qwen-14B (id=71).
Overall recap
| Metric | Qwen3.5-9B (27 records) | DeepSeek-R1-Distill-Qwen-14B (18 records) |
|---|---|---|
| Avg decode tok/s | 109.44 | 108.01 |
| Params | 9B | 14B (Qwen-distilled) |
| Type | Native | R1 distilled |
| VRAM needed | 5-9 GB | 8-14 GB |
| Recommended HW | 12-16GB cards | 16-24GB cards |
Key observation: DeepSeek-R1-Distill-Qwen-14B hits 108 tok/s on 12-16GB cards — distilled model inference speed matches native 9B, but retains R1 reasoning trace capability.
DeepSeek-R1-Distill-Qwen-14B killer feature
R1 distillation core value: cram R1's reasoning capability (long chain-of-thought) into Qwen 14B's size.
| Model | Params | Reasoning capability | Deployment cost |
|---|---|---|---|
| DeepSeek-R1 full | 671B | ⭐⭐⭐⭐⭐ | ❌ needs H100 cluster |
| DeepSeek-R1-Distill-Qwen-32B | 32B | ⭐⭐⭐⭐ | 32GB card |
| DeepSeek-R1-Distill-Qwen-14B | 14B | ⭐⭐⭐⭐ | 12-16GB card |
| DeepSeek-R1-Distill-Qwen-7B | 7B | ⭐⭐⭐ | 8GB card |
| Qwen3.5-9B | 9B | ⭐⭐⭐ | 12GB card |
14B distilled = "run R1 reasoning on a 16GB card" sweet spot.
9B model horizontal review (10-13 batch)
| Model | Params | Avg tok/s | Feature |
|---|---|---|---|
| Qwen3.8-9B | 9B | 119.08 | Latest 9B |
| Qwen3.5-9B | 9B | 109.44 | Stable 9B |
| Qwen3-8B | 8B | 135.79 | Classic 8B |
| gemma-4-12B-it | 12B | 101.71 | Google 9B |
| DeepSeek-R1-Distill-Qwen-14B | 14B | 108.01 | R1 reasoning |
Qwen3.5-9B is 8% slower than Qwen3.8-9B, but ecosystem is more stable (more quant options).
Build recommendation (12-16GB card fans)
| Budget | Setup | Model | tok/s | Reasoning |
|---|---|---|---|---|
| ¥2500 | RTX 3060 12GB | Qwen3.5-9B | 60-70 | ⭐⭐⭐ |
| ¥2500 | RTX 3060 12GB | DeepSeek-R1-Distill-Qwen-14B Q4_K_M | 55-65 | ⭐⭐⭐⭐ |
| ¥4000 | RTX 4060 Ti 16GB | Qwen3.5-9B | 90-100 | ⭐⭐⭐ |
| ¥4000 | RTX 4060 Ti 16GB | DeepSeek-R1-Distill-Qwen-14B Q5_K_M | 80-95 | ⭐⭐⭐⭐ |
| ¥6000 | RTX 5060 Ti 16GB | Qwen3.5-9B | 130+ | ⭐⭐⭐ |
If you want "reasoning capability + 16GB card": DeepSeek-R1-Distill-Qwen-14B is the only choice in ¥2500-6000 range.
Real use cases for R1 distilled
| Scenario | 14B distilled capable? |
|---|---|
| Math reasoning (MATH / GSM8K) | ✅ strong (retains R1 CoT) |
| Code generation (with reasoning steps) | ✅ strong |
| Scientific QA | ✅ strong |
| Translation | ⚠️ average (not R1's strength) |
| Long-form generation | ⚠️ average |
| Casual chat | ⚠️ average |
14B distilled wins on "tasks needing long chain-of-thought" — complements Qwen3.5-9B's "general chat".
Evidence
- 109.44 / 108.01 averages: DB 27 + 18 records
- "R1 distilled retains CoT": DeepSeek official paper + actual generation style
- "Qwen3.5-9B is 8% slower than 3.8-9B": 109.44 / 119.08 = 0.92
- "12-16GB cards are sweet spot": VRAM 8-14GB estimate
Sources
- 27 Qwen3.5-9B records: DB
benchmark_record, model_id=49 - 18 DeepSeek-R1-Distill-Qwen-14B records: DB benchmark_record, model_id=71
- R1 distillation technique: DeepSeek official R1 paper