NVIDIA RTX 5070 Ti
GPUBlackwell 架构(GB203),16GB GDDR7 / 256-bit 位宽,显存带宽 896 GB/s,整卡功耗 300W。与 RTX 5080 同用 GB203 芯片但核心数更少。
111
运行记录
25
覆盖模型
8
覆盖框架
L0 自报 × 86 · L2 跨框架验证 × 25
证据分布
典型性能表现
同硬件上各模型 × 框架 × 量化的已上架实测均值。
- llama.cpp · IQ1_S Decode 182.74 · Prefill 910.54 ·
- vLLM · IQ4_XS Decode 131.32 · Prefill 455.29 ·
- vLLM · Q4_K_M Decode 116.07 · Prefill 492.31 ·
- vLLM · FP16 Decode 121.99 · Prefill 561.36 ·
- vLLM · Q6_K Decode 174.9 · Prefill 685.31 ·
- Strata · Q5_K_M Decode 251.18 · Prefill 1222.13 ·
- Strata · Q6_K Decode 237.15 · Prefill 1031.12 ·
- vLLM · IQ3_S Decode 103.59 · Prefill 387.95 ·
- vLLM · Q3_K_M Decode 98.53 · Prefill 475.97 ·
- Ollama · IQ3_XS Decode 89.42 · Prefill 466.92 ·
- Ollama · Q2_K Decode 98.9 · Prefill 326.08 ·
- llama.cpp · Q3_K_M Decode 110.54 · Prefill 601.05 ·
- Ollama · Q4_K_S Decode 82.54 · Prefill 372.44 ·
- Ollama · IQ3_XS Decode 80.61 · Prefill 400.97 ·
- llama.cpp · Q4_K_S Decode 108.27 · Prefill 466.84 ·
- vLLM · IQ3_S Decode 128.33 · Prefill 495.87 ·
- vLLM · Q2_K Decode 134.24 · Prefill 721.19 ·
- vLLM · Q2_K Decode 145.89 · Prefill 573.9 ·
- Strata · IQ3_S Decode 180.15 · Prefill 920.56 ·
- Ollama · Q4_K_S Decode 228.86 · Prefill 764.69 ·
- Strata · Q4_K_M Decode 361.74 · Prefill 1475.89 ·
- vLLM · IQ1_S Decode 131.31 · Prefill 466.87 ·
- vLLM · IQ3_S Decode 95.38 · Prefill 330.41 ·
- Strata · IQ3_S Decode 132.6 · Prefill 715.44 ·
- Strata · Q3_K_S Decode 129.13 · Prefill 489.23 ·
- llama.cpp · Q6_K Decode 22.1 · Prefill 165.8 ·
- llama.cpp · Q5_K_S Decode 28.8 · Prefill 216 ·
- llama.cpp · IQ3_S Decode 47.9 · Prefill 359.3 ·
- llama.cpp · IQ4_XS Decode 36 · Prefill 270 ·
- SGLang · Q4_K_M Decode 3.8 · Prefill 28.5 ·
- SGLang · IQ3_S Decode 5.7 · Prefill 42.8 ·
- SGLang · IQ4_XS Decode 4.3 · Prefill 32.3 ·
- vLLM · Q4_K_M Decode 3.9 · Prefill 29.3 ·
- vLLM · IQ3_S Decode 5.9 · Prefill 44.3 ·
- vLLM · IQ4_XS Decode 4.4 · Prefill 33 ·
- vLLM · Q4_K_M Decode 4.4 · Prefill 33 ·
- vLLM · IQ3_S Decode 6.6 · Prefill 49.5 ·
- vLLM · IQ4_XS Decode 5 · Prefill 37.5 ·
- llama.cpp · Q4_K_M Decode 4.1 · Prefill 30.7 ·
- llama.cpp · IQ3_S Decode 6.1 · Prefill 45.8 ·
- llama.cpp · IQ4_XS Decode 4.6 · Prefill 34.5 ·
- Strata · Q4_K_M Decode 8.2 · Prefill 61.5 ·
- Strata · IQ3_XXS Decode 14.2 · Prefill 106.5 ·
- Strata · IQ3_S Decode 12.3 · Prefill 92.3 ·
- Strata · IQ4_XS Decode 9.2 · Prefill 69 ·
- llama.cpp · Q4_K_M Decode 6.4 · Prefill 48 ·
- llama.cpp · IQ3_XXS Decode 11.1 · Prefill 83.3 ·
- llama.cpp · IQ4_XS Decode 7.2 · Prefill 54 ·
- Strata · Q5_K_S Decode 49.2 · Prefill 369 ·
- Strata · IQ3_S Decode 82.2 · Prefill 616.5 ·
Fit 判定
按显存与内存带宽做物理估算:选择模型,查看各量化能否装下及预计速度
估算公式:tok/s =(带宽 ÷ 模型占用)× 0.55,借鉴 llmfit;实测以运行记录为准
核心规格
- 厂商
- NVIDIA
- 架构
- Blackwell
- 显存
- 16 GB
- 显存带宽
- 896 GB/s
- TDP 功耗
- 300 W
- 发布日期
- 2025-02-20
运行记录
共 50 组配置
| 模型 | 框架 | 量化 | Decode | 生成耗时 | 样本数 |
|---|---|---|---|---|---|
| Qwen3.5-4B | llama.cpp | IQ1_S | 182.7 tok/s | — | 1 |
| Qwen3.5-4B | vLLM | IQ4_XS | 131.3 tok/s | — | 1 |
| Qwen3.5-4B | vLLM | Q4_K_M | 116.1 tok/s | — | 1 |
| Qwen3.5-0.8B | vLLM | FP16 | 122 tok/s | — | 1 |
| Qwen3.5-0.8B | vLLM | Q6_K | 174.9 tok/s | — | 1 |
| Qwen3.5-0.8B | Strata | Q5_K_M | 251.2 tok/s | — | 1 |
| Qwen3.5-0.8B | Strata | Q6_K | 237.2 tok/s | — | 1 |
| Qwen3.8-9B | vLLM | IQ3_S | 103.6 tok/s | — | 1 |
| Qwen3.8-9B | vLLM | Q3_K_M | 98.5 tok/s | — | 1 |
| Qwen3.8-9B | Ollama | IQ3_XS | 89.4 tok/s | — | 1 |
| Qwen3.8-9B | Ollama | Q2_K | 98.9 tok/s | — | 1 |
| Qwen3-8B | llama.cpp | Q3_K_M | 110.5 tok/s | — | 1 |
| Qwen3-8B | Ollama | Q4_K_S | 82.5 tok/s | — | 1 |
| gemma-4-12B-it | Ollama | IQ3_XS | 80.6 tok/s | — | 1 |
| Llama-3.1-8B-Instruct | llama.cpp | Q4_K_S | 108.3 tok/s | — | 1 |
| Llama-3.1-8B-Instruct | vLLM | IQ3_S | 128.3 tok/s | — | 1 |
| Llama-3.1-8B-Instruct | vLLM | Q2_K | 134.2 tok/s | — | 1 |
| Llama-3-8B-Instruct | vLLM | Q2_K | 145.9 tok/s | — | 1 |
| Llama-3-8B-Instruct | Strata | IQ3_S | 180.2 tok/s | — | 1 |
| Llama-3.2-3B-Instruct | Ollama | Q4_K_S | 228.9 tok/s | — | 1 |
| Llama-3.2-3B-Instruct | Strata | Q4_K_M | 361.7 tok/s | — | 1 |
| Kimi-K3 | vLLM | IQ1_S | 131.3 tok/s | — | 1 |
| Kimi-K3 | vLLM | IQ3_S | 95.4 tok/s | — | 1 |
| Kimi-K3 | Strata | IQ3_S | 132.6 tok/s | — | 1 |
| Kimi-K3 | Strata | Q3_K_S | 129.1 tok/s | — | 1 |
| Qwen3.5-35B-A3B | llama.cpp | Q6_K | 22.1 tok/s | — | 1 |
| Qwen3.5-35B-A3B | llama.cpp | Q5_K_S | 28.8 tok/s | — | 1 |
| Qwen3.5-35B-A3B | llama.cpp | IQ3_S | 47.9 tok/s | — | 1 |
| Qwen3.5-35B-A3B | llama.cpp | IQ4_XS | 36 tok/s | — | 1 |
| GLM-5.3-Flash | SGLang | Q4_K_M | 3.8 tok/s | — | 1 |
| GLM-5.3-Flash | SGLang | IQ3_S | 5.7 tok/s | — | 1 |
| GLM-5.3-Flash | SGLang | IQ4_XS | 4.3 tok/s | — | 1 |
| GLM-5.3-Flash | vLLM | Q4_K_M | 3.9 tok/s | — | 1 |
| GLM-5.3-Flash | vLLM | IQ3_S | 5.9 tok/s | — | 1 |
| GLM-5.3-Flash | vLLM | IQ4_XS | 4.4 tok/s | — | 1 |
| DeepSeek-V4-Flash | vLLM | Q4_K_M | 4.4 tok/s | — | 1 |
| DeepSeek-V4-Flash | vLLM | IQ3_S | 6.6 tok/s | — | 1 |
| DeepSeek-V4-Flash | vLLM | IQ4_XS | 5 tok/s | — | 1 |
| DeepSeek-V4-Flash | llama.cpp | Q4_K_M | 4.1 tok/s | — | 1 |
| DeepSeek-V4-Flash | llama.cpp | IQ3_S | 6.1 tok/s | — | 1 |
| DeepSeek-V4-Flash | llama.cpp | IQ4_XS | 4.6 tok/s | — | 1 |
| Qwen3.8-Flash-Next | Strata | Q4_K_M | 8.2 tok/s | — | 1 |
| Qwen3.8-Flash-Next | Strata | IQ3_XXS | 14.2 tok/s | — | 1 |
| Qwen3.8-Flash-Next | Strata | IQ3_S | 12.3 tok/s | — | 1 |
| Qwen3.8-Flash-Next | Strata | IQ4_XS | 9.2 tok/s | — | 1 |
| Qwen3.8-Flash-Next | llama.cpp | Q4_K_M | 6.4 tok/s | — | 1 |
| Qwen3.8-Flash-Next | llama.cpp | IQ3_XXS | 11.1 tok/s | — | 1 |
| Qwen3.8-Flash-Next | llama.cpp | IQ4_XS | 7.2 tok/s | — | 1 |
| Qwen3.8-27B | Strata | Q5_K_S | 49.2 tok/s | — | 1 |
| Qwen3.8-27B | Strata | IQ3_S | 82.2 tok/s | — | 1 |