NVIDIA RTX 4080 Super
GPUAda Lovelace 架构(AD103),16GB GDDR6X / 256-bit 位宽,显存带宽 736 GB/s,整卡功耗 320W。
80
运行记录
20
覆盖模型
5
覆盖框架
L0 自报 × 30 · L2 跨框架验证 × 50
证据分布
典型性能表现
同硬件上各模型 × 框架 × 量化的已上架实测均值。
- Ollama · Q2_K Decode 222.5 · Prefill 1136.51 ·
- vLLM · IQ3_XS Decode 194.23 · Prefill 865.43 ·
- Ollama · Q3_K_S Decode 155.17 · Prefill 518.65 ·
- llama.cpp · Q8_0 Decode 237.08 · Prefill 981.54 ·
- Strata · Q4_K_M Decode 305.26 · Prefill 1292.84 ·
- llama.cpp · Q4_K_M Decode 162.68 · Prefill 521.64 ·
- Ollama · Q3_K_L Decode 145.8 · Prefill 516.93 ·
- Strata · IQ3_XS Decode 244.72 · Prefill 1076.88 ·
- vLLM · Q4_K_S Decode 190.65 · Prefill 787.91 ·
- Ollama · IQ2_M Decode 242.65 · Prefill 957.57 ·
- Ollama · Q2_K Decode 231.86 · Prefill 790.98 ·
- vLLM · Q3_K_S Decode 302.38 · Prefill 993.94 ·
- vLLM · Q4_K_S Decode 370.71 · Prefill 1429.25 ·
- Ollama · IQ3_S Decode 352.12 · Prefill 1849.99 ·
- Ollama · IQ3_XS Decode 393.56 · Prefill 2131.15 ·
- vLLM · IQ2_M Decode 181.18 · Prefill 718.16 ·
- Strata · Q3_K_M Decode 213.25 · Prefill 900.93 ·
- Strata · Q3_K_S Decode 210.74 · Prefill 935.82 ·
- llama.cpp · Q3_K_S Decode 168.36 · Prefill 655.5 ·
- vLLM · IQ2_M Decode 214.44 · Prefill 870.17 ·
- llama.cpp · Q3_K_M Decode 201.4 · Prefill 1061.69 ·
- Strata · IQ2_XS Decode 307.75 · Prefill 1104.97 ·
- Strata · Q5_K_M Decode 179.72 · Prefill 584.63 ·
- Ollama · IQ3_XS Decode 190.92 · Prefill 657.75 ·
- Strata · IQ2_XS Decode 334.52 · Prefill 1091.81 ·
- llama.cpp · IQ2_XS Decode 372.98 · Prefill 1597.99 ·
- llama.cpp · IQ4_XS Decode 289.96 · Prefill 1071.96 ·
- Strata · FP16 Decode 277.08 · Prefill 1141.1 ·
- llama.cpp · IQ1_S Decode 208.08 · Prefill 731.51 ·
- vLLM · Q5_K_M Decode 152.06 · Prefill 772.68 ·
- Ollama · Q3_K_L Decode 202.33 · Prefill 955.67 ·
- llama.cpp · Q4_K_M Decode 344.73 · Prefill 1737.55 ·
- vLLM · IQ1_S Decode 314.79 · Prefill 1409.84 ·
- llama.cpp · IQ1_S Decode 204.16 · Prefill 1111.21 ·
- Ollama · IQ2_XS Decode 174.28 · Prefill 597.11 ·
- Ollama · Q3_K_M Decode 141.22 · Prefill 455.03 ·
- llama.cpp · IQ2_XS Decode 195.22 · Prefill 907.33 ·
- Ollama · Q3_K_S Decode 129.2 · Prefill 551.46 ·
- vLLM · IQ2_M Decode 224.52 · Prefill 1147.46 ·
- vLLM · Q3_K_L Decode 177.01 · Prefill 908.47 ·
- Ollama · IQ2_M Decode 204.79 · Prefill 1109.54 ·
- vLLM · Q3_K_S Decode 189.4 · Prefill 632.63 ·
- Ollama · IQ1_S Decode 212.31 · Prefill 982.11 ·
- vLLM · IQ3_S Decode 334.5 · Prefill 1159.32 ·
- Strata · Q3_K_L Decode 358.87 · Prefill 1331.79 ·
- Strata · IQ2_M Decode 252.15 · Prefill 1022.42 ·
- llama.cpp · IQ2_XS Decode 184.95 · Prefill 620.72 ·
- Strata · IQ1_S Decode 255.29 · Prefill 866.27 ·
- llama.cpp · Q6_K Decode 18.2 · Prefill 136.5 ·
- llama.cpp · Q5_K_S Decode 23.6 · Prefill 177 ·
Fit 判定
按显存与内存带宽做物理估算:选择模型,查看各量化能否装下及预计速度
估算公式:tok/s =(带宽 ÷ 模型占用)× 0.55,借鉴 llmfit;实测以运行记录为准
核心规格
- 厂商
- NVIDIA
- 架构
- Ada Lovelace
- 显存
- 16 GB
- 显存带宽
- 736 GB/s
- TDP 功耗
- 320 W
- 发布日期
- 2024-01-31
运行记录
共 50 组配置
| 模型 | 框架 | 量化 | Decode | 生成耗时 | 样本数 |
|---|---|---|---|---|---|
| gemma-4-E4B-it | Ollama | Q2_K | 222.5 tok/s | — | 1 |
| Ministral-3-8B-Instruct-2512 | vLLM | IQ3_XS | 194.2 tok/s | — | 1 |
| Ministral-3-8B-Instruct-2512 | Ollama | Q3_K_S | 155.2 tok/s | — | 1 |
| iFlytek Spark-X2.5-4B | llama.cpp | Q8_0 | 237.1 tok/s | — | 1 |
| iFlytek Spark-X2.5-4B | Strata | Q4_K_M | 305.3 tok/s | — | 1 |
| NVIDIA-Nemotron-3.5-Bolt-E4B-4GB | llama.cpp | Q4_K_M | 162.7 tok/s | — | 1 |
| elephant-1 | Ollama | Q3_K_L | 145.8 tok/s | — | 1 |
| elephant-1 | Strata | IQ3_XS | 244.7 tok/s | — | 1 |
| Qwen3.5-4B | vLLM | Q4_K_S | 190.7 tok/s | — | 1 |
| Qwen3.5-4B | Ollama | IQ2_M | 242.7 tok/s | — | 1 |
| Qwen3.5-4B | Ollama | Q2_K | 231.9 tok/s | — | 1 |
| Qwen3.5-0.8B | vLLM | Q3_K_S | 302.4 tok/s | — | 1 |
| Qwen3.5-0.8B | vLLM | Q4_K_S | 370.7 tok/s | — | 1 |
| Qwen3.5-0.8B | Ollama | IQ3_S | 352.1 tok/s | — | 1 |
| Qwen3.5-0.8B | Ollama | IQ3_XS | 393.6 tok/s | — | 1 |
| Qwen3.8-9B | vLLM | IQ2_M | 181.2 tok/s | — | 1 |
| Qwen3.8-9B | Strata | Q3_K_M | 213.3 tok/s | — | 1 |
| Qwen3.8-9B | Strata | Q3_K_S | 210.7 tok/s | — | 1 |
| Qwen3-8B | llama.cpp | Q3_K_S | 168.4 tok/s | — | 1 |
| Qwen3-8B | vLLM | IQ2_M | 214.4 tok/s | — | 2 |
| Llama-3.1-8B-Instruct | llama.cpp | Q3_K_M | 201.4 tok/s | — | 1 |
| Llama-3.1-8B-Instruct | Strata | IQ2_XS | 307.8 tok/s | — | 1 |
| Llama-3.1-8B-Instruct | Strata | Q5_K_M | 179.7 tok/s | — | 1 |
| Llama-3-8B-Instruct | Ollama | IQ3_XS | 190.9 tok/s | — | 2 |
| Llama-3-8B-Instruct | Strata | IQ2_XS | 334.5 tok/s | — | 1 |
| Llama-3.2-3B-Instruct | llama.cpp | IQ2_XS | 373 tok/s | — | 1 |
| Llama-3.2-3B-Instruct | llama.cpp | IQ4_XS | 290 tok/s | — | 1 |
| Llama-3.2-3B-Instruct | Strata | FP16 | 277.1 tok/s | — | 1 |
| DeepSeek-R1-Distill-Qwen-14B | llama.cpp | IQ1_S | 208.1 tok/s | — | 1 |
| Qwen3.5-4B | vLLM | Q5_K_M | 152.1 tok/s | — | 1 |
| Qwen3.5-4B | Ollama | Q3_K_L | 202.3 tok/s | — | 1 |
| Qwen3.5-0.8B | llama.cpp | Q4_K_M | 344.7 tok/s | — | 1 |
| Qwen3.5-0.8B | vLLM | IQ1_S | 314.8 tok/s | — | 1 |
| Qwen3.8-9B | llama.cpp | IQ1_S | 204.2 tok/s | — | 1 |
| Qwen3.8-9B | Ollama | IQ2_XS | 174.3 tok/s | — | 1 |
| Qwen3.8-9B | Ollama | Q3_K_M | 141.2 tok/s | — | 1 |
| Qwen3-8B | llama.cpp | IQ2_XS | 195.2 tok/s | — | 1 |
| gemma-4-12B-it | Ollama | Q3_K_S | 129.2 tok/s | — | 1 |
| Llama-3.1-8B-Instruct | vLLM | IQ2_M | 224.5 tok/s | — | 1 |
| Llama-3.1-8B-Instruct | vLLM | Q3_K_L | 177 tok/s | — | 1 |
| Llama-3.1-8B-Instruct | Ollama | IQ2_M | 204.8 tok/s | — | 1 |
| Llama-3-8B-Instruct | vLLM | Q3_K_S | 189.4 tok/s | — | 1 |
| Llama-3-8B-Instruct | Ollama | IQ1_S | 212.3 tok/s | — | 1 |
| Llama-3.2-3B-Instruct | vLLM | IQ3_S | 334.5 tok/s | — | 1 |
| Llama-3.2-3B-Instruct | Strata | Q3_K_L | 358.9 tok/s | — | 1 |
| DeepSeek-R1-Distill-Qwen-14B | Strata | IQ2_M | 252.2 tok/s | — | 1 |
| Kimi-K3 | llama.cpp | IQ2_XS | 184.9 tok/s | — | 1 |
| Kimi-K3 | Strata | IQ1_S | 255.3 tok/s | — | 1 |
| Qwen3.5-35B-A3B | llama.cpp | Q6_K | 18.2 tok/s | — | 1 |
| Qwen3.5-35B-A3B | llama.cpp | Q5_K_S | 23.6 tok/s | — | 1 |