NVIDIA RTX 3090 Ti
GPUNVIDIA Ampere (GA102): 24 GB GDDR6X on a 384-bit bus, 1008 GB/s of memory bandwidth, 450 W board power. Memory bandwidth matches the RTX 4090's, at higher power and lower compute.
5
Run records
2
Models covered
2
Frameworks covered
L0 Self-reported × 5
Evidence mix
Typical measured performance
Published benchmark averages per model × framework × quantization on this hardware.
- llama.cpp · IQ2_XS Decode 3.1 · Prefill 23.3 ·
- llama.cpp · IQ3_S Decode 2.5 · Prefill 18.8 ·
- vLLM · IQ2_XS Decode 3.3 · Prefill 24.8 ·
- vLLM · IQ3_S Decode 2.8 · Prefill 21 ·
- llama.cpp · Q5_K_S Decode 50.84 · Prefill 1376.9 ·
Fit check
Physics-based estimate from VRAM and memory bandwidth: pick a model to see which quantizations fit and the expected speed
Formula: tok/s = (bandwidth ÷ model size) × 0.55, adapted from llmfit; see run records for measured numbers
Key specs
- Vendor
- NVIDIA
- Architecture
- Ampere
- VRAM
- 24 GB
- Memory bandwidth
- 1008 GB/s
- TDP
- 450 W
- Release date
- 2022-03-29
Run records
5 configurations
| Model | Framework | Quantization | Decode | Gen time | Samples |
|---|---|---|---|---|---|
| DeepSeek-V4.1-Flash | llama.cpp | IQ2_XS | 3.1 tok/s | — | 1 |
| DeepSeek-V4.1-Flash | llama.cpp | IQ3_S | 2.5 tok/s | — | 1 |
| DeepSeek-V4.1-Flash | vLLM | IQ2_XS | 3.3 tok/s | — | 1 |
| DeepSeek-V4.1-Flash | vLLM | IQ3_S | 2.8 tok/s | — | 1 |
| Qwen3.6-27B | llama.cpp | Q5_K_S | 50.8 tok/s | — | 1 |