2x NVIDIA RTX 5070 Ti

GPU

8

Run records

2

Models covered

2

Frameworks covered

L0 Self-reported × 8

Evidence mix

Typical measured performance

Published benchmark averages per model × framework × quantization on this hardware.

No published records for this metric under the same model + hardware yet

Fit check

Physics-based estimate from VRAM and memory bandwidth: pick a model to see which quantizations fit and the expected speed

Formula: tok/s = (bandwidth ÷ model size) × 0.55, adapted from llmfit; see run records for measured numbers

Key specs

Vendor
NVIDIA
VRAM
32 GB
Memory bandwidth
1792 GB/s

Run records

8 configurations

ModelFrameworkQuantizationDecodeGen timeSamples
GLM-5.3SGLangIQ2_XS——1
GLM-5.3SGLangIQ3_S——1
GLM-5.3vLLMIQ2_XS——1
GLM-5.3vLLMIQ3_S——1
DeepSeek-V4.1-FlashSGLangIQ2_XS——1
DeepSeek-V4.1-FlashSGLangIQ3_S——1
DeepSeek-V4.1-FlashvLLMIQ2_XS——1
DeepSeek-V4.1-FlashvLLMIQ3_S——1