NVIDIA RTX 3070

GPU

Ampere; 8GB VRAM, 448 GB/s bandwidth, 220W TDP, CUDA cores 5,888. Release date: 2020-10-29. Source: LCZ hardware catalog, with missing fields supplemented from NVIDIA official materials.

2

Run records

1

Models covered

1

Frameworks covered

L0 Self-reported × 2

Evidence mix

Typical measured performance

Published benchmark averages per model × framework × quantization on this hardware.

  1. llama.cpp · Q5_K_M Decode 116.5 · Prefill 873.8 ·
  2. llama.cpp · Q4_K_M Decode 145.6 · Prefill 1092 ·

Fit check

Physics-based estimate from VRAM and memory bandwidth: pick a model to see which quantizations fit and the expected speed

Formula: tok/s = (bandwidth ÷ model size) × 0.55, adapted from llmfit; see run records for measured numbers

Key specs

Vendor
NVIDIA
Architecture
Ampere
VRAM
8 GB
Memory bandwidth
448 GB/s
TDP
220 W
Release date
2020-10-29

Run records

2 configurations

ModelFrameworkQuantizationDecodeGen timeSamples
Qwen3.5-4Bllama.cppQ5_K_M116.5 tok/s—1
Qwen3.5-4Bllama.cppQ4_K_M145.6 tok/s—1