Arao AI
Back to search results

DeepSeek-V4-Flash · UD-IQ2_XXS

4x NVIDIA RTX 309096GBllama.cpp club3090-moecache-e0a2089L0 Self-reported

This page aggregates 1 real-world runs of DeepSeek-V4-Flash (UD-IQ2_XXS) on 4x NVIDIA RTX 3090 with llama.cpp, contributed by 1 independent source platforms; metrics are averages of published measurements.

35.68 tok/s

Decode

Decode speed

420.73 tok/s

Prefill

Prefill speed

0.11 s

TTFT

Time to first token

87.16 GB

VRAM

VRAM usage

L0 Self-reported

1

Measured runs

1

Independent sources

GitHub

Source platforms

Today

Last verified

Performance

  1. llama.cpp · UD-IQ2_XXS (current)Decode 35.68 · Prefill 420.73 ·

Core figures

Decode (avg)
35.68 tok/s
Prefill (avg)
420.73 tok/s
TTFT (avg)
0.11 s
VRAM (avg)
87.16 GB
MTP acceptance rate
—
TTFB
— GB
Power draw
591.93 W

Configuration

Member-level fields are taken from the most recent run

Model
DeepSeek-V4-Flash
Quantization
UD-IQ2_XXS
Framework
llama.cpp
Version
club3090-moecache-e0a2089
Context length
204800 tokens
Batch size
2048
GPU layers
99

Hardware

Nominal and measured figures are shown side by side; whether it runs is the reader's call

GPU
4x NVIDIA RTX 3090
Nominal VRAM
96 GB
Measured VRAM (avg)
87.16 GB
OS
Ubuntu 24.04.4 LTS
Driver
595.84
CUDA
13.2
Power draw
591.93 W

Get started

Startup command

Command from the latest benchmark run
model=/models/deepseek-v4-flash-0731-gguf/UD-IQ2_XXS/DeepSeek-V4-Flash-0731-UD-IQ2_XXS-00001-of-00003.gguf offload=blk\.[0-9]+\.ffn_(gate|up|down)_exps\.weight=CPU threads=32 ubatch=2048 kv=q8_0 ngl=99 split=1,1,1,1

Sources & evidence

1 measured records in total, each traceable to its original source

  1. L0 Self-reportedGitHubOriginal link Verified on 2026-09-20

    club3090-moecache-e0a2089 · Ubuntu 24.04.4 LTS · CUDA 13.2 · 204800 ctx

    35.68 tok/s

    Decode

    420.73 tok/s

    Prefill

    0.11 s

    TTFT

    87.16 GB

    VRAM

    —

    MTP

    591.93 W

    Power draw

    [bench] narrative run 1/5: 1000 tok in 29.4s (34.11 decode TPS, ttft 121ms) [bench] narrative run 2/5: 1000 tok in 29.1s (34.46 decode TPS, ttft 109ms) [bench] narrative run 3/5: 1000 tok in 27.5s (36.46 decode TPS, ttft 113ms) [bench] narrative run 4/5: 1000 tok in 27.3s (36.81 decode TPS, ttft 107ms) [bench] narrative run 5/5: 1000 tok in 27.4s (36.58 decode TPS, ttft 105ms)