AMD Radeon AI PRO R9700 32GB workstation first measurements: 33 records avg 210 tok/s

10-13 batch has 33 AMD Radeon AI PRO R9700 (32GB GDDR6) L2 records — AMD workstation card first time in DB.

Overall recap

Metric	Value
Records	33
Avg decode tok/s	210.14
Peak decode tok/s	394.48
Avg VRAM used	9.46 GB (30% of 32GB)
Model range	0.5B → 31B

Key observation: average VRAM 9.46GB = 30% of 32GB — R9700 has 70% VRAM headroom for 27B and below models. Plenty of room for future 70B quantized.

R9700 vs same-price / same-tier N-cards

Hardware	VRAM	Records	Avg tok/s	Price tier
RTX 5090	32GB	35	270.04	¥14000
AMD R9700	32GB	33	210.14	¥9000 (est.)
AMD RX 7900 XTX	24GB	21	147.03	¥6500
RTX 4080	16GB	23	202.72	¥8500
RTX 4090	24GB	23	174.22	¥12000

R9700 is 22% slower than 5090, 4% faster than 4080, 43% faster than 7900 XTX — AMD workstation cards now rival N-card mid-to-high tier on inference.

R9700 5 most dramatic single records

Model	Quant	tok/s	VRAM
Qwen3.5-0.8B	IQ2_XS	394.48	2.91 GB
Qwen3.5-0.8B	Q2_K	380+	3.0 GB
Qwen2.5-0.5B-Instruct	Q5_K_M	384.75	3.08 GB
Qwen3-1.7B	Q4_K_M	280+	5.0 GB
Qwen3.5-27B	Q4_K_M	95	22 GB

Qwen3.5-0.8B IQ2_XS + R9700 + llama.cpp = 394.48 tok/s — one of the all-time fastest single records in DB.

Key insight: AMD inference ecosystem catching up

Dimension	AMD R9700	RTX 5090
Inference speed	210 tok/s	270 tok/s
ROCm compatibility	✅ 6.4+ full support	N/A
llama.cpp support	✅ Vulkan + ROCm	✅ CUDA
Strata support	✅ from 0.5.3	✅
vLLM support	✅ ROCm	✅ CUDA
Price	¥9000 (est.)	¥14000

AMD in 2026 has inference ecosystem that can compete with N-cards — 3 years ago the 7900 XTX couldn't make this comparison.

Build recommendation

Budget	Setup	27B tok/s	Recommendation
¥9000	R9700 + 64GB DDR5 + NVMe 4.0	95	⭐⭐⭐⭐ value
¥12000	RTX 4090 + 64GB DDR5 + NVMe 4.0	91	⭐⭐⭐ classic
¥14000	RTX 5090 + 64GB DDR5 + NVMe 4.0	175	⭐⭐⭐⭐⭐ top

R9700 is the only ¥9000-tier card that runs 27B at full speed + has 32GB VRAM headroom.

Evidence

• 210.14 average: DB 33 records, hardware_id=79
• 394.48 peak: DB record id=1450
• 9.46GB VRAM average: DB actual vram_gb field
• ¥9000 price: AMD official 2025-09 pricing + domestic discounts
• ROCm 6.4+: AMD official docs

Sources

• 33 R9700 records: DB benchmark_record table, hardware_id=79
• Peak single: DB record id=1450 (Qwen3.5-0.8B + IQ2_XS + llama.cpp)