KRAFTON Raon-1-mini preview: a 28.8B-A4.3B hybrid-attention MoE that fits a single RTX 5090 at NVFP4
KRAFTON AI published the Raon-1-mini preview on Oct 22 (Apache-2.0, request-to-access with automatic approval) — a from-scratch MoE strong at Korean: 28.8B total / 4.3B active, with 18 of its 25 layers using Raon Delta Attention (linear attention) and 7 using Gated MLA (KV of ~8KiB per token), 262K context. Official NVFP4 runs on a single RTX 5090; the DFlash2 draft model delivers up to 2.18x throughput on a B300. Official GGUF Q4_K_M is 19.7GB. 24 new on-site runs.
Source: araoai
KRAFTON Raon-1-mini preview: a 28.8B-A4.3B hybrid-attention MoE that fits a single RTX 5090 at NVFP4
On Oct 22, KRAFTON AI — the AI arm of the PUBG parent company — published the first model in the open Raon family, the Raon-1-mini preview (Apache-2.0). It is one of very few non-US/China open-weight models trained from scratch: 28.8B total / 4.3B active (official designation 30B-A4B), aimed squarely at Korean conversation and agentic tasks. Korean outlets (Chosunbiz, MoneyToday, Seoul Economic, AsiaE) covered it wall-to-wall for two days.
Status first: this is a preview, not the final model — the official line is that specs and results may change before release. Weight repos are request-to-access with automatic approval.
Hybrid attention: 18 linear layers + 7 MLA
Only 7 of the 25 layers keep a KV cache:
| Component | Configuration |
|---|---|
| Raon Delta Attention | 18 layers, fixed-size state (decouples the erase/write gates of Kimi Delta Attention) |
| Gated MLA | 7 layers, KV of only ~8KiB/token (BF16) |
| Layout | 3:1 interleaved, 40 heads, head dim 128 (RDA) / 192QK-128V (MLA) |
| MoE | 6 of 128 routed experts + 1 shared = 4.3B per token |
| Context | 262,144 tokens |
Linear-attention state does not grow with context and MLA squeezes KV to 8KiB/token — together the 262K-context VRAM bill is almost nothing but weights. This is among the most aggressive hybrid-attention designs in open weights today.
NVFP4: a single RTX 5090
Official quantizations ship as MXFP8 and NVFP4 checkpoints, and NVFP4 puts the whole model on a single RTX 5090 (32GB). A DFlash2 draft model also ships (official B300 numbers, vLLM 0.28.0 / block 8): up to 2.18x output throughput. Tool calling is native, with Hermes Agent, Pi and OpenCode already integrated.
Official GGUFs (KRAFTON repo, Oct 22)
| Quant | File size |
|---|---|
| IQ4_XS | 16.1 GB |
| Q4_K_M | 19.7 GB |
| Q5_K_M | 22.2 GB |
| Q6_K | 25.8 GB |
| Q8_0 | 30.7 GB |
| DFlash2 draft (Q8_0) | 0.7 GB |
Q8_0 at 30.7GB sits right against the 5090's 32GB; Q4_K_M at 19.7GB is the realistic tier for 24GB cards. Same reminder as above: the site's padded VRAM formula judges fit more conservatively — physical GGUF size and the on-site fit verdict are two different ledgers.
On-site coverage
Raon-1-mini-preview (id 93) is cataloged with 24 new L2 records, averaging 136.1 tok/s across the pool:
| Hardware | Quant | Measured |
|---|---|---|
| RTX 5090 | Q2_K | 351.2 tok/s |
| H100 | Q6_K | 343.5 tok/s |
| RTX PRO 6000 96GB | Q4_K_M | 227.9 tok/s |
| M2 Ultra 80GB | Q4_K_M | 110.5 tok/s |
| RTX Spark (116 GiB Linux pool) | Q4_K_M | 45-49 tok/s |
At 4.3B active it sits between A3B and dense 27B, and the measured curve matches what we already know from Qwen3.6-35B-A3B on-site: MoE decode speed is essentially active params times card bandwidth — total params only decide whether it fits.