NVIDIA ships two Nemotron 3.5 models in one day: the 120B-A12B Super VL and a 6GB edge model, Bolt

On Oct 23 NVIDIA released two Nemotron 3.5 models on the same day — one huge, one tiny: a 120B vision-language model that handles video, tool use and a 1M context, and a 6GB edge model aimed at on-device voice assistants.

Super-VL-120B-A12B-NVFP4

A 120B total / 12B active vision-language MoE, released this time in NVFP4 (quantization-aware distillation from the BF16 build). The architecture interleaves Mamba-2 and Transformer attention layers in a hybrid MoE, with a C-RADIO vision encoder and a vision-language connector; context runs to 1,000,000 tokens.

Engineering details worth noting:

• Native MTP speculative decoding: an official LiLiCorr drafter, seven speculative tokens per step; on the vLLM side via --speculative_config.method dflash
• Switchable reasoning tier: off / on (medium) / on, controlled through the chat template; medium saves tokens but can cost accuracy
• Deployment targets: 1x DGX Spark (GB10), 1x RTX PRO 6000, 1x H200, 2x H100, 1x GB200; the official vLLM command line also carries --mamba-backend flashinfer, --video-pruning-rate 0.5 and --media-io-kwargs with video fps 2 / 256 frames
• Weights measure 75.0GB (88 safetensors shards added up on the HF repo)

Quantization costs almost nothing (NVFP4 / BF16):

Benchmark	NVFP4	BF16
GPQA-Diamond	89.90	89.52
Terminal-Bench 2.1	62.78	60.67
SWE-bench Verified	71.07	71.10
SWE-bench Pro	51.30	51.78
tau2-bench retail	69.30	69.24
tau2-bench airline	75.75	76.50
Video-MME v2	32.54	32.85
OSWorld-Verified	52.3	51.14
AA-Omniscience index	5.3	4.03
GDPval-AA v2	1206	1211

The license is OpenMDW-1.1, commercial use permitted.

On-site status: NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4 (id 96) is cataloged with zero on-site runs — for the same two-ledger reason as the earlier RTX Spark story: physically, 75GB of NVFP4 weights fit a 96GB RTX PRO 6000, but the site runs a deliberately padded accounting where 120B at NVFP4 books above 180GB, so 96GB-class cards get no fit verdict. If you want to run it, follow NVIDIA's deployment matrix.

Bolt-E4B-4GB: a 6.18GB edge model

The other direction: NVIDIA-Nemotron-3.5-Bolt-E4B-4GB is built on Google's Gemma-4-E4B-it (Apache-2.0) and post-trained by NVIDIA on nemotron-post-training-v3, with 8B total / 4B effective parameters and a May 2026 post-training cutoff.

• The Q4_K_M GGUF measures 6.18GB (the "4GB" in the repo name refers to the 4B effective-parameter class, not the file size)
• Official llama.cpp recipe: ghcr.io/ggml-org/llama.cpp:server-cuda13-b11459 with -c 65536 and --jinja, plus a DFlash draft model for speculative decoding (--spec-type draft-dflash --spec-draft-n-max 7)
• 64K context in and out, 36 languages
• The chat template controls thinking mode and conversational mode, with a tunable thinking budget
• Positioning is explicit: local voice and device assistants, interactive personas, embedded edge

Official Q4_K_M benchmarks (against Gemma-4-E4B-it and Qwen3.5-4B at Q4_K_M):

Benchmark	Bolt-E4B	Gemma-4-E4B-it	Qwen3.5-4B
MultiChallenge	55.60	39.36	52.23
RoleBench	71.21	60.51	52.35
RoleMRC	79.12	75.24	62.61
ToolSandbox	74.13	68.31	74.51
IHEval	82.40	66.61	62.87
Tau2-Bench	52.32	38.72	61.74
BFCL-v3	67.31	65.03	70.56
RULER (64K)	85.76	86.14	95.37
LongMemEval	69.92	67.95	76.60

The clearest gains are in tool use and role play (ToolSandbox, IHEval, RoleBench); on long documents and long context (RULER / LongMemEval / Tau2-Bench) Qwen3.5-4B still leads — if your local workload is long-document work, Bolt is not the best pick.

On-site coverage

• NVIDIA-Nemotron-3.5-Bolt-E4B-4GB (id 97) is cataloged with 26 new L2 records (Q4_K_M GGUF + BF16), averaging 142.3 tok/s across the pool:

Hardware	Framework / quant	Measured
H100	Ollama Q4_K_M	345.3 tok/s
RTX 5090	Ollama Q4_K_M / vLLM BF16	279.8 / 171.5 tok/s
RTX PRO 6000 96GB	Ollama Q4_K_M / vLLM BF16	264.8 / 177.6 tok/s
RTX 4090	llama.cpp Q4_K_M	160.6 tok/s (DFlash acceptance 0.54)
M2 Ultra 80GB	llama.cpp Q4_K_M	119.5 tok/s (acceptance 0.66)
RTX Spark (116 GiB Linux pool)	llama.cpp / Ollama Q4_K_M	80.7 / 79.2 tok/s

• The batch adds 103 runs in total, also closing hardware gaps for three older models: gemma-4-E4B-it (23 runs on hardware it had never touched — 4090, 3090, 7900 XTX, 5080, 4080S, 3060, H100, H20, RTX PRO 6000, M5 Ultra, Arc B580, RTX Spark), Ministral-3-8B-Instruct-2512 (28 runs) and iFlytek Spark-X2.5-4B (26 runs).