NVIDIA-Nemotron-3.5-Bolt-E4B-4GB

NVIDIA's Nemotron 3.5 Bolt, released 2026-10-23 (Google Gemma-4-E4B-it on Apache-2.0, post-trained by NVIDIA, post-training data cutoff May 2026): 8B total parameters with 4B effective, aimed at local voice and device assistants, interactive personas and embedded edge interfaces. Its focus is multi-turn tool use, instruction following and input grounding; the chat template controls thinking mode and conversational mode, with a tunable thinking budget. The official Q4_K_M GGUF measures **6.18GB** (the "4GB" in the repo name refers to the 4B effective-parameter class), and NVIDIA ships a llama.cpp Docker recipe (ghcr.io/ggml-org/llama.cpp:server-cuda13-b11459, -c 65536, --jinja) plus a DFlash draft model for speculative decoding (--spec-type draft-dflash --spec-draft-n-max 7). Context is 64K tokens in and out, across 36 languages. Q4_K_M benchmarks (against Gemma-4-E4B-it Q4_K_M / Qwen3.5-4B Q4_K_M): MultiChallenge 55.60, RoleBench 71.21, RoleMRC 79.12, ToolSandbox 74.13, IHEval 82.40, IFBench 64.00, StructEval-T 79.99, BFCL-v3 67.31, Tau2-Bench 52.32, AgentIF 22.95, RULER (64K) 85.76, LongMemEval 69.92, LongBench v2 (48K) 48.67 — the clearest gains over the base model are in tool use and role play. License OpenMDW-1.1 (the Gemma 4 base remains under Apache-2.0).