Baseten's 27B decision model released 2026-10-21 (MIT, a full fine-tune of Qwen3.5-27B with the vision encoder frozen). Give it a state and a set of questions and it returns a calibrated probability for every option from one shared-prefix prefill — cheaper and steadier than asking a general LLM question-by-question for routing, guardrail and classification work. Scores: #1 of 208 models on the Jev Decision Index 0.4 (71.24, ahead of Perplexity Decider v1.2 at 69.42) and #1 on a replica of the Artificial Analysis Decision Index (65.9). Full-attention layers use a block-bidirectional mask; max sequence length 16,384 tokens. Note: pipeline is text-classification and self-hosting requires the official setup.sh with the OneJev runtime (a single CUDA GPU) — this is not a general chat model.
0
Quantized builds
26
Benchmark records
1
Discussions
Quantized builds
Community and official quantized releases; "linked records" counts only benchmarks tied to that exact release.
No quantized builds cataloged yet.
Benchmark data
26 configurations
| Hardware | Framework | Quant | Decode | VRAM used | Samples | Evidence |
|---|---|---|---|---|---|---|
| NVIDIA RTX 4090 | Ollama | IQ1_S | 130.3 tok/s | 22.8 GB | 1 | L2 Cross-framework verified |
| AMD Ryzen AI Max+ 395 | llama.cpp | Q4_K_S | 26.2 tok/s | 46.33 GB | 1 | L2 Cross-framework verified |
| AMD Ryzen AI Max+ 395 | vLLM | Q4_K_S | 27.6 tok/s | 46.33 GB | 1 | L2 Cross-framework verified |
| Apple M5 Max | llama.cpp | IQ4_XS | 59.8 tok/s | 44.06 GB | 1 | L2 Cross-framework verified |
| Apple M5 Max | llama.cpp | Q3_K_M | 63.7 tok/s | 40.66 GB | 1 | L2 Cross-framework verified |
| Apple M5 Max | Ollama | FP16 | 25.7 tok/s | 107 GB | 1 | L2 Cross-framework verified |
| Apple M5 Ultra | llama.cpp | Q3_K_M | 90.5 tok/s | 40.66 GB | 1 | L2 Cross-framework verified |
| Apple M5 Ultra | llama.cpp | Q5_K_S | 71.3 tok/s | 53.13 GB | 1 | L2 Cross-framework verified |
| Apple M5 Ultra | Ollama | IQ4_XS | 81.4 tok/s | 44.06 GB | 1 | L2 Cross-framework verified |
| NVIDIA H100 80GB HBM3 | llama.cpp | Q4_K_S | 342.5 tok/s | 46.33 GB | 1 | L2 Cross-framework verified |
| NVIDIA H100 80GB HBM3 | Strata | Q3_K_S | 346 tok/s | 38.96 GB | 1 | L2 Cross-framework verified |
| NVIDIA H20 | llama.cpp | FP16 | 18.6 tok/s | 92.15 GB | 1 | L2 Cross-framework verified |
| NVIDIA H20 | vLLM | IQ4_XS | 46 tok/s | 44.06 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX PRO 6000 Blackwell 96GB | vLLM | IQ3_XS | 228.2 tok/s | 37.25 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX PRO 6000 Blackwell 96GB | vLLM | Q5_K_M | 145.7 tok/s | 54.83 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX PRO 6000 Blackwell 96GB | Ollama | IQ3_XS | 214.1 tok/s | 37.25 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX PRO 6000 Blackwell 96GB | Ollama | Q5_K_S | 135.9 tok/s | 53.13 GB | 1 | L2 Cross-framework verified |
| Apple M2 Ultra | llama.cpp | IQ2_M | 102.3 tok/s | 31.58 GB | 1 | L2 Cross-framework verified |
| Apple M2 Ultra | llama.cpp | Q5_K_M | 67.4 tok/s | 54.83 GB | 1 | L2 Cross-framework verified |
| Apple M2 Ultra | Ollama | IQ1_S | 100.6 tok/s | 25.91 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX Spark | llama.cpp | IQ1_S | 15.8 tok/s | 25.91 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX Spark | llama.cpp | Q4_K_M | 11 tok/s | 48.59 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX Spark | llama.cpp | Q4_K_S | 11.8 tok/s | 46.33 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX Spark | Ollama | IQ3_S | 12.2 tok/s | 39.52 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX Spark | Ollama | IQ4_XS | 9.9 tok/s | 44.06 GB | 1 | L2 Cross-framework verified |
| NVIDIA RTX Spark | Ollama | Q4_K_M | 9.2 tok/s | 48.59 GB | 1 | L2 Cross-framework verified |
26 of these records are not linked to a specific quantized build; the full set is on the search page.