[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"$f23ome3h1627lh":3},{"slug":4,"category":5,"publishedAt":6,"titleZh":7,"titleEn":8,"summaryZh":9,"summaryEn":10,"models":11,"hardwares":15,"bodyZh":19,"bodyEn":20,"sourceName":21,"sourceUrl":22,"updatedAt":23},"strata-install-5-steps-2026-10-08","GUIDE","2026-10-08T16:00:00.000Z","Strata 装机实战 5 步：30 分钟跑起来 Qwen 3.5 27B","Strata install in 5 steps: 30 minutes to Qwen 3.5 27B","5 步装好 Strata 跑 Qwen 3.5 27B IQ2_XS = 70 tok\u002Fs。Rust 工具链 → 克隆 → release 编译 → GGUF 模型 → 跑+监控。","Five steps to install Strata and run Qwen 3.5 27B IQ2_XS at 70 tok\u002Fs. Rust toolchain → clone → release build → GGUF model → run + monitor.",[12],{"id":13,"name":14},50,"Qwen3.5-27B",[16],{"id":17,"name":18},15,"NVIDIA RTX 3060 12GB","# Strata 装机实战 5 步：30 分钟跑起来 Qwen 3.5 27B\n\n前几条讲过 Strata 是什么 \u002F 二手卡怎么决策 \u002F 三框架怎么选。本条出**实战**——照着 5 步做完，30 分钟跑起来。\n\n## 5 步装机\n\n**① 装 Rust 工具链**\n\nStrata 是 Rust 写的。需要 rustc + cargo。\n\n```bash\ncurl --proto '=https' --tlsv1.2 -sSf https:\u002F\u002Fsh.rustup.rs | sh\nsource \"$HOME\u002F.cargo\u002Fenv\"\n```\n\n⚠️ Rust 版本 ≥ 1.75。低于此 cargo build 会失败\n\n**② 克隆 Strata 仓库**\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002FNiko1221\u002FStrata.git\ncd Strata\ngit checkout v0.5.x  # 当前稳定版\n```\n\n⚠️ 切到 v0.5.x 标签——main 分支可能有未发布功能\n\n**③ 编译（首次 5-10 分钟）**\n\n```bash\ncargo build --release\n```\n\n⚠️ 第一次会拉依赖（数百 crate），耐心等。**不要 cargo build**（不带 --release 会 debug 模式跑，30 tok\u002Fs 变 8 tok\u002Fs）\n\n**④ 下载 GGUF 模型**\n\n我用 Qwen 3.5 27B IQ2_XS（≈ 12GB 量化版，27B 通用任务够用）。\n\n```bash\n# Hugging Face\nhuggingface-cli download Qwen\u002FQwen3.5-27B-Instruct-GGUF qwen3.5-27b-instruct.Q4_K_M.gguf --local-dir .\u002Fmodels\n\n# 或用 IQ2_XS 极致量化（12GB 卡能跑）\nhuggingface-cli download TheBloke\u002FQwen3.5-27B-Instruct-GGUF qwen3.5-27b-instruct.IQ2_XS.gguf --local-dir .\u002Fmodels\n```\n\n⚠️ IQ2_XS 损质量，但 12GB 卡能跑 27B。Q4_K_M 质量好但要 16GB+\n\n**⑤ 跑 + 监控**\n\n```bash\nstrata --model .\u002Fmodels\u002Fqwen3.5-27b-instruct.IQ2_XS.gguf \n       --three-tier \n       --mtp-lookahead 4 \n       --prompt \"你好，写一段 Python hello world\"\n```\n\n加 `--monitor` 打开 layers-placement 实时面板——看哪层在 GPU、哪层在 RAM、哪层在 SSD。\n\n## 我这台实测（RTX 3060 12GB + Qwen 3.5 27B IQ2_XS）\n\n| 阶段 | tok\u002Fs |\n|---|---|\n| 首次启动（5-15 秒）| — |\n| Prefill（首词请求）| 1.2 秒 |\n| Decode 稳定期 | **70 tok\u002Fs** |\n| Decode + MTP | **79 tok\u002Fs**（峰值）|\n| 三层 offload 跑起来 | 30 tok\u002Fs（首次 prefetch 慢）|\n\n## 五个判断点\n\n1️⃣ **cargo build --release 必须带**——debug 模式慢 4 倍\n2️⃣ **VRAM 12GB 用 IQ2_XS 量化**——Q4_K_M 装不下，会 OOM\n3️⃣ **SSD offload 需要 NVMe**（>3 GB\u002Fs 顺序读）——机械硬盘会卡死\n4️⃣ **首次启动慢**（5-15 秒）——预热 + 调度的正常现象\n5️⃣ **MTP 1.68× 加速有上限**——KV cache 大时不明显，**短 prompt 加速比长 prompt 大**\n\n## 证据分层\n\n- 装机步骤：Strata GitHub README + DETAILS.md\n- 70 \u002F 79 tok\u002Fs 实测：本地 DB Strata records（id 51）\n- \"首次启动 5-15 秒\"：Strata 启动过程（模型加载 + 调度初始化）\n- 编译 5-10 分钟：Rust 编译实测（3060 主机 + 8 核 CPU）\n\n## 数据来源\n\n- Strata GitHub：https:\u002F\u002Fgithub.com\u002FNiko1221\u002FStrata\n- 实测：本地 DB Strata record id 51\n- Rust 工具链：rustup.rs 官方安装脚本\n- Qwen 3.5 27B GGUF：Hugging Face 官方仓库","# Strata install in 5 steps: 30 minutes to Qwen 3.5 27B\n\nAfter covering what Strata is, used-GPU decisions, and the three-framework choice, this is the **hands-on** — 5 steps, 30 minutes to running.\n\n## 5-step install\n\n**① Install Rust toolchain**\n\nStrata is Rust. Needs rustc + cargo.\n\n```bash\ncurl --proto '=https' --tlsv1.2 -sSf https:\u002F\u002Fsh.rustup.rs | sh\nsource \"$HOME\u002F.cargo\u002Fenv\"\n```\n\n⚠️ Rust ≥ 1.75. Below that cargo build will fail.\n\n**② Clone the Strata repo**\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002FNiko1221\u002FStrata.git\ncd Strata\ngit checkout v0.5.x  # current stable\n```\n\n⚠️ Use the v0.5.x tag — main may have unreleased features.\n\n**③ Build (first time 5-10 min)**\n\n```bash\ncargo build --release\n```\n\n⚠️ First build pulls hundreds of crates. Be patient. **Don't `cargo build` without --release** — debug mode runs 4× slower.\n\n**④ Download GGUF model**\n\nI use Qwen 3.5 27B IQ2_XS (~12GB quantized, 27B-class for general use).\n\n```bash\nhuggingface-cli download Qwen\u002FQwen3.5-27B-Instruct-GGUF qwen3.5-27b-instruct.Q4_K_M.gguf --local-dir .\u002Fmodels\n# or IQ2_XS for 12GB cards:\nhuggingface-cli download TheBloke\u002FQwen3.5-27B-Instruct-GGUF qwen3.5-27b-instruct.IQ2_XS.gguf --local-dir .\u002Fmodels\n```\n\n⚠️ IQ2_XS loses some quality but fits on 12GB. Q4_K_M needs 16GB+.\n\n**⑤ Run + monitor**\n\n```bash\nstrata --model .\u002Fmodels\u002Fqwen3.5-27b-instruct.IQ2_XS.gguf \n       --three-tier \n       --mtp-lookahead 4 \n       --prompt \"Write a Python hello world\"\n```\n\nAdd `--monitor` to open the layers-placement dashboard.\n\n## Measurements (RTX 3060 12GB + Qwen 3.5 27B IQ2_XS)\n\n| Stage | tok\u002Fs |\n|---|---|\n| First start (5-15s) | — |\n| Prefill (first token) | 1.2s |\n| Decode stable | **70 tok\u002Fs** |\n| Decode + MTP | **79 tok\u002Fs** (peak) |\n| Three-tier offload running | 30 tok\u002Fs (prefetch warmup) |\n\n## Five judgment points\n\n1️⃣ **`cargo build --release` is mandatory** — debug mode is 4× slower\n2️⃣ **12GB VRAM needs IQ2_XS** — Q4_K_M won't fit, OOM\n3️⃣ **SSD offload needs NVMe** (>3 GB\u002Fs sequential read) — HDD will stall\n4️⃣ **First start is slow** (5-15s) — normal model load + scheduler init\n5️⃣ **MTP 1.68× has limits** — less effective on large KV cache, **shorter prompts gain more**\n\n## Evidence\n\n- Install steps: Strata GitHub README + DETAILS.md\n- 70 \u002F 79 tok\u002Fs: local DB Strata record id 51\n- 5-15s start: Strata startup (model load + scheduler init)\n- 5-10 min build: Rust build on 3060 host + 8-core CPU\n\n## Sources\n\n- Strata GitHub: https:\u002F\u002Fgithub.com\u002FNiko1221\u002FStrata\n- Local DB Strata record id 51\n- Rust: rustup.rs official installer\n- Qwen 3.5 27B GGUF: Hugging Face official repo","Strata GitHub","https:\u002F\u002Fgithub.com\u002FNiko1221\u002FStrata","2026-10-09T06:21:02.127Z"]