Unsloth 还是 AirLLM？一个炼丹，一个硬跑

同一个问题，两种答案：你想在 4G 显存的旧卡上跑 70B，还是想在 24G 卡上把一个开源模型调教成你自己的形状？

前者找 AirLLM，后者找 Unsloth。网上教程常把这两个工具放在一起比，但多数人没搞明白：它们一个管炼丹，一个管硬跑，根本不在同一条赛道上——虽然 2026 年的今天，两家都在偷偷往对方的地盘伸脚。

一句话核心区分：Unsloth = 高效微调（炼丹）优先，推理为辅；AirLLM = 超大模型低显存推理优先，微调为辅。 定位不同，不互斥，还能组合用。

本文所有数据基于 2026-09-09 两仓库官方 README 与文档（unslothai/unsloth 75.9k★、lyogavin/airllm 33.9k★）。推理框架是时间函数，版本不锁死的对比都是耍流氓。

Unsloth：把炼丹炉搬到消费级显卡

它现在到底是什么

Unsloth 2023 年 11 月开源，Apache-2.0。到今天它已经不是当年那个纯训练库了——官方现在的自我介绍是「第一个本地跑模型 + 训练模型的桌面应用」：有 Desktop 桌面应用、Studio Web UI、Core 代码库三种形态，能跑 GGUF/MLX、能连 Claude Code/Codex/Hermes 这类 agent、能做 RAG、能开 OpenAI 兼容 API。

但它的王牌始终没变：微调。

核心原理（为什么它快）

• 手写 Triton GPU 内核，算子融合：把矩阵运算、LoRA、归一化、RoPE 等算子熔成一个内核，消除大量中间张量——显存占用大幅下降，反向传播大幅加速。这是它相对原生 HF 快和省的根本。
• 训练侧优化拉满：梯度检查点、padding-free 序列 packing、长上下文 Flash Attention、原生 4bit 动态 QLoRA 量化。
• 底层改造 transformers/PEFT/TRL，API 兼容 HuggingFace 生态——会 HF 就会 Unsloth，训练完能导出 GGUF、MLX、safetensors、NVFP4/FP8 给下游引擎用。

官方当前口径：训练 2x 提速、最高 70% 省显存，无精度损失；MoE 模型训练宣称最高 12x（DeepSeek/GLM/Qwen/gpt-oss 家族）。

支持的任务和模型

SFT 监督微调、DPO/ORPO 偏好优化、GRPO 强化学习（推理模型训练）、连续预训练、多模态 VLM、TTS、embedding、扩散模型微调。

模型家族基本全收：Qwen3.8、DeepSeek-V4、GLM-5.x、Kimi K3/K2.7、MiniMax、Gemma 4、Llama、gpt-oss 等。

显存真相：一张表辟谣

Unsloth 官方文档给了 QLoRA（4bit）和 LoRA（16bit）的微调最低显存表，直接看 2026-09 现值：

翻译成人话：

• 8B 微调只要 6GB——免费 Colab 的 T4（16GB）绰绰有余，这是 Unsloth「免费炼丹」传说的来源。
• 70B QLoRA 最低 41GB——需要两张 24G 卡或一张 48G 卡（A6000/RTX 6000 Ada 级别）。网上流传的「单卡 T4/3090/4090 微调 70B」是早期营销的残余印象，一张 4090（24GB）塞不下。官方自己的免费 notebook 也只给到 8B~20B 档。
• 405B 微调要 237GB——那是多卡或数据中心的事，消费级别想。

所以 Unsloth 的真实射程：显存 6~24GB 的个人电脑，低成本做 8B~27B 的领域定制；70B 起步请先摸摸自己的钱包。

优缺点

优点：

• 炼丹神器。同硬件下训练速度和显存利用率碾压原生 HF，个人电脑低成本做领域模型
• 生态兼容好，开箱 notebook 多，新手友好；训练完直接导出 GGUF/MLX 喂 llama.cpp、Ollama、vLLM
• 长上下文优化强，官方宣称 500K 上下文训练（20B 模型 + 80GB 卡）
• 2026 年硬件支持面已经铺开：Windows/Linux/macOS，NVIDIA + AMD ROCm + Intel + CPU + Vulkan 都能训练

缺点：

• 推理不是它的主场。走的是常规 GGUF/整体权重路线，模型权重整体进显存或内存，不是 AirLLM 那种逐层流式——「4G 显存跑 70B」不是它的设计场景
• 纯推理速度不如专职的 vLLM、llama.cpp
• 全家桶形态意味着功能多、学习面宽，只想快速起个推理服务的人会被 UI 和概念淹没

适用场景

本地 QLoRA/LoRA 微调、行业定制模型、数据集迭代、DPO/GRPO 对齐、微调完导出给生产引擎。适合你已经有够用的显存、想训练自己模型的场景。

AirLLM：把 2.8T 模型塞进 4GB 显存

核心原理：显存只留一层

AirLLM 2023 年 11 月首发，Apache-2.0。核心思路一句话：模型按 Transformer 层预先拆开存硬盘，推理时 GPU 一次只加载一层，算完立刻释放；后台多线程预取下一层，掩盖磁盘 IO 延迟。MoE 模型更狠——只加载当前 token 路由命中的专家。

推论很反直觉：显存峰值 ≈ 单层权重大小，和模型总参数量无关。 671B 和 7B 的显存需求可能落在同一个数量级，差的是硬盘和耐心。

官方能力清单（实测值）

AirLLM README 直接给了各模型的实测显存：

注意三点：

1. 全部是不开量化、原生 FP16 全精度的显存——这是 AirLLM 和 GGUF/AWQ 全局量化的本质区别：它不靠丢精度换体积，靠的是「一次只搬一层」。
2. 「4GB 跑 70B」的 4GB 是显存需求，不是整机内存。模型本体照样要完整躺在硬盘上（70B FP16 ≈ 140GB，Kimi K3 这种 2.8T 要按 TB 算），SSD 越大越好，机械硬盘直接劝退。
3. Kimi K3 是官方实测 3.72GB（RTX 6000 Ada 上跑通）——2.8T 参数，目前最大的开源模型，塞进 4G 显存。

平台与速度：别指望它聊天

• 平台：NVIDIA（CUDA）+ Apple Silicon（M 系，走 MLX）。AMD 目前没有官方支持——GitHub 上好几个 AMD/ROCm 的 issue 还开着，别被二手文章骗了。
• 速度：瓶颈在磁盘 IO。官方只给相对值：块级 4/8bit 压缩最高 3x 提速（瓶颈是搬运大小，量化权重就行，不用量化激活，精度损失很小）、预取约 10% 提升。绝对速度官方没给统一 benchmark——社区实测大模型普遍是个位数 token/s，取决于你的 NVMe 顺序读速度和模型层大小。什么概念？一篇 1000 token 的回答，可能要等 5~15 分钟。适合离线批量，不适合实时对话。

2026 年新变化：它也开始炼丹了

2026 年 9 月 AirLLM 加了训练支持：冻结的基座权重还是分层从磁盘流式进出，只有 LoRA adapter 常驻显存。官方示例：Qwen3.8-Flash-Next（125B MoE）在 6GB 显存（RTX 3060 Ti）下能训；Qwen3.8-27B 在 ~2GB 下能训（seq 512）。

但别激动——这是「能跑」不是「好用」：

• 训练脚本是示例级的（train_qwen38_flash_next_lora.py），不是 Unsloth 那种完整训练栈
• 每个 step 权重都要从硬盘过一遍，训练速度被磁盘 IO 卡死，只适合超小数据集 + 原型验证
• 目前示例集中在 Qwen3.8 家族，要吃别的模型得自己改

一句话：AirLLM 的微调是 2026 年附赠的入场券，和 Unsloth 的炼丹炉不是一个物种。

优缺点

优点：

• 显存门槛极致低：老笔记本、4G 亮机卡都能加载千亿、万亿参数模型，全精度无损推理
• 稠密和 MoE 都支持，MoE 只加载路由命中的专家，效率更高
• HF 风格 API，AutoModel.from_pretrained() 一行代码换模型，模型家族覆盖极广（Llama/Qwen/DeepSeek/Mistral/Phi/Gemma/ChatGLM/Baichuan/InternLM/Yi/Kimi K3）
• 可选块级 4/8bit 压缩提速，精度损失小

缺点：

• 慢。瓶颈是 SSD 顺序读，普通 NVMe 跑大模型就是个位数 token/s，机械硬盘是灾难
• 硬盘占用巨大：模型要预拆分层保存，需要大容量 NVMe；首次拆分也极耗盘（官方 FAQ 第一条就是拆到磁盘满的报错）
• 不适合高并发，只适合单条串行推理
• 微调是后来附加的能力，性能远弱于 Unsloth

适用场景

低配电脑体验千亿/万亿级基座、离线长文档批量摘要、超大 MoE 原型验证。硬件显存很小、只想跑大模型、不追求速度。

横向对比总表

组合玩法：Unsloth 炼丹，AirLLM 硬跑

两个工具不打架，还能接力：

1. 在显存充足的机器上（4090 双卡/48G 单卡），用 Unsloth 对 7B~27B（70B 需要 41GB+）基座做 QLoRA 微调，得到 LoRA 或合并后的模型；
2. 把 LoRA 合并回基座，导出 HF safetensors 格式（AirLLM 吃完整权重，不是 adapter）；
3. 拷到只有 4G/6G 显存的低配笔记本，用 AirLLM 分层加载推理你微调后的模型。

真实限制必须说清楚：

• AirLLM 跑微调后的模型，速度依然被 SSD IO 锁死——适合离线批量任务（文档摘要、批量打分），不适合实时对话；
• 你的低配机器要装得下合并后的完整模型（70B FP16 ≈ 140GB 硬盘）；
• 微调数据量小、追求效果就选这条路；想要实时对话，趁早放弃「低配机器 + 大模型」这个组合，上云或换 llama.cpp 量化。

选型速断

1. 想训练/微调模型：直接选 Unsloth。8B 级 6GB 显存就能起步，免费 Colab 就能跑。
2. 显存很小（4G/6G），只想跑 70B+ 超大模型看效果、不炼丹：直接选 AirLLM。能跑，但请接受个位数 token/s 和按小时的等待。
3. 显存 12G+，想要快速本地对话推理：别选 AirLLM，也别迷信 Unsloth 推理——这两个都不是专职推理引擎，优先 vLLM/llama.cpp 量化方案。
4. 低配电脑想跑自己微调后的模型：Unsloth 炼丹 + AirLLM 推理接力，前提是离线任务能忍。
5. 显存有 24G 且要实时对话：Unsloth 全家桶或 vLLM 都行，AirLLM 的逐层流式在这里没有优势。

收尾一句：炼丹的别拿锤子敲钉子，硬跑的别指望破卡跑出光速。工具认路，不认信仰——先算清楚你的显存、硬盘和耐心，再选工具。

版权没有，随意转载。过意不去就加一句「原文发表于 cn-res.vip」，不加也无所谓。

Same question, two answers: you want to run a 70B model on a 4GB GPU, or you want to shape an open-weight model into your own thing on a 24GB card?

The first one is AirLLM territory. The second is Unsloth's. Tutorials keep putting these two tools side by side, but most people miss the point: one is a training furnace, the other is a stubborn runner — they are not on the same track, even though both projects have started sneaking into each other's lane by 2026.

The one-line distinction: Unsloth = efficient fine-tuning first, inference as a side dish; AirLLM = huge-model low-VRAM inference first, fine-tuning as a side dish. Different positioning, not mutually exclusive — and they can be chained together.

All figures below are from the official READMEs and docs of both repos as of 2026-09-09 (unslothai/unsloth 75.9k★, lyogavin/airllm 33.9k★). Inference frameworks are functions of time — a comparison without pinned versions and dates is worthless.

Unsloth: a training furnace for consumer GPUs

What it actually is now

Open-sourced in November 2023, Apache-2.0. Unsloth is no longer just a training library. Its own pitch today: "the first desktop app to run and train models" — a Desktop app, a Studio web UI, and the Core code library. It runs GGUF/MLX, connects local models to agent harnesses (Claude Code, Codex, Hermes), does RAG, and exposes an OpenAI-compatible API.

But its ace has never changed: fine-tuning.

Why it is fast

• Hand-written Triton kernels with operator fusion: matrix multiplies, LoRA, normalization, RoPE are fused into single kernels, eliminating intermediate tensors — much lower VRAM, much faster backward pass. This is the real source of its edge over vanilla HuggingFace.
• Training-side optimization, maxed out: gradient checkpointing, padding-free sequence packing, long-context Flash Attention, native 4-bit dynamic QLoRA.
• Built on top of transformers/PEFT/TRL with an HF-compatible API — if you know HuggingFace, you know Unsloth. After training you can export GGUF, MLX, safetensors, NVFP4/FP8 for downstream engines.

Official current claim: 2× faster training, up to 70% less VRAM, no accuracy loss; MoE training up to 12× faster (DeepSeek/GLM/Qwen/gpt-oss family).

Tasks and model support

SFT, DPO/ORPO, GRPO (reasoning-model RL), continued pretraining, multimodal VLM, TTS, embeddings, diffusion fine-tuning. Model families: Qwen3.8, DeepSeek-V4, GLM-5.x, Kimi K3/K2.7, MiniMax, Gemma 4, Llama, gpt-oss, and more.

The VRAM truth, in one table

Unsloth's official docs publish minimum VRAM for QLoRA (4-bit) and LoRA (16-bit) fine-tuning. Current values (2026-09):

In plain words:

• 8B fine-tuning needs only 6GB — a free Colab T4 (16GB) handles it easily. This is the origin of the "free fine-tuning" legend.
• 70B QLoRA needs at least 41GB — two 24GB cards or one 48GB card (A6000 / RTX 6000 Ada class). The old "fine-tune 70B on a single T4/3090/4090" claim floating around the web is leftover marketing — a single 4090 (24GB) cannot hold it. Even Unsloth's own free notebooks top out at the 8B–20B tier.
• 405B fine-tuning needs 237GB — multi-GPU or datacenter territory. Not for consumers.

Unsloth's real range: 6–24GB VRAM machines doing cost-effective 8B–27B domain customization. At 70B, check your wallet first.

Pros and cons

Pros:

• Training speed and VRAM efficiency crush vanilla HF on the same hardware. Cheap domain models on a personal PC.
• HF-ecosystem compatible, tons of ready-made notebooks, beginner friendly; exports to GGUF/MLX for llama.cpp, Ollama, vLLM.
• Strong long-context story — official claim of 500K-context training (20B model on an 80GB GPU).
• Hardware coverage broadened a lot by 2026: Windows/Linux/macOS, NVIDIA + AMD ROCm + Intel + CPU + Vulkan can all train.

Cons:

• Inference is not its home turf. It follows the conventional GGUF / full-weights path — weights resident in VRAM or RAM, not AirLLM-style layer streaming. "Run 70B on 4GB VRAM" is not its design point.
• Pure inference throughput trails dedicated engines like vLLM and llama.cpp.
• The all-in-one form means more surface area; people who just want a quick inference server may drown in UI and concepts.

When to use it

Local QLoRA/LoRA fine-tuning, domain-specific models, dataset iteration, DPO/GRPO alignment, exporting tuned models to production engines. For when you already have enough VRAM and want to train your own model.

AirLLM: stuffing a 2.8T model into 4GB of VRAM

The trick: only one layer lives on the GPU

AirLLM first shipped in November 2023, Apache-2.0. Core idea in one sentence: the model is pre-split into per-layer shards on disk; at inference time the GPU loads one layer at a time, computes, and frees it immediately — while background threads prefetch the next layer to hide disk latency. For MoE models it is even leaner: only the experts the current token actually routes to get loaded.

The counter-intuitive corollary: peak VRAM ≈ the size of a single layer, independent of total parameter count. A 671B model and a 7B model can land in the same VRAM ballpark. What differs is disk space and patience.

Official capability list (measured)

AirLLM's README lists measured VRAM per model:

Three caveats:

1. All of these are native FP16 full precision, no quantization — that is the essential difference from GGUF/AWQ global quantization: AirLLM doesn't trade precision for size, it just never holds more than one layer at a time.
2. "4GB for 70B" means VRAM, not total memory. The full model still has to sit on disk (70B FP16 ≈ 140GB; a 2.8T model like Kimi K3 means terabytes). The bigger the NVMe the better; mechanical hard drives are a non-starter.
3. Kimi K3 measured at 3.72GB (run end-to-end on an RTX 6000 Ada) — 2.8T parameters, currently the largest open model, squeezed into 4GB of VRAM.

Platforms and speed: don't expect a chat partner

• Platforms: NVIDIA (CUDA) + Apple Silicon (M-series, via MLX). No official AMD support — several AMD/ROCm issues are still open on GitHub. Don't trust recycled blog posts claiming otherwise.
• Speed: the bottleneck is disk I/O. The official docs only give relative numbers: block-wise 4/8-bit compression up to 3× faster (the bottleneck is how much gets moved, so quantizing weights alone — not activations — is enough, with minimal accuracy loss), prefetching ~10%. No official absolute-throughput benchmark exists — community measurements on large models typically land at single-digit tokens/s, depending on your NVMe sequential read speed and per-layer size. A 1000-token answer can mean waiting 5–15 minutes. Fine for offline batch work, not for real-time chat.

New in 2026: it started training too

In September 2026 AirLLM added training support: frozen base weights still stream from disk layer by layer, only LoRA adapters stay resident in VRAM. Official examples: Qwen3.8-Flash-Next (125B MoE) trains under 6GB (RTX 3060 Ti); Qwen3.8-27B trains in ~2GB at seq 512.

Don't get excited — this is "it runs", not "it's good":

• Training scripts are example-grade (train_qwen38_flash_next_lora.py), not a full training stack like Unsloth's.
• Every step drags weights across the disk — training throughput is strangled by I/O, viable only for tiny datasets and prototype validation.
• Official examples currently cover the Qwen3.8 family; other architectures need DIY work.

In short: AirLLM's fine-tuning is a 2026 admission ticket, not a competitor to Unsloth's furnace.

Pros and cons

Pros:

• Extremely low VRAM threshold: old laptops and 4GB GPUs can load hundred-billion and trillion-parameter models, lossless full precision.
• Both dense and MoE supported; MoE loads only routed experts.
• HF-style API — AutoModel.from_pretrained() one-liner, model families covered broadly (Llama/Qwen/DeepSeek/Mistral/Phi/Gemma/ChatGLM/Baichuan/InternLM/Yi/Kimi K3).
• Optional block-wise 4/8-bit compression for speed, minimal accuracy loss.

Cons:

• Slow. Bottlenecked by SSD sequential reads; single-digit tokens/s on ordinary NVMe at scale. Mechanical disks are a disaster.
• Huge disk footprint: models must be pre-split into layer shards; large NVMe required, and the first split eats disk space aggressively (the official FAQ's first entry is literally the "disk full" error).
• No high concurrency — single-stream serial inference only.
• Fine-tuning is a bolt-on; nowhere near Unsloth's performance.

When to use it

Experiencing 100B–1T+ class base models on a weak machine, offline long-document batch summarization, prototype validation of huge MoEs. For when your VRAM is tiny, you just want to run big models, and you don't care about speed.

Side-by-side comparison

The combo: train with Unsloth, run with AirLLM

These tools don't fight; they can relay:

1. On a well-equipped machine (dual 4090 / single 48GB), fine-tune a 7B–27B base with Unsloth QLoRA (70B needs 41GB+, remember), producing a LoRA or a merged model;
2. Merge the LoRA back into the base and export HF safetensors (AirLLM consumes full weights, not adapters);
3. Copy it to a low-end 4G/6G laptop and run your fine-tuned model with AirLLM layer streaming.

The honest caveats:

• AirLLM's throughput on your tuned model is still locked to SSD I/O — offline batch work (document summarization, batch scoring) yes, real-time chat no;
• The low-end machine must hold the full merged model on disk (70B FP16 ≈ 140GB);
• If your dataset is small and you care about the result, this path makes sense. If you want real-time chat, drop the "weak machine + huge model" dream entirely — go cloud or llama.cpp quantization.

Quick decision guide

1. You want to train/fine-tune: pick Unsloth. 8B-class starts at 6GB VRAM; free Colab works.
2. Tiny VRAM (4G/6G), just want to run 70B+ monsters and see what they do: pick AirLLM. It works — accept single-digit tokens/s and hour-long waits.
3. 12GB+ VRAM, want fast local chat: skip both. Neither is a dedicated inference engine — go vLLM or llama.cpp with quantization.
4. Weak machine, want to run your own fine-tuned model: Unsloth for training + AirLLM for inference relay — only if offline batch is acceptable.
5. 24GB VRAM and real-time chat: Unsloth's all-in-one or vLLM both work; AirLLM's layer streaming buys you nothing here.

Last word: a blacksmith doesn't hammer with a wrench, and a stubborn runner doesn't expect a broken GPU to produce light speed. Tools follow the route, not your faith — count your VRAM, your disk, and your patience before you pick.

No copyright reserved — share it freely. If you feel like it, a credit line to cn-res.vip is appreciated. Or don't. Either way.