04 · SFT + RAFT — Fine-tuning
SFT + RAFT
Stage 04: LoRA fine-tuning where every batch carries oracle documents plus deliberate distractors, so the student internalizes grounded reasoning instead of memorized answers.
● available now — LoRA adapter training on MLX (Apple Silicon unified memory) — real M-series runs
Roadmap: the CUDA backend (unsloth)
Roadmap: per-subdomain RAFT configuration evolved by AutoResearch
The technical how — trained through the noise
Stage 04 of the Nucleus pipeline — the fine-tuning step where a small student (1–7B class; e.g. Falcon-H1R-7B, Qwen3-8B) internalizes the reasoning corpus produced by SCoTD + CoTD. The deliberate contrast is RAFT versus plain SFT: supervised fine-tuning on bare question–answer pairs teaches answers, while Retrieval-Augmented Fine-Tuning teaches grounded reasoning — every training pair wraps the question in oracle documents that contain the evidence plus 2–3 deliberate distractors drawn from other subdomains, with a gold answer grounded in the oracles and carrying the full symbolic chain-of-thought. After training the student no longer needs the documents at inference; the reasoning is internalized in the weights. Mechanically this is the strict training half of the pipeline's agent/training boundary: no LLM calls and no message bus inside the trainer — corpus in, gradient descent, .safetensors checkpoints out. Today the trainer runs LoRA adapters (rank 8, alpha 16, lr 1e-4) on the MLX backend, exploiting Apple Silicon unified memory, and real M-series runs have produced sealed lineages; a CUDA backend (unsloth) and per-subdomain RAFT configs evolved automatically by AutoResearch are the design's next step, not yet the shipped path.
Anatomy of one training batch
Plain SFT on bare question–answer pairs teaches answers. A RAFT batch teaches grounded reasoning — the question arrives surrounded by evidence and noise, and only answers that stand on the oracle are rewarded.
One training pair
The gradient does the discipline: pattern-matching the distractors is punished, so the cheapest strategy left to the student is genuinely discriminating evidence. A fraction of pairs ships with no oracle at all, forcing knowledge into the weights — after training, the student no longer needs the documents at inference.
The three moving parts
RAFT
The training recipe that fine-tunes a model on questions packaged with both helpful and irrelevant documents, so it learns to tell evidence from noise.
Retrieval-Augmented Fine-Tuning (Zhang et al., UC Berkeley): a recipe for adapting a language model to domain-specific 'open-book' settings. Each training example presents a question together with a set of retrieved documents — oracles that contain the answer and distractors that do not — and trains the model to ignore the distractors, grounding its answer by citing the relevant span from the oracle and reasoning chain-of-thought style. A fraction of examples ships with no oracle at all, which forces the model to also consolidate knowledge into its weights rather than lean entirely on whatever is retrieved. In Nucleus, pair construction is implemented in the Synthesizer's RAFTBuilder and runs end-to-end today; evolving each subdomain's oracle/distractor mix automatically via AutoResearch is the designed next step.
In the World →Distractor Injection
Deliberately salting every training batch with plausible-but-irrelevant documents, so surface pattern-matching stops paying off.
The noise half of a RAFT training pair: alongside the oracle documents, each example carries 2–3 distractors — documents chosen to be plausible enough to fool a naive model yet distinguishable through genuine reasoning. In Nucleus they are drawn from different subdomains and extraction layers of the same corpus, which makes them topically adjacent rather than randomly wrong. The mechanism is in the gradient: only answers grounded in the oracle are rewarded, so the cheapest strategy left to the student is actually discriminating evidence. The Swarm's Adversary later attacks the same weakness from the outside at evaluation time; distractor injection bakes the defense in during training.
In the World →LoRA Adapter
Small trainable low-rank matrices grafted onto a frozen base model — the only weights Nucleus fine-tuning actually updates.
Low-Rank Adaptation (Hu et al.) freezes the pre-trained base weights and injects trainable rank-decomposition matrices into the transformer's projection layers; only those small matrices receive gradients, cutting trainable parameters by orders of magnitude while matching full fine-tuning quality at this scale. Nucleus trains its 1–7B students with rank 8 and alpha 16 at learning rate 1e-4 on the MLX backend, where Apple Silicon unified memory means no GPU data copies. The adapters ship as .safetensors checkpoints and are first-class provenance artifacts: the adapter config and trained adapter weights are hashed into the seal's component hashes, and Hybrid Versioning carries adapters forward across model versions while the base architecture stays frozen per lineage.
In the World →Mechanically this is the strict training half of the pipeline's agent/training boundary: no LLM calls and no message bus inside the trainer — corpus in, gradient descent, .safetensors checkpoints out. ● Available now — LoRA training on the MLX backend (Apple Silicon unified memory), with real M-series runs behind sealed lineages. Roadmap: the CUDA backend (unsloth) and per-subdomain RAFT configs evolved automatically by AutoResearch.
Inputs
Symbolic chains · oracle docs + distractors
Outputs
Trained student weights (1–7B) · LoRA adapters
In the literature
The vocabulary
scotd-cotd
SCoTD + CoTD
The synthesis stage: teacher rationales are decomposed into explicit symbolic steps, and those reasoning chains — not raw answers — become the training data.
in the dictionary →
swarm
SWARM
The four-agent adversarial loop that probes, traps, scores, and corrects the student until it can no longer be broken.
in the dictionary →
raft
RAFT
The training recipe that fine-tunes a model on questions packaged with both helpful and irrelevant documents, so it learns to tell evidence from noise.
in the dictionary →
distractor-injection
Distractor Injection
Deliberately salting every training batch with plausible-but-irrelevant documents, so surface pattern-matching stops paying off.
in the dictionary →
lora-adapter
LoRA Adapter
Small trainable low-rank matrices grafted onto a frozen base model — the only weights Nucleus fine-tuning actually updates.
in the dictionary →
For agents
$ curl "https://qukaizen.com/what?term=sft-raft&world=nucleus"
# the raw compiled artifact: /worlds/nucleus/terms.json