3-GATE

Regression Gate

The no-forgetting bar: specialization may not cost the model its general mind, and version N may never score below version N-1.

A two-layer discipline against the classic failure of fine-tuning — catastrophic forgetting. Layer one is Gate 1 at first certification: a regression budget measured against the student's own pre-fine-tuning base, demanding average retention of 85% or better across MMLU, HellaSwag, ARC-Challenge, Winogrande, GSM8K and IFEval, no single benchmark below 70% retention, and GSM8K at 90% or better — forgetting is measured, never assumed away. Layer two fires from v1.1 onward, every time the pedagogical loop produces a new version: R1, no-forgetting — v1.N must score at least v1.(N-1) on the prior version's full three-gate test set, within a 0.01 forgetting tolerance; and R2, genuine improvement — at or above baseline on the new test set, so a release must earn its version number. Prior selection is lineage-aware through the recorded version graph, and comparisons across pipeline modes are refused loudly rather than silently mis-scored against the wrong ancestor. The lineage layer runs today, implemented and unit-tested in the certifier; the benchmark-suite layer declares its bar ahead of the full lm-eval-harness integration we aim for.

Hallucination Audit

Gate 3: measures what the model makes up — a single fabricated entity is a hard fail at any score.

The third certification gate audits truthfulness and calibration across five probe categories: fabricated entities, plausible-but-wrong claims, temporal hallucination, cross-domain boundary violations, and confidence calibration. The bar: domain hallucination rate under 2%; out-of-domain refusal calibration at 90% or better, because a specialist should decline what it does not know rather than improvise; reasoning faithfulness above 85%, tested by perturbing a premise and requiring the conclusion to move with it (the SCOTT-style faithfulness check applied to the reasoning chains the student was trained on); and zero fabricated entities — a hard fail that overrides every other number, since an invented citation or API is worse than a wrong one. The benchmark lineage is HalluLens (Bang et al., ACL 2025), whose taxonomy separates extrinsic hallucination (deviating from what the model was trained on) from intrinsic (deviating from the input at hand) and generates test sets dynamically so the audit cannot be memorized, alongside TruthfulQA and custom domain probes. These targets are the declared bar; full HalluLens and TruthfulQA harness integration is what we aim for — today's audit refusal-checks fictional-entity probes at prototype scale under the same hard-fail rule.

LLM-as-Judge

Scoring free-form answers with a strong model against an explicit rubric — adopted because keyword scoring proved gameable.

An evaluation method in which a strong language model grades another model's free-form output against an explicit rubric, following the MT-Bench protocol (Zheng et al., NeurIPS 2023 — strong LLM judges reach over 80% agreement with human preferences, the same level humans reach with each other). Nucleus uses it in two places: the swarm's Evaluator scores probe responses every adversarial cycle, and Gate 2 of certification scores domain mastery on three axes — reasoning quality, domain accuracy, and rubric coverage — with the pass bar set within one point of the teacher's own score on the same holdout questions. The judge replaced a legacy keyword scorer that proved structurally gameable: verbose, keyword-dense prose could pass without a single correct inference, exactly the failure a rubric-driven judge catches; the old scorer survives only behind an explicit legacy flag. The method's known biases — position, verbosity, self-enhancement — are why the judge is one instrument among three gates rather than the sole authority. The Gate 2 judge runs in today's prototype certifier; teacher-anchored judging across the full 20% holdout at HELM scale is what we aim for.