The AI / ML Dictionary.

Removing one component to measure how much it actually contributes.

An ablation study turns a part off — a layer, a loss term, a data source — and measures the drop, isolating what really matters. It's how you separate the ingredient from the marketing.

The set of things an agent can do — internal (reason, retrieve) and external (call tools, act in the world).

In the CoALA framework an agent's action space splits into internal actions (reasoning, retrieval from memory, learning) and external/grounding actions (tool calls, environment steps). Defining a clear, bounded action space is what makes an agent controllable and safe.

Trade compute for memory by recomputing activations in the backward pass instead of storing them.

Activation (gradient) checkpointing discards intermediate activations during the forward pass and recomputes them when needed for backprop, cutting memory at the cost of an extra forward pass. It is essential for training large models or long sequences on limited memory.

The nonlinear function applied to neuron outputs, letting networks model more than straight lines.

An activation function applies a nonlinearity to a layer's outputs; without it, stacked linear layers collapse into a single linear map. Modern transformers favor smooth gated variants (GELU, SwiGLU) over older ReLU for better gradients and quality. The choice sits inside the feed-forward block.

Adaptive moment estimation — per-parameter adaptive LR via running mean and variance of gradients.

Adam (Kingma & Ba, 2015) maintains exponential moving averages of both the gradient (first moment, m) and the squared gradient (second moment, v) for each parameter. The adaptive per-parameter LR means that parameters with sparse or noisy gradients still receive meaningful updates. Adam is the default optimizer for most deep learning; AdamW is preferred for transformer fine-tuning (decoupled weight decay).

The default optimizer for training transformers — Adam with decoupled weight decay.

AdamW adapts the learning rate per parameter using running estimates of gradient mean and variance, and decouples weight decay from the gradient update for cleaner regularization. It is the workhorse optimizer for LLM training.

Small bottleneck modules inserted into transformer layers — trained while base model is frozen.

Adapter layers (Houlsby et al., 2019) insert small two-layer bottleneck modules (down-project → nonlinearity → up-project) inside each transformer layer. Only adapter parameters are trained during fine-tuning; the base model is frozen. This enables multi-task fine-tuning by swapping adapter sets and is a PEFT method. LoRA has largely superseded adapters for LLM fine-tuning but adapters remain common in multi-modal and multi-task settings.

Small trainable modules inserted into a frozen model to add new skills without retraining it.

Adapters are tiny bottleneck layers added between a frozen model's existing layers; only the adapters train. They are a parameter-efficient way to teach new tasks, and you can keep a library of swappable adapters for one base. LoRA is a popular low-rank flavor of this idea.

Cap gradient norms before the optimizer step to prevent destabilizing updates.

Gradient clipping rescales the gradient vector so its L2 norm does not exceed a threshold (commonly 1.0), preventing any single step from making a catastrophically large parameter update. It is the standard defense against exploding gradients in deep or recurrent models. In PyTorch, applied via `torch.nn.utils.clip_grad_norm_(parameters, max_norm=1.0)` before `optimizer.step()`.

Apply dropout, weight decay, or data augmentation to reduce overfitting.

When the train/val loss gap is large (overfitting), regularization constrains the model from over-specializing to training data. Options: weight decay (L2) penalizes large weights via the optimizer; dropout randomly zeroes activations during training; data augmentation expands effective dataset size; label smoothing prevents overconfident predictions. For fine-tuning, LoRA is an implicit regularizer (low-rank constraint).

A loop of agents (interrogate, challenge, evaluate, correct) that hardens a model until it stops breaking.

The Adversarial Swarm Reactor pits Interrogator, Adversary, Evaluator, and Corrector agents (plus data-collection agents) against the student in cycles, systematically hunting and eliminating hallucination pathways. The model graduates not by passing a fixed test but when the swarm can no longer break it.

QuKaiZen's inference engine that streams frontier models off disk so they run without full GPU residency.

AeroLLM is the inference layer that makes disk-streamed teachers practical — layer streaming plus speculative decoding to claw back speed. It is how QuKaiZen serves 400B+ teachers on workstations that lack the VRAM to hold them.

[ROADMAP] QuKaiZen's OSS inference engine for running SLMs without full GPU residency.

[ROADMAP] AeroLLM is QuKaiZen's open-source inference engine for running small language models on consumer hardware without requiring the model to reside fully in GPU VRAM. AeroLLM the OSS engine exists (separate repo). Its integration into the QuKaiZen bake pipeline — specifically, running the baked domain-specialist SLM at 43+ tok/s as the runtime serving component — is ROADMAP within this pipeline. The label is ROADMAP for the pipeline integration specifically; the engine itself is independently available.

An LLM that takes actions — calls tools, makes decisions — toward a goal, not just chats.

An agent wraps a model with tools, memory, and a control loop so it can plan, act, observe, and iterate. PaperAgents declares teams of small specialist agents; ARAIL's Buddy is a lab agent.

The repeating perceive-decide-act cycle that drives an autonomous agent.

An agent loop iterates: observe the environment/state, decide the next action (reason, retrieve, or call a tool), act, then observe the result — repeating until the goal is met or a stop condition fires. It is the control structure underlying ReAct and CoALA's decision procedure.

Software built around autonomous, tool-using model agents.

Agentic systems give models autonomy to decide and act over many steps using tools and feedback, instead of producing a single response. The tradeoff is power vs. predictability — hence guardrails and declared workflows.

Position handling that biases attention scores by distance instead of adding position embeddings.

Attention with Linear Biases adds a distance-proportional penalty to attention scores rather than using explicit positional encodings. This lets a model trained on short contexts extrapolate to longer ones at inference with less degradation.

Making a model's behavior match human intent and values.

Alignment is the work of making models helpful, honest, and harmless — via methods like RLHF and DPO plus evaluation for refusal and faithfulness. Misalignment shows up as unsafe or off-intent output.

Ramp the LR from near-zero to peak over N steps before the main schedule.

Starting training with the full learning rate before the optimizer has accumulated good gradient statistics can cause early divergence. A warmup phase ramps the LR linearly from near-zero to the peak LR over a fixed number of steps (commonly 1–5% of total steps, or 500–2000 steps for large models), giving the model time to settle before the optimizer takes large steps. After warmup, a cosine or linear decay schedule is applied.

The ratio of compute to memory traffic; it determines whether a workload is compute- or memory-bound.

Arithmetic intensity is FLOPs per byte moved. Low intensity (like LLM decoding) means the workload waits on memory; high intensity (like prefill or large batches) means it's limited by compute. The roofline model uses it to predict achievable performance.

The mechanism that lets each token weigh and pull information from every other token.

Attention computes, for each token, a weighted sum of all tokens' value vectors, where weights come from the similarity (dot product) of its query with others' keys. It is how transformers model long-range relationships, and its quadratic cost is what FlashAttention and the KV-cache optimize.

Initial tokens that attention disproportionately fixates on; preserving them stabilizes long/streaming generation.

Models learn to dump excess attention weight onto the first few tokens (an 'attention sink'). Keeping those tokens in the KV-cache while evicting middle ones lets a model stream indefinitely without the quality collapse a naive sliding window causes.

Encoder-decoder trained to reconstruct its own input — learns a compressed representation.

An autoencoder trains an encoder (maps input to a lower-dimensional latent code) and a decoder (reconstructs the input from the code) by minimizing reconstruction loss. The bottleneck forces the model to learn a compact, meaningful representation. Used for dimensionality reduction, denoising, and as a component in generative models (VAE) and tokenizers for image generation (VQ-VAE).

Letting software run repeatable work end-to-end with no human in the loop.

Automation captures a repeatable process so it runs on its own, reliably and on schedule. PaperAgents automates with small specialist agents reconciled to a desired state.

The swarm's brain — it evolves the rubrics every other agent consults.

AutoResearch is a first-class meta-agent that evolves the rubrics driving probes, traps, and scoring, and independently fact-checks certification. It is never merged into another service.

Low-bit quantization that protects the small fraction of weights tied to large activations, preserving accuracy.

Activation-aware Weight Quantization observes that a small set of weight channels — those multiplying large activations — matter disproportionately, and scales them to reduce their quantization error before quantizing the rest to low bits. It yields accurate 4-bit models that are fast to run and is widely used for deployment.

The algorithm that computes how to nudge every weight by propagating error gradients backward.

Backpropagation applies the chain rule to compute the gradient of the loss with respect to every parameter, flowing from the output layer back to the input. Those gradients tell the optimizer which direction to move each weight to reduce error.

[ROADMAP] The third stage of the QuKaiZen knowledge lifecycle — RAW → COMPILED → BAKED.

[ROADMAP] BAKED is the third stage of QuKaiZen's knowledge lifecycle. RAW is unsourced/unverified content; COMPILED is a gate-passed terms.json (produced today by assemble-world.mts); BAKED is a sealed domain-specialist SLM trained on the compiled corpus by Nucleus. The COMPILED stage is BUILT today; the BAKED stage is ROADMAP — it requires the full Nucleus bake pipeline. Once BAKED, the specialist model is the delivery artifact, not the terms.json.

A reference result you compare against to judge whether a change actually helped.

A baseline is the established point of comparison for an experiment — a prior model, a simple method, or the unchanged system — against which a new approach is measured. Without a baseline, a benchmark number is meaningless; the whole value of an ablation or eval is the delta from baseline.

Normalizes activations across the batch dimension to stabilize training.

Batch normalization (Ioffe & Szegedy, 2015) normalizes activations across the mini-batch, then applies learned scale and shift parameters. It reduces the sensitivity to initialization and allows higher learning rates. Standard in CNNs and MLPs; replaced by layer normalization in transformer models. At inference, batch statistics are replaced by running estimates accumulated during training.

How many training examples are processed before each weight update.

Batch size sets how many samples contribute to one gradient estimate. Larger batches give smoother gradients and better hardware utilization but need scaled learning rates and more memory; gradient accumulation simulates large batches on limited memory. It interacts tightly with learning rate.

A decoding strategy that keeps the top-k partial sequences each step to find a higher-probability output.

Beam search explores several candidate sequences (beams) in parallel, expanding and pruning to the k most probable at each step. It yields higher-likelihood, more deterministic outputs than greedy decoding — good for translation and structured tasks, but it can be bland for open-ended generation.

A standardized test set used to measure and compare model capability.

Benchmarks score models on fixed tasks — knowledge, reasoning, code — so results are comparable. QuKaiZen's Gate 1 uses MMLU, HellaSwag, ARC, GSM8K, and IFEval to verify capability survives distillation.

A 16-bit float with the same exponent range as FP32 — the default precision for training LLMs.

bfloat16 keeps FP32's 8-bit exponent (the same huge dynamic range) but truncates the mantissa to 7 bits. That range makes it numerically stable for training without loss scaling, at half the memory and bandwidth of FP32.

Distill a model into a fresh copy of identical size — the student often beats the teacher.

Born-again networks distill a trained model into a new network of the same architecture and size, using the teacher's soft predictions as targets. Despite no capacity gain, the student frequently outperforms its teacher because soft labels carry richer inter-class information than hard labels. Chaining generations (teacher -> student -> next student) can compound the gain.

ARAIL's local companion agent — a context-aware lab partner you learn alongside, running entirely on your own hardware.

ARAIL began with Buddy: a local agent to learn alongside. Buddy needed an environment, and that environment became a lab — pluggable, observable, and entirely owned by you. Buddy drives the lab in plain language and draws on your knowledge base for real context, so it can answer "what should I do next?" or "what's interesting in today's pull?" — offline, with no telemetry.

[BUILT] Frontier model used only during corpus authoring — never at runtime.

[BUILT] The build-time teacher is the frontier LLM (e.g., Claude) used during World authoring and corpus compilation. It assists in drafting definitions, sourcing verification, and knowledge synthesis — but it is never deployed as a runtime component. The pattern is: frontier model as authoring teacher → compiled World → baked SLM as runtime. This ensures the high cost of frontier inference is paid once, at build time, not per query. BUILT: this is the live authoring method for every World including this one.

A subword tokenization that iteratively merges the most frequent character pairs into tokens.

Byte-Pair Encoding builds a vocabulary by starting from characters/bytes and repeatedly merging the most frequent adjacent pair, yielding tokens that range from characters to whole words. It balances vocabulary size against sequence length and handles unseen words gracefully by falling back to subwords.

Running a small representative dataset through a model to set quantization ranges or scales.

In post-training quantization, calibration passes a small, representative sample through the model to measure activation/weight statistics, which set the scales and zero-points (or salient channels) used to map values to low precision. A poor or out-of-distribution calibration set degrades the quantized model's accuracy.

When fine-tuning on a new task erases capabilities the model previously had.

Catastrophic forgetting is the tendency of a network to overwrite old knowledge when trained on new data, because the same weights encode everything. It is why aggressive fine-tuning can wreck general ability, and why PEFT, rehearsal, and model merging are used to preserve it.

Prompting a model to show its intermediate steps, which sharply improves reasoning.

Chain-of-thought elicits step-by-step intermediate reasoning before the final answer. Wei et al. (2022) showed it dramatically improves math and logic; QuKaiZen distills symbolic CoT into small students.

A saved snapshot of model weights (and often optimizer state) you can resume or deploy from.

A checkpoint persists the model's parameters — and during training, the optimizer state and step — so a run can resume after interruption or a version can be evaluated and shipped. Modern checkpoints use safetensors for safe, fast loading.

The finding that, for a fixed compute budget, model size and training tokens should grow together.

Chinchilla showed that many large models were undertrained: for compute-optimal training, parameters and training tokens should scale in roughly equal proportion (~20 tokens per parameter as a rule of thumb). It reframed how teams allocate compute between bigger models and more data.

Training data is dominated by a few classes — rare classes are ignored.

When training data has severely unequal class frequencies, the model minimizes loss by predicting the majority class, achieving high accuracy while performing poorly on rare classes. The model has not learned the minority distribution. Addressed by oversampling rare classes, undersampling majority, loss reweighting, or focal loss.

A framework (Princeton, 2023) organizing language agents into memory modules, an action space, and a decision-making loop.

CoALA — Cognitive Architectures for Language Agents — is a conceptual framework that structures an LLM-based agent like a classical cognitive architecture. It separates the agent's memory into modules (working, episodic, semantic, procedural), defines an action space split into internal actions (reasoning, retrieval, learning) and external actions (grounding in the world via tools/environments), and a decision-making procedure that loops: propose, evaluate, and select the next action. It gives a shared vocabulary for comparing agent designs.

Align a model to an explicit written set of principles, using the model to critique and revise its own outputs.

Constitutional AI aligns a model against a 'constitution' — a list of written principles — by having the model critique and revise its responses to better follow them, then training on those revisions and on AI-generated preference labels (RLAIF). It reduces reliance on large volumes of human harm-labeling and makes the values steering the model explicit and auditable.

Restrict generation at each step to tokens allowed by a grammar or schema, guaranteeing valid output.

Constrained (guided) decoding masks the logits so only tokens permitted by a formal grammar, regex, or JSON schema can be sampled, guaranteeing the output parses. It is how reliable structured output and JSON modes are enforced without hoping the model complies.

The maximum number of tokens a model can attend to at once — its working span of input plus output.

The context window is the hard cap on how many tokens (prompt + generated output) a model can process in a single pass. Everything outside it is invisible to the model, which is why long documents are chunked and agents need external memory. Larger windows cost more compute and KV-cache memory, roughly with length.

Resume pretraining on domain data before task fine-tuning to build domain fluency.

Continued pretraining (also: domain-adaptive pretraining, DAPT) continues the language model objective on a domain-specific corpus before instruction fine-tuning. This fills domain vocabulary into the model weights before any task-specific adaptation, improving downstream fine-tuning efficiency and final quality. The learning rate is typically lower than original pretraining to avoid catastrophic forgetting of the base model general capabilities.

Swapping requests in and out of a running batch every step to keep the GPU saturated.

Continuous (in-flight) batching removes finished sequences and adds new ones each step, instead of waiting for a whole batch to complete — dramatically improving serving throughput and latency.

Graduation by exhaustion — the model is done when the swarm can't break it anymore.

Rather than a fixed number of rounds, QuKaiZen runs until convergence: 95%+ evaluator scores, exhausted experiments, and no further reasoning gains. Quality is measured by swarm exhaustion, then verified by three gates.

A model graduates when the adversarial swarm gives up trying to break it — not at a fixed cycle limit.

Instead of a fixed number of rounds, QuKaiZen runs until convergence: 95%+ evaluator scores, exhausted experiments, and no further reasoning improvement. Quality is measured by swarm exhaustion, then verified by a three-gate certification before the Nucleus Seal is minted.

[BUILT] The SHA-256 hash pinning the compiled corpus — the DaC CD lockfile.

[BUILT] corpus_sha256 is the SHA-256 hash of the compiled terms.json (or bake corpus bundle) stamped by bake-corpus.mts. It serves as the Content-Delivery lockfile for the bake pipeline: any downstream consumer (Nucleus training run, model version tag) pins this hash to guarantee it is training on the exact same compiled corpus. Analogous to a package.lock — the corpus is reproducible and auditable. BUILT: bake-corpus.mts produces and writes this hash today.

Decay the learning rate along a cosine curve from its peak down toward zero over training.

A cosine learning-rate schedule ramps up during warmup, then decays the rate following a half-cosine from peak to a small final value. The smooth, front-loaded-then-gentle decay tends to train stably and finish in a good minimum; it is a default for large pretraining runs.

A measure of how aligned two vectors are by the angle between them — the standard relevance score for embeddings.

Cosine similarity is the cosine of the angle between two vectors, ranging from -1 to 1, ignoring their magnitudes. It is the default metric for comparing embeddings in retrieval and RAG: nearer angle, more semantically similar.

Attention where queries come from one sequence and keys/values from another.

In cross-attention the queries are drawn from one stream (e.g. the text being generated) while keys and values come from a different stream (e.g. an encoded image or source sentence). It is how encoder-decoder and multimodal models let one modality or sequence condition on another, in contrast to self-attention where all three come from the same sequence.

The standard LM loss: penalize the model by the negative log-probability it gave the correct token.

Cross-entropy measures the gap between the model's predicted distribution and the true distribution (a one-hot target for the actual next token). Minimizing it maximizes the log-likelihood of the data; exponentiating the mean cross-entropy gives perplexity. It is the workhorse loss for next-token prediction.

NVIDIA's platform/language for general-purpose GPU computing — the substrate most ML runs on.

CUDA is NVIDIA's parallel-computing API and toolkit that lets code run on GPUs. Frameworks compile their tensor ops down to CUDA kernels (and libraries like cuBLAS/cuDNN), which is why GPU availability and CUDA versions dominate ML ops.

Capture a fixed sequence of GPU operations once and replay it, eliminating per-step launch overhead.

CUDA Graphs record a static graph of GPU work and replay it as a single submission, removing the CPU-side kernel-launch overhead that otherwise dominates small, repetitive steps like token decoding. They meaningfully speed up low-latency inference.

Train on easier examples first, then progressively harder ones, like a teaching syllabus.

Curriculum learning orders training data from simple to complex instead of presenting it randomly, on the intuition that early easy examples build a foundation that makes hard examples learnable. It can speed convergence and improve final quality on tasks with a natural difficulty gradient.

Expand or vary training data with label-preserving transformations to improve robustness.

Data augmentation synthesizes additional training examples by transforming existing ones in ways that preserve meaning — paraphrasing, back-translation, noise injection for text; crops and flips for images. It enlarges effective dataset size and improves generalization, and in LLMs increasingly means generating synthetic data with another model.

When benchmark or test data leaks into training, inflating scores and invalidating the eval.

Data contamination happens when evaluation examples (or near-duplicates) appear in the training corpus, so high scores reflect memorization rather than ability. It is a serious threat to benchmark validity given web-scale training data, and is checked with n-gram overlap and canary strings.

Validation/test data has leaked into training — metrics are invalid.

Data leakage occurs when information from the validation or test split is visible during training, either through preprocessing that uses the full dataset (normalization statistics, tokenizer training) or through contaminated splits. The model learns to exploit the leaked information and achieves artificially high validation metrics that do not reflect real-world performance.

Replicate the model across devices, split the batch, and average gradients each step.

Data parallelism puts a full copy of the model on each device, feeds each a different slice of the batch, and synchronizes gradients (all-reduce) so all replicas stay identical. It is the simplest way to scale training throughput; ZeRO/FSDP shard the replicated state to save memory.

ReLU units stuck at zero — never activate, never learn.

A 'dead' ReLU neuron is one whose pre-activation is always negative, so it always outputs zero and receives no gradient. Once dead, the neuron cannot recover without reinitialization. Dead neurons reduce the effective capacity of the network. Caused by large negative weight initializations or by large learning rates that push weights into the negative region. Mitigated by using GELU or Leaky ReLU activations, or by careful initialization.

The token-by-token generation phase, bottlenecked by memory bandwidth rather than compute.

After prefill, decoding generates one token per step, each reading the entire KV-cache and weights — so it is memory-bandwidth bound, not compute bound. This is why KV-cache size, GQA, and quantization dominate generation speed.

The autoregressive transformer design used by most LLMs: predict the next token, attending only to the past.

A decoder-only model uses causal (masked) self-attention so each position can attend only to earlier tokens, and is trained to predict the next token. This single-stack design — no separate encoder — is what nearly all modern generative LLMs use, scaling cleanly and unifying understanding and generation in one objective.

Machine learning with many-layered neural networks that learn features automatically from raw data.

Deep learning uses neural networks with many layers so that early layers learn simple features and later layers compose them into abstract ones, removing the need for hand-engineered features. Depth plus large data and compute is what powers modern language and vision models.

The end state you declare; the system's job is to make reality match it.

Desired-state configuration means you describe what you want — the team, the config — not the steps to get there, and a controller reconciles reality to it. Idempotent and version-controlled.

Whether a model returns the same output for the same input every time — LLMs are non-deterministic by default.

A process is deterministic if identical inputs always produce identical outputs. LLM generation is non-deterministic by default: sampling (temperature, top-p) injects randomness, and even at temperature 0, floating-point order and parallel execution (batching, GPU kernels) can cause small variations. You make it near-deterministic with greedy decoding (temperature 0), a fixed random seed, and a pinned runtime.

Transfer a big teacher model's behavior into a small student model.

Knowledge distillation trains a small student to mimic a large teacher — matching its outputs, probabilities, or reasoning traces — so the student captures much of the teacher's capability at a fraction of the size and cost. It is the core of QuKaiZen's pipeline.

Training and deployment data have different distributions — model degrades at inference.

When the statistical distribution of inputs at deployment differs from the training distribution, model performance degrades. Types include covariate shift (input distribution changes), label shift (output distribution changes), and dataset shift (both). Common in fine-tuning: a model trained on one domain's text degrades on another. Continued pretraining on the target domain mitigates this.

Training loss climbs without bound instead of decreasing.

Loss increases monotonically or oscillates upward past warmup, often reaching inf or NaN. Distinct from a transient loss spike that self-recovers. Divergence means the optimizer is not converging to any useful basin — the run must be restarted from a checkpoint after the root cause is fixed.

QuKaiZen's framework: a declarative, curated source of truth that compiles into a knowledge app or bakes into a model you own.

Documentation as Code treats knowledge the way engineering treats code. You declare a theme and its trusted sources; an agent swarm gathers, compiles, curates, and gates it (every entry sourced) into a versioned source of truth; then you either serve it as an app or bake it into an owned model. This AI Dictionary is itself a derivative of DaC, one app built from one curated World. The same framework builds any themed knowledge product, or, through Nucleus, a Super Skill you own. The one lever you tune is the number of terms: a World can open as a 101-term primer and deepen, organically, toward an exhaustive corpus.

Specialize a general model to a target domain, often via continued pretraining on domain text.

Domain adaptation shifts a model toward a specific field (legal, medical, code) by continued pretraining and/or fine-tuning on in-domain data, raising in-domain quality while risking some general-ability loss. It is the bridge between a broad base model and a Super Skill specialist.

A model adapted to excel in one domain by fine-tuning, distillation, and domain-adaptive pretraining.

A domain-specialist model is a foundation model adapted via continued pretraining, fine-tuning, and/or distillation to specialize in a particular domain (medical, legal, ML engineering, horticulture). By trading general capability for domain depth, a specialist can outperform a much larger generalist on domain tasks. The core mechanism: a smaller model with deep domain grounding can match or exceed a larger generalist on in-domain benchmarks.

A LoRA refinement that decomposes weight updates into magnitude and direction for better quality.

Weight-Decomposed Low-Rank Adaptation splits each weight into a magnitude and a direction, applying the low-rank update to the direction while learning magnitude separately. It often closes the gap between LoRA and full fine-tuning at similar cost.

Quantize the quantization constants themselves to squeeze out extra memory, as in QLoRA.

Double quantization, introduced with QLoRA, quantizes the per-block scaling constants of an already-quantized model, saving a further fraction of a bit per parameter. The savings are small per value but meaningful across billions of parameters.

Align to preferences directly from good/bad answer pairs — no reward model or RL loop.

DPO skips RLHF's separate reward model and PPO loop, reframing alignment as a simple classification-style loss over (preferred, rejected) pairs that directly raises the likelihood of preferred answers. Simpler and more stable than PPO-based RLHF, with comparable results.

The small, fast model that proposes candidate tokens in speculative decoding.

The draft model is a smaller, cheaper model that guesses the next several tokens; the large target model then verifies them together. The closer the draft tracks the target, the more tokens are accepted per pass.

When the real state of a system diverges from its declared desired state over time.

Drift is the gap that opens when a running system changes out from under its specification — manual edits, partial failures, or external mutation leave reality and the declared desired state out of sync. Reconciliation loops detect drift and converge the system back to desired state; documentation-as-code treats drift in docs the same way.

Randomly zeroing activations during training to prevent overfitting.

Dropout randomly sets a fraction of activations to zero each training step, forcing the network not to rely on any single unit and improving generalization. It is disabled at inference. Large pretraining often uses little or none, but it is common when fine-tuning on small data.

Training data contains repeated or benchmark-contaminated examples.

Duplicate training examples cause the model to see certain patterns disproportionately, biasing the learned distribution. Contamination from benchmark or test data gives the model unfair advantage on evaluation and makes training metrics misleading. Large web-scraped corpora commonly have >10% duplication before deduplication. MinHash / n-gram deduplication is standard practice.

Halt training when validation performance stops improving, to avoid overfitting.

Early stopping monitors a held-out validation metric and stops (or rolls back to the best checkpoint) once it plateaus or worsens, even if training loss is still falling. It is a simple, effective regularizer.

A fast, modern public-key signature scheme used to cryptographically sign and verify artifacts.

Ed25519 is an elliptic-curve digital signature algorithm prized for speed, small keys and signatures, and resistance to common implementation pitfalls. It lets a producer sign an artifact with a private key so anyone can verify authenticity and integrity with the public key — the basis for tamper-evident model provenance and seals.

Exponential moving average of weights kept alongside training for a smoother, often better, final model.

An exponential moving average maintains a slowly-updated running average of the model's weights during training; the averaged weights are frequently more stable and generalize better than the raw final ones. It is cheap insurance widely used in large training runs.

Maps discrete token IDs to dense vectors — the model's vocabulary lookup table.

An embedding layer is a learned matrix of shape [vocab_size, d_model] that maps each integer token ID to a dense real-valued vector. It is the first layer of all transformer language models and is often tied (shared) with the output projection (lm_head). Embedding representations encode token semantics in a continuous space.

Dense numeric vectors representing tokens or text so similar meanings sit close together.

An embedding maps a token or piece of text to a vector in high-dimensional space where geometric closeness reflects semantic similarity. Models learn input embeddings for tokens; separate embedding models turn whole documents into vectors for search and RAG.

Capabilities that appear only past a certain model scale, absent in smaller models.

Emergent abilities are skills — multi-step reasoning, certain in-context learning, instruction following — that small models lack but larger ones display, sometimes appearing sharply with scale. Whether the sharpness is real or an artifact of how it's measured is debated, but the practical effect is that scaling unlocks qualitatively new behavior.

A two-stack design: an encoder reads the full input, a decoder generates output attending to it via cross-attention.

The original transformer is encoder-decoder: a bidirectional encoder builds a representation of the whole input, and an autoregressive decoder generates the output, using cross-attention to look back at the encoding. It suits transduction tasks like translation and summarization, where input and output are distinct sequences.

An agent's memory of specific past experiences — what happened, when, in which session.

Episodic memory stores concrete past events the agent lived through: prior conversations, tool calls and their results, successes and failures, each tied to its context. The agent retrieves relevant episodes to inform the current decision ('last time I tried X it failed'). It is the experiential, time-stamped counterpart to semantic memory's general facts.

One full pass of the optimizer over the entire training dataset.

An epoch is a complete sweep through all training examples. Small fine-tuning runs may use several epochs; large pretraining often uses roughly one pass over a huge corpus, since repeating data risks memorization. Tracking loss per epoch helps spot overfitting.

The practice of measuring model quality with repeatable tests — from public benchmarks to task-specific graders.

An eval is any repeatable measurement of how well a model does something: a public benchmark, a private held-out set, an LLM-as-judge rubric, or a unit-test-style check. Good evals are the steering wheel of model building — without them you cannot tell whether a change helped. QuKaiZen's certification gates are the evals a student model must pass before it graduates.

A single tracked training or evaluation run with a fixed configuration, used to test one change against a baseline.

An experiment isolates one variable — a hyperparameter, a data change, an architecture tweak — and measures its effect against a baseline under otherwise identical conditions. Each run logs its config, metrics, and artifacts so results are reproducible and comparable. In ARAIL, autoresearch agents run experiments continuously and score each against evolving rubrics — what gets measured gets improved.

How a sparse MoE assigns each token to a subset of experts so only part of the model runs per token.

Expert routing is the mechanism (usually top-k gating) that activates only a few of an MoE's many experts per token, giving large total capacity at small per-token compute. Balancing the routing so all experts are used is a central training challenge.

Gradient norms spike to very large values, destabilizing updates.

When gradients grow exponentially through deep or recurrent layers, parameter updates become destructively large, driving the loss toward divergence. Observable by logging gradient norms: a healthy run keeps them bounded; exploding gradients produce norm values orders of magnitude above baseline. The standard intervention is gradient clipping.

Whether a model's output is actually supported by its inputs or stated reasoning — not just plausible.

Faithfulness measures how well an output reflects its evidence: whether a summary is true to the source, whether a RAG answer is backed by the retrieved passages, and whether a chain-of-thought genuinely drives the final answer rather than being post-hoc rationalization. It is distinct from fluency or plausibility — an unfaithful answer can read perfectly while being unsupported.

The per-token two-layer MLP in each transformer block, where most parameters and stored knowledge live.

Each transformer block pairs attention (which mixes information across tokens) with a position-wise feed-forward network applied independently to every token: expand to a larger hidden dimension, apply a nonlinearity, project back. It holds the majority of a model's parameters and is widely viewed as where much factual knowledge is stored — and what MoE makes sparse.

Prompting a model with a handful of worked examples to demonstrate the desired task.

Few-shot prompting includes a small number of input-output examples in the prompt so the model infers the pattern and applies it to a new input — relying on in-context learning. It often sharply beats zero-shot on format-sensitive or unusual tasks, at the cost of longer prompts.

Continue training a pretrained model on new data to specialize it for a task or domain.

Fine-tuning takes a general pretrained model and trains it further on a focused dataset so it adapts to a domain, style, or task. It can be full (all weights) or parameter-efficient (LoRA/PEFT), and is the bridge from a generic base to a useful specialist.

Adapt a pretrained model to a target task or domain by continued gradient updates.

Fine-tuning initializes a model from pretrained weights and continues training on a task-specific or domain-specific dataset. Full fine-tuning updates all parameters; PEFT methods update only a small subset. Fine-tuning on too little data or for too many epochs risks catastrophic forgetting. Fine-tuning is the primary path from a general-purpose foundation model to a domain-specialist model.

An exact attention kernel that is fast and memory-light by never materializing the full attention matrix.

FlashAttention computes exact attention in tiles that stay in fast on-chip SRAM, avoiding the quadratic N-by-N matrix in slow HBM. It cuts memory from quadratic to linear and speeds up training and inference, enabling much longer contexts.

Accumulated rounding errors degrade model quality over many steps.

Every floating-point operation introduces a small rounding error. Over millions of training steps with many operations per step, these errors can accumulate into meaningful precision loss, particularly in optimizer accumulators (Adam's m and v tensors). Keeping optimizer state in fp32 (standard in mixed-precision training) mitigates this by providing a wider mantissa for accumulation.

Floating-point operations — the raw arithmetic count used to measure model and training cost.

FLOPs count the floating-point operations a computation requires; training cost is often quoted in total FLOPs and hardware in FLOP/s (per second). For a dense transformer, a forward pass is roughly 2 x parameters x tokens FLOPs, making it a handy back-of-envelope for cost.

fp16's limited dynamic range causes activations or gradients to overflow to inf.

Half-precision (fp16) has a maximum representable value of ~65504. When activations, loss values, or gradients exceed this, they overflow to inf, which propagates through the computation and produces NaN in the loss or weights. PyTorch's GradScaler addresses this by multiplying the loss by a large scale factor before the backward pass and dividing afterwards, keeping gradients in fp16 range. If the scale factor itself is too large, the scaled gradients overflow — producing the same NaN symptom.

An 8-bit floating-point format for faster training and inference on H100-class hardware.

FP8 represents numbers in 8 bits (e4m3 or e5m2 variants), halving memory and doubling throughput versus BF16 on supporting GPUs. It needs careful scaling but is increasingly used for both training and high-throughput inference.

Shards model parameters, gradients, and optimizer state across GPUs so huge models fit in training.

FSDP (PyTorch) splits parameters, gradients, and optimizer states across all data-parallel GPUs, gathering each shard only when needed. It trains models far larger than a single GPU's memory, with less overhead than older model-parallel schemes.

A structured protocol for a model to request a specific tool with typed arguments.

Function calling has the model emit a structured call — a name plus JSON arguments — that your code executes and returns, for the model to use. It's the reliable mechanism beneath most tool use.

The router in a mixture-of-experts that decides which experts handle each token.

In an MoE layer the gating network scores the experts for each token and routes it to the top-k, weighting their outputs. Its design governs load balance and quality; poor gating leaves experts under-used or overloaded.

A smooth activation function used in transformer feed-forward layers.

GELU multiplies an input by the probability it is positive under a Gaussian, giving a smooth, slightly negative-tolerant alternative to ReLU. Its smoothness helps gradient flow, and it is the default activation in many transformer MLP blocks (with SwiGLU now common too).

How well a model performs on new, unseen data rather than the data it trained on.

Generalization is the whole point of learning: a model that only fits its training set has memorized, not learned. It is measured on held-out data and improved with more/diverse data and regularization. The train-vs-test gap is the practical signal of how well a model generalizes.

Generator and discriminator trained adversarially — generator fools the discriminator.

A GAN (Goodfellow et al., 2014) consists of a generator (G) that produces samples from noise and a discriminator (D) that tries to distinguish real from generated samples. G is trained to fool D; D is trained to distinguish. The adversarial dynamic produces sharp, high-quality samples in well-designed architectures. Mode collapse (G finds a few samples that always fool D) is the canonical failure mode. Largely superseded by diffusion models for image generation.

The C/C++ tensor library underpinning llama.cpp, enabling efficient CPU and edge inference.

GGML is a lightweight tensor library written in C/C++ that powers llama.cpp, supporting quantized CPU/GPU inference with no heavy framework dependency. The GGUF file format is its model container.

A single-file binary format for quantized models, built for fast local inference (llama.cpp).

GGUF packs weights (usually quantized), tokenizer, and metadata into one memory-mappable file so a model loads fast and runs on commodity hardware. It is the format used by llama.cpp and friends, superseding the older GGML format.

A one-shot, layer-by-layer post-training quantization method that minimizes per-layer error using second-order info.

GPTQ quantizes a trained model to low bit-widths (e.g. 4-bit) one layer at a time, choosing rounded weights that minimize the layer's output error using approximate second-order (Hessian) information on a small calibration set. It made accurate 4-bit quantization of large models practical without retraining.

The vector of partial derivatives telling how the loss changes as you tweak each weight.

A gradient points in the direction of steepest increase of the loss; training steps move weights the opposite (descent) way. Gradient magnitude and stability (vanishing/exploding) are central concerns, handled with clipping, normalization, and good optimizers.

Sum gradients over several micro-batches before updating, simulating a large batch on limited memory.

Gradient accumulation runs several forward/backward passes, adding their gradients, and only then steps the optimizer — so a small GPU can train with a large effective batch size. It trades extra time for memory headroom.

Cap the gradient's magnitude each step to prevent exploding updates from destabilizing training.

Gradient clipping rescales the gradient when its norm exceeds a threshold, so a rare huge gradient can't blow up the weights. It is standard insurance for transformer training, where occasional spikes (from hard batches or numerical issues) would otherwise cause loss to diverge.

The core optimization: repeatedly step parameters in the direction that most reduces the loss.

Gradient descent computes the gradient of the loss with respect to the parameters and nudges them in the opposite (downhill) direction, iterating until the loss is low. Variants (SGD, AdamW) differ in how they estimate and scale that step. It is how essentially all deep models are trained.

High-variance gradient estimates slow convergence and require larger batches or LR tuning.

Stochastic gradient descent introduces gradient noise because each mini-batch is a sample of the full dataset gradient. At small batch sizes, this noise is high and limits the effective LR (linear scaling rule: halve the batch → halve the LR to keep stability). Data corruption, noisy labels, and large LR all amplify gradient noise. Gradient clipping and larger batches reduce its impact.

Always pick the single highest-probability next token — deterministic but can be repetitive.

Greedy decoding takes the argmax token at every step. It is deterministic and fast, ideal when you want reproducible or single 'best' answers, but it can get stuck in repetition and miss globally better sequences that require a locally lower-probability step (which beam search or sampling can reach).

Connecting an agent's language to the real world via tools, environments, or retrieved facts.

Grounding is how a language agent's words map onto reality: executing tools, observing an environment, or anchoring claims to retrieved sources. In CoALA, grounding actions are the external actions that affect or read the outside world, as opposed to internal reasoning. Ungrounded agents hallucinate; grounded ones can verify.

Share key/value heads across groups of query heads to shrink the KV-cache with little quality loss.

Grouped-query attention is the middle ground between full multi-head attention (one K/V per query head) and multi-query attention (one K/V for all). Query heads are partitioned into groups that share a single key/value head, cutting KV-cache memory and bandwidth — the main inference bottleneck for long contexts — while keeping most of MHA's quality. It is standard in recent large models.

A PPO-style RL method that drops the value network, scoring each sample relative to a group of samples for the same prompt.

Group Relative Policy Optimization estimates advantages by sampling a group of completions per prompt and comparing each to the group's average reward, removing the separate value (critic) model PPO needs. This makes RL fine-tuning cheaper and simpler, and it has been central to recent reasoning-model training.

Around 8,500 grade-school math word problems that test multi-step arithmetic reasoning.

GSM8K (Grade School Math 8K) is a dataset of about 8,500 linguistically diverse grade-school word problems, each needing two to eight reasoning steps. It became the standard probe for whether a model reasons step by step instead of pattern-matching — and the benchmark that made chain-of-thought prompting famous.

Runtime checks around a model that block, filter, or reshape unsafe inputs and outputs.

Guardrails are the deployment-time controls layered around a model — input/output classifiers, content filters, schema/format validators, and policy checks — that catch what alignment training missed. Unlike alignment baked into weights, guardrails are external, fast to update, and independently auditable.

When a model states fluent, confident information that is fabricated or unsupported.

Hallucination is the generation of plausible-sounding but false or ungrounded content — invented citations, wrong facts, fabricated details. It stems from models optimizing for likely text rather than truth. Retrieval grounding, verifiers, and calibration reduce it; benchmarks like HalluLens measure it.

A benchmark for measuring how often an LLM hallucinates — asserts unsupported or fabricated facts.

HalluLens is a hallucination benchmark that separates extrinsic hallucination (claims grounded in no source) from intrinsic hallucination (contradicting the given input), and probes models with tasks designed to surface confident-but-false answers. It exists because fluency hides unreliability — a model can sound right while being wrong.

Passing control and context from one agent to another so work continues without losing state.

A handoff transfers a task between agents — often via a committed artifact rather than chat memory — so a specialist picks up exactly where the previous one left off. Clean handoffs (explicit inputs and outputs) are what let multi-agent systems stay coherent and prevent context rot across a long pipeline.

Stanford's broad, multi-metric benchmark suite that scores models across many scenarios, not just accuracy.

HELM (Holistic Evaluation of Language Models), from Stanford CRFM, evaluates models across a wide matrix of scenarios and metrics — accuracy, calibration, robustness, fairness, bias, toxicity, and efficiency — so a model is judged on many axes at once instead of a single headline score.

The 'helpful, honest, harmless' framing of what an aligned assistant should be.

HHH summarizes three often-competing alignment goals: be helpful (actually assist), honest (don't deceive or hallucinate), and harmless (avoid harm). Much alignment work is about navigating their tensions — e.g. refusing a harmful request is harmless but less 'helpful' to that request.

The vector a model holds for each token at each layer — its evolving internal representation.

A hidden state is the intermediate activation vector for a token at a given layer, carrying the model's current understanding of that token in context. Hidden states are transformed layer by layer; the final layer's states are projected to logits. Probing and interpretability work studies what these vectors encode.

The hub and libraries (Transformers, Datasets, Hub) that are the de facto registry for open models.

Hugging Face hosts model and dataset repositories and maintains the Transformers, Datasets, and Tokenizers libraries that standardize loading and running models. It is where most open weights, including QuKaiZen-style releases, are published and pulled from.

A testable prediction you set out to confirm or refute with an experiment.

A hypothesis states, in advance, what you expect a change to do and how you'll measure it — turning a hunch into something falsifiable. Good research lives or dies on sharp hypotheses.

An extremely lightweight PEFT method that learns to rescale activations with a few vectors.

IA3 learns small per-feature scaling vectors that multiply keys, values, and FFN activations, freezing all original weights. It adds even fewer parameters than LoRA, making it attractive when many tasks must be stored cheaply.

An operation that produces the same result whether applied once or many times.

An idempotent operation can be repeated safely: applying it again on an already-correct system changes nothing. It is the property that makes reconciliation loops and declarative pipelines robust — you can re-run them after a crash or partial failure without compounding side effects or corrupting state.

A benchmark of machine-verifiable instructions that measures how precisely a model obeys format and constraint requests.

IFEval (Instruction-Following Eval) uses prompts whose compliance can be checked programmatically — answer in exactly three bullet points, avoid a given word, respond in JSON. Because each rule is machine-verifiable, it scores obedience objectively, with no human or judge model in the loop.

A model learns a task from examples in its prompt at inference time, with no weight updates.

In-context learning is the ability of large models to infer a task purely from instructions and examples placed in the prompt, adapting behavior without any gradient update. It is what makes few-shot prompting work and is an emergent property that strengthens with scale.

Use a larger effective batch size to stabilize gradient estimates and improve throughput.

A larger batch size provides a lower-variance gradient estimate, which can smooth convergence and allow a higher learning rate (linear scaling rule). When GPU VRAM prevents a large physical batch, gradient accumulation accumulates gradients over multiple forward passes before each optimizer step, achieving the same effective batch size. This also improves GPU utilization for small per-step batches.

Running a trained model to produce outputs — the deployment side, as opposed to training.

Inference is using a trained model to generate predictions for real inputs. For LLMs it is autoregressive: produce one token, append it, repeat. Latency, throughput, and memory (the KV-cache) are the central concerns, distinct from the one-time cost of training.

Fine-tune a base model on instruction-response pairs so it follows natural-language commands.

Instruction tuning is the SFT stage that turns a raw next-token base model into an assistant by training on many (instruction, good response) pairs across diverse tasks. It teaches the model to follow directions and generalize to unseen instructions before any alignment step.

4-bit integer weights — the aggressive quantization that makes big models fit on small hardware.

INT4 stores each weight in 4 bits (16 levels), roughly 8x smaller than FP32. Schemes like GPTQ, AWQ, and NF4 pick scales and zero-points to preserve quality. Small models tolerate 4-bit well; frontier models often need 8-bit for the same fidelity.

8-bit integer representation — a common, low-risk quantization that roughly halves memory versus 16-bit.

INT8 stores weights and/or activations as 8-bit integers with a scale factor, cutting memory and enabling fast integer matrix multiply on supported hardware. It is the conservative quantization choice: accuracy loss is usually negligible, unlike the more aggressive 4-bit formats. Mixed approaches keep sensitive parts in higher precision.

Distribution of layer activations shifts during training, slowing convergence.

As the weights of earlier layers change during training, the distribution of inputs to later layers shifts continuously, forcing later layers to constantly re-adapt. This was the original motivation for batch normalization (Ioffe & Szegedy, 2015). In practice, the term is used loosely to describe unstable activation distributions that slow convergence. Layer normalization addresses a similar problem for sequence models.

A DPO variant that adds regularization to avoid overfitting to deterministic preferences.

Identity Preference Optimization reformulates the preference objective to directly control how far the policy moves, addressing a DPO failure mode where near-deterministic preferences push the model to extremes. It is one of several offshoots refining direct preference optimization.

An input crafted to bypass a model's safety training and elicit disallowed behavior.

A jailbreak is a prompt — roleplay framing, obfuscation, or instruction-smuggling — that circumvents alignment to make the model produce content it would normally refuse. Jailbreaks are the offensive side of red-teaming and motivate layered guardrails beyond weight-level alignment.

The GGUF family of mixed-bit quantization schemes that allocate more bits to important weights.

K-quants are llama.cpp/GGUF quantization formats (Q4_K, Q5_K, Q6_K, etc.) that use a mix of bit-widths within a block, spending more bits on the parts of the weight matrix that matter most. They give better quality per byte than uniform low-bit quantization.

Combine multiple GPU operations into one kernel to cut memory round-trips and launch overhead.

Kernel fusion merges several elementwise or sequential operations into a single GPU kernel so intermediate results stay in fast on-chip memory instead of being written to and read from global memory. FlashAttention is a famous fused kernel; compilers like torch.compile fuse automatically.

QuKaiZen's agent that extracts certified, verifiable domain knowledge in six layers.

KICE mines a corpus for rare concepts, edge cases, historical conflicts, subsystem interactions, nuanced reasoning, and ambiguity — knowledge that can be verified against authoritative sources. It feeds the distillation pipeline with high-quality, checkable material.

A measure of how far one distribution is from another — used to keep an RL-tuned model near its base.

KL divergence quantifies how much one probability distribution diverges from a reference. In RLHF it is added as a penalty so the policy doesn't drift too far from the original (SFT) model while chasing reward, preventing reward hacking and gibberish. It also underlies distillation objectives that match a teacher's distribution.

An external, queryable store of facts and documents a model retrieves from instead of relying on weights alone.

A knowledge base is the curated, updatable corpus a retrieval system draws on — documents, facts, or embeddings indexed for search. It is the external memory that makes RAG work: keeping knowledge outside the model means it can be updated, cited, and audited without retraining. In CoALA terms it backs the agent's semantic memory.

Transfer knowledge from a large teacher model to a smaller student model.

Knowledge distillation (Hinton et al., 2015) trains a smaller student model to match the output distribution (soft targets/logits) of a larger teacher model, rather than hard labels. The teacher's soft predictions encode richer information about class relationships than one-hot labels. Distillation can significantly improve a small model's performance without access to the teacher at inference time.

Preference alignment from simple good/bad labels rather than paired comparisons.

Kahneman-Tversky Optimization aligns a model using per-example binary signals (this output was desirable or not) instead of A-vs-B pairs, drawing on prospect theory. It eases data collection since you needn't produce matched pairs.

Cached key/value tensors from past tokens so generation does not recompute the whole sequence each step.

During autoregressive generation each new token attends to all previous tokens. The KV-cache stores the keys and values already computed, so each step only processes the new token — turning quadratic regeneration into linear. It is the main consumer of inference memory.

Soften one-hot targets slightly so the model doesn't become over-confident.

Label smoothing replaces hard 0/1 targets with values like 0.9/0.1 spread over classes, discouraging the model from driving any probability to extremes. It improves calibration and generalization and connects conceptually to the soft targets used in distillation.

The delay before and during a model's response — time-to-first-token and per-token time.

Latency is how quickly a single request responds, distinct from throughput (total volume). Keeping the model warm and prefetching weights cut it.

The learned, compressed vector space in which a model represents meaning.

Latent space is the high-dimensional space of a model's internal representations, where semantically similar inputs land near each other. Embeddings live in latent space; arithmetic and similarity there power retrieval, clustering, and interpolation.

Normalizes activations across the feature dimension within each example.

Layer normalization (Ba et al., 2016) normalizes activations across the feature dimension (not the batch dimension), computing mean and variance per-example, per-layer. This makes it suitable for sequence models where batch normalization is inapplicable (variable-length sequences, small batch sizes). Standard in all transformer architectures. Applied before or after each sub-layer (Pre-LN vs Post-LN, with Pre-LN being more stable for deep models).

Load one transformer layer from disk, compute, discard — running 400B+ models on tiny VRAM.

Layer-streaming inference (AeroLLM's core primitive) streams a model layer by layer from SSD: load a layer's weights, compute, free, repeat. It trades latency for the ability to run frontier-scale teachers (70B-671B) on commodity hardware with a few GB of VRAM.

Normalizes activations within each layer to keep training stable; modern LLMs often use RMSNorm.

Layer normalization rescales each token's activation vector to zero mean and unit variance (RMSNorm skips the mean), stabilizing and speeding training. Placement (pre-norm vs post-norm) and the variant chosen materially affect deep-transformer stability.

How big a step the optimizer takes down the gradient — the most consequential training hyperparameter.

The learning rate scales each weight update. Too high and training diverges or oscillates; too low and it crawls or sticks in poor regions. It is usually warmed up, then decayed (e.g. cosine) over training. Picking and scheduling it well is often the difference between a model that converges and one that doesn't.

A plan for how the learning rate changes over the course of training.

Rather than using a fixed LR, schedules vary the rate over time. Common schedules: linear warmup + linear decay; cosine annealing (LR follows a cosine curve to a near-zero minimum); step decay (multiplies LR by a factor every N steps); constant (no decay, only warmup). The HF Trainer supports these via `lr_scheduler_type`. The schedule interacts with the optimizer and batch size; getting it wrong causes plateaus or oscillation.

Peak LR exceeds what the schedule/optimizer can stabilize.

A peak learning rate too large for the warmup length and batch size drives parameter updates past the stable basin, producing divergence or oscillation. The relationship between LR and batch size is roughly linear (linear scaling rule): larger batches tolerate larger LRs. A 1B+ parameter model with a 100-step warmup is especially sensitive because the model is not yet pre-conditioned. The fix is to reduce the peak LR and/or lengthen the warmup.

LR is so small that the optimizer barely moves — training stalls.

When the learning rate is too low, gradient updates are so small that the model barely changes per step. The loss either plateaus prematurely or converges too slowly to be useful within the compute budget. Often set accidentally when copying a LR from a much larger batch-size run without rescaling, or when a cosine schedule decays to near-zero too quickly.

A lean C/C++ inference engine that runs quantized LLMs efficiently on CPUs, Macs, and modest GPUs.

llama.cpp is a portable, dependency-light engine built on GGML that popularized running quantized models (via GGUF/k-quants) on commodity hardware, including Apple Silicon and CPUs. It made local LLM inference broadly accessible.

A transformer trained on vast text to predict the next token, yielding broad language ability.

A large language model is a big transformer trained on internet-scale text with next-token prediction; scale plus instruction tuning yields general capability. QuKaiZen distills that capability into small, owned models.

The model's raw, unnormalized output scores over the vocabulary, before softmax makes them probabilities.

Logits are the final layer's raw scores — one per vocabulary token — not yet normalized into probabilities. Sampling controls (temperature, top-k/p) operate on logits before softmax converts them into the next-token distribution.

An agent's durable store that survives across sessions, beyond the context window.

Long-term memory is any persistent store the agent reads from and writes to across runs — usually an external database or vector index holding episodic and semantic memories. It is the answer to the context window's hard limit: instead of cramming everything into the prompt, the agent retrieves only what's relevant now. Writing, organizing, and forgetting are first-class problems.

Fine-tune a model by training tiny low-rank adapter matrices while the base weights stay frozen.

LoRA freezes the original weights and injects small trainable rank-decomposition matrices into each layer. You train only those low-rank matrices — often under 1% of the parameters — which slashes memory and lets a single GPU fine-tune models that would otherwise need a cluster.

The scalar that measures how wrong a model's predictions are — what training minimizes.

The loss function turns a batch of predictions and targets into a single number quantifying error; training adjusts weights to reduce it via gradient descent. For language models it is almost always cross-entropy over next-token predictions. The choice of loss defines what 'good' means to the optimizer.

Loss stops improving for many steps — training is stalled.

A plateau means the optimizer is stuck: the learning rate may be too low to escape a saddle point or local minimum, the schedule may have decayed too aggressively, the data may be exhausted, or the model has no more capacity. It differs from convergence (which is intentional) by occurring earlier than expected and being confirmed by no improvement on held-out loss.

A sharp, transient jump in loss that may or may not recover.

A brief jump in training loss — often 2–10× the running baseline — that either recovers within a few hundred steps (a recoverable spike) or becomes a divergence. Spikes correlate with bad batches, data contamination, or a learning rate that is at the boundary of instability. Distinguishing recoverable from diverging requires observing the trend after the spike.

An open standard for connecting models to tools and data sources.

MCP lets agents discover and call external tools, resources, and data through a uniform interface, so capabilities plug in without bespoke glue per integration.

How fast data moves between memory and compute — the usual bottleneck for LLM inference.

Memory bandwidth is the rate at which weights and the KV-cache can be read from device memory. Because LLM decoding reads huge amounts of data per token while doing relatively little math, it is bandwidth-bound — which is why quantization and smaller KV-caches speed it up more than raw FLOPs.

A time-ordered log of an agent's observations, scored by recency, importance, and relevance for retrieval.

Popularized by the 'generative agents' work, a memory stream is an append-only list of natural-language memory records. To act, the agent retrieves a subset ranked by a blend of recency, importance, and relevance to the current situation, and periodically synthesizes higher-level reflections back into the stream.

Model FLOPs Utilization — the fraction of a chip's peak FLOP/s your training actually achieves.

Model FLOPs Utilization is realized useful FLOPs divided by hardware peak, a single number for how efficiently a training run uses its accelerators. Real large-scale runs often land in the 30-50% range; raising MFU directly cuts cost and time.

Use lower precision for most math but keep sensitive parts in higher precision for stability.

Mixed-precision computation runs the bulk of operations in a low-precision format (FP16/BF16/FP8) for speed and memory while keeping numerically sensitive pieces — master weights, accumulations, certain norms — in higher precision. It is standard for both training (with loss scaling) and inference, capturing most of the speedup without the instability of going fully low-precision.

Use fp16 or bf16 for forward/backward passes while keeping fp32 master weights.

Mixed-precision training stores model weights as fp32 (master copy) but performs forward and backward passes in fp16 or bf16. This approximately halves memory footprint for activations and tensors, and speeds up compute on hardware with fp16/bf16 tensor cores. fp16 requires a loss scaler (GradScaler) to avoid underflow; bf16 does not (wider dynamic range). Most modern GPU fine-tuning uses bf16 or fp16 with AMP.

Apple's array framework for running and training models on Apple Silicon's unified memory.

MLX uses the shared CPU/GPU memory of Apple Silicon for zero-copy inference and fine-tuning — no host↔device transfers and much lower power. AeroLLM targets it for Mac deployments.

A benchmark of ~16,000 multiple-choice questions across 57 subjects, measuring an LLM's breadth of knowledge.

MMLU (Massive Multitask Language Understanding) tests a model with four-option multiple-choice questions spanning 57 subjects, from elementary math to law, medicine, and ethics. It is the standard yardstick for general knowledge and reasoning breadth; scores range from 25% (random guessing) to roughly 90% for frontier models.

Generator produces only a few outputs — diversity collapses.

In generative models (GANs, VAEs, certain RL fine-tuning setups), mode collapse is when the model learns to generate a narrow subset of valid outputs. The discriminator or reward model can be fooled by the same outputs repeatedly. In GAN training, the generator finds a 'safe' mode that always fools the discriminator and stops exploring. In RLHF, reward hacking produces similar behavior — the model finds a narrow pattern that maximizes reward without being generally helpful.

Combine multiple fine-tuned models into one by arithmetic on their weights, no extra training.

Model merging blends the weights of several models (often fine-tunes of a shared base) into a single model that inherits multiple skills, using averaging, SLERP, or task-vector arithmetic. It is a cheap way to fuse capabilities and mitigate catastrophic forgetting.

A model split into many expert sub-networks where a router activates only a few per token.

Mixture-of-Experts replaces a dense layer with many parallel expert networks plus a router that picks a small subset (e.g., 2 of 64) per token. Total parameters balloon while compute per token stays modest — huge capacity at a fraction of dense FLOPs.

Several specialized agents collaborating, each owning a function.

A multi-agent system splits work across specialist agents that hand off to one another — often cheaper and more reliable than one giant generalist. PaperAgents reconciles a declared team of them.

Run several attention operations in parallel, each in its own subspace, then concatenate.

Multi-head attention splits the model dimension into several 'heads', each with its own learned query/key/value projections, runs attention independently per head, and concatenates the results. Different heads specialize — some track syntax, some long-range coreference — letting one layer attend to multiple kinds of relationship at once.

All query heads share a single key/value head — the most aggressive KV-cache reduction.

Multi-query attention keeps many query heads but collapses to one shared key and value projection. This minimizes KV-cache size and memory bandwidth during decoding, dramatically speeding long-context inference, at some cost to quality — which grouped-query attention later recovered.

Models that take in or produce more than one kind of data — text, images, audio, video.

A multimodal model represents multiple data types in a shared space so it can, e.g., answer questions about an image or caption a video. Typically a modality encoder maps non-text inputs into tokens the language model attends to via cross-attention.

A contiguous sequence of n tokens; the basis of pre-neural language models and still used for metrics.

An n-gram is a run of n consecutive tokens (bigram = 2, trigram = 3). Classic language models estimated the probability of the next token from n-gram counts. Today n-grams persist in evaluation metrics (BLEU, ROUGE) and in detecting training-data overlap.

Loss value becomes Not-a-Number — the run is numerically broken.

A NaN in the loss typically means a numerical overflow or a division by zero somewhere in the forward pass or loss computation. In fp16/bf16 mixed-precision training this commonly traces to a loss scale overflow. Once a NaN propagates into gradients, the optimizer corrupts model weights and the run must be restored from a last-good checkpoint.

Layers of simple weighted units that transform inputs into outputs, learning the weights from data.

A neural network stacks layers of units (neurons), each computing a weighted sum of its inputs followed by a nonlinearity. Training adjusts the weights via gradient descent so the network maps inputs to desired outputs. Transformers are a specific, attention-based neural-network architecture.

A 4-bit 'normal float' data type, used in QLoRA, tuned for the bell-curve distribution of weights.

NF4 (4-bit NormalFloat) is an information-theoretically motivated 4-bit format whose quantization levels match the roughly normal distribution of neural-network weights, giving lower error than uniform 4-bit. It is the storage format behind QLoRA.

Training data contains incorrectly labeled examples — the model learns corrupted signal.

Label noise means some fraction of training examples have incorrect ground-truth labels. The model attempts to fit these incorrect labels, wasting capacity and potentially degrading generalization. In instruction tuning, low-quality completions act as noisy labels. Label smoothing provides a partial defense by preventing the model from fitting labels with full confidence.

[ROADMAP] QuKaiZen's training pipeline for baking domain-specialist SLMs.

[ROADMAP] Nucleus is the QuKaiZen training infrastructure that takes a baked corpus (compiled World + corpus_sha256 manifest) and runs the fine-tuning/distillation pipeline to produce a sealed domain-specialist SLM. The training run lives on the M5 (Apple Silicon); engine-side plumbing (corpus preparation, bake-corpus.mts) exists today, but the full end-to-end Nucleus bake pipeline is in development. ROADMAP because no sealed specialist SLM has been produced yet.

An Ed25519 cryptographic provenance chain proving how a Super Skill model was made.

The Nucleus Seal binds a model's DNA — teacher hash, corpus hash, pipeline config, audit, and AutoResearch report — into a signed Ed25519 chain. It is cryptographic proof the pipeline distilled the model correctly, and seals are dynamically monitored and revocable.

Values exceed the representable range and become inf — NaN propagates downstream.

Numerical overflow is the counterpart to underflow: a value grows beyond the maximum representable number for the floating-point format and becomes inf. inf in any computation typically produces NaN (inf - inf = NaN, inf × 0 = NaN). In fp16 training, this is the dominant source of NaN loss. In fp32 training it is rare except with very high LR or unnormalized weights.

Values become too small to represent and round to zero — silent precision loss.

Numerical underflow occurs when a floating-point computation produces a value smaller than the minimum representable normal number for the format, causing it to round to zero (or to a subnormal). In fp16, the minimum normal is ~6e-5. Log-probabilities and softmax computations are most vulnerable. Underflow in gradients causes them to vanish silently — the model stops learning without any error message.

A local runtime that packages and serves models with one command, built on llama.cpp.

Ollama wraps model download, quantization, and serving behind a simple CLI and local API, making it easy to run open models on a personal machine. QuKaiZen uses Ollama on its VM to power on-box generation features.

Teacher and student train together at the same time instead of distilling from a frozen teacher.

In online (or co-) distillation there is no pre-trained frozen teacher: a cohort of models trains simultaneously and each learns from the others' current predictions. It removes the separate teacher-training phase and can scale across many workers, with each worker's model acting as a peer teacher.

An open, framework-neutral format for exchanging models between training and inference runtimes.

ONNX (Open Neural Network Exchange) is a portable graph format that lets a model trained in one framework run in another or in optimized runtimes (ONNX Runtime, TensorRT). It decouples authoring from deployment.

Coordinating multiple agents or services into one coherent flow.

Orchestration sequences and supervises the parts of a multi-step system — who runs when, with what inputs — handling retries and handoffs. QuKaiZen orchestrates the swarm; PaperAgents orchestrates a team.

A single-stage method that combines instruction tuning and preference alignment without a separate reward model or reference model.

Odds Ratio Preference Optimization folds preference alignment into SFT by adding an odds-ratio penalty on dispreferred responses, removing the need for a separate reward model and reference model. It simplifies the alignment pipeline into one stage.

Loss bounces between high and low values without a clear downward trend.

When the loss oscillates — alternating high and low values — rather than following a smooth descent, the learning rate is typically too large for the batch size or the optimizer is not suited to the curvature. Oscillation differs from noise (random variation around a trend) by having a regular pattern. Reducing the LR or switching to a more adaptive optimizer usually smooths it.

GPU runs out of VRAM — the process crashes with a CUDA OOM.

A CUDA out-of-memory error means the model, activations, gradients, and optimizer states together exceed the available GPU VRAM. OOM can be triggered by a large batch, a large sequence length, or optimizer states (Adam keeps 2 extra fp32 copies per parameter). Solutions involve reducing batch size, using gradient accumulation to maintain effective batch size, or switching to more memory-efficient training (mixed precision, gradient checkpointing).

When a model memorizes training-set quirks and fails to generalize to new data.

Overfitting is the gap between strong training performance and weak performance on unseen data: the model has fit noise and idiosyncrasies rather than the underlying pattern. It is diagnosed by a diverging train-vs-validation curve and countered with more data, regularization, or a smaller model.

Storing the KV-cache in non-contiguous pages so long contexts fit without waste.

PagedAttention (from vLLM) manages attention key/value cache in fixed-size pages like virtual memory, eliminating fragmentation and letting many requests share memory — large serving-throughput gains.

A single learned number in a model; their count (e.g. 7B) is the rough measure of model size.

Parameters are the model's learned values — the weights and biases adjusted during training. Their total count (billions for modern LLMs) is shorthand for capacity and largely sets memory footprint: at 16-bit, each parameter is two bytes, so a 7B model needs ~14GB just to hold weights. Quantization shrinks the bytes per parameter, not their number.

An umbrella for methods (LoRA, adapters, prefix-tuning) that tune a tiny fraction of parameters.

PEFT covers techniques that adapt a model by training only a small set of new or selected parameters while freezing the rest — LoRA, adapters, prefix/prompt tuning, and more. It is also the name of Hugging Face's library implementing them.

A measure of how surprised a model is by text — lower means it predicts the text better.

Perplexity is the exponentiated average negative log-likelihood a model assigns to a sequence — roughly the effective number of equally likely choices it faces each step. Lower is better, but it is an intrinsic metric, not a substitute for task benchmarks.

Place different layers on different devices and stream micro-batches through them like an assembly line.

Pipeline parallelism splits the model by layer across devices; micro-batches flow through the stages so multiple are in flight at once. Scheduling matters — naive pipelines waste time in 'bubbles' while stages wait. It complements data and tensor parallelism in large-scale training.

An agent breaks a goal into an ordered set of subtasks before (or while) acting.

Planning is the internal action of decomposing a high-level goal into steps and sequencing them, optionally revising the plan as observations arrive. Approaches range from plan-then-execute (fix the whole plan up front) to interleaved planning (replan each step, as in ReAct). Good planning keeps long-horizon tasks coherent.

Information added to tokens so the otherwise order-blind transformer knows their sequence positions.

Attention is permutation-invariant — it sees a bag of tokens — so models inject position information via positional encodings: fixed sinusoids, learned embeddings, or rotary methods (RoPE) that rotate query/key vectors by position. The choice strongly affects how well a model extrapolates to longer contexts than it trained on.

Quantize a trained model without further training — fast but some quality loss.

Post-training quantization (PTQ) converts a trained fp16/fp32 model to a lower-bit format (int8, int4) without any additional training. It requires a small calibration dataset to compute quantization scales. PTQ is faster and simpler than QAT but trades some quality for convenience. GPTQ and bitsandbytes NF4 are popular PTQ methods for LLMs.

VAE latent variables collapse to the prior — the encoder becomes useless.

In variational autoencoders (VAEs), posterior collapse occurs when the decoder learns to ignore the latent code entirely, generating outputs from the prior alone. The KL divergence term in the ELBO objective drives the posterior toward the prior, and if the decoder is expressive enough, it learns to do without the latent information. Addressed by KL annealing, free bits, or beta-VAE weighting.

The RL algorithm classically used to optimize a model against a reward model in RLHF.

PPO is a policy-gradient method that improves a model while clipping each update to stay close to the previous policy, preventing destructive jumps. In RLHF it is the optimizer that pushes the model to maximize reward-model scores.

Datasets of 'A is better than B' human judgments used to train reward models or do DPO.

Preference data consists of prompts paired with two or more candidate responses and a human (or AI) judgment of which is better. It is the raw material for reward modeling and for direct methods like DPO, encoding the values and quality bar the model should be aligned to.

Loading the next layer from disk while the current compute runs, hiding I/O latency.

Prefetching overlaps disk reads with computation: while the GPU works on layer N, layer N+1 is already streaming in, so the model rarely waits on storage. It's what makes layer streaming fast.

The compute-heavy first phase where the model ingests the whole prompt in parallel.

Prefill processes all prompt tokens at once to build the KV-cache before generation begins; it is compute-bound and largely sets the time to first token. It contrasts with the memory-bound decode phase that emits tokens one at a time.

Prepend trainable key/value vectors to every layer's attention, freezing the base model.

Prefix tuning learns task-specific key/value 'prefixes' injected into each attention layer while the model stays frozen. It is more expressive than input-only prompt tuning because it influences every layer, and remains parameter-efficient.

The first, largest training stage: learn general language/knowledge from a huge unlabeled corpus.

Pretraining trains a model from scratch on a massive, mostly unlabeled corpus with a self-supervised objective (usually next-token prediction). It produces a 'base model' with broad knowledge and capabilities but no instruction-following polish; later stages (SFT, alignment) specialize it. It dominates the total compute budget.

An agent's memory of how to do things — its skills, routines, and the agent code itself.

Procedural memory is knowledge of *how* to act: learned skills, reusable routines, and in CoALA the agent's own implementation (its prompts, tools, and decision logic). Some of it is implicit in the model's weights; some is explicit, editable code or saved skills the agent can extend over time. It is the 'muscle memory' versus episodic/semantic's 'facts'.

A reward model that scores each step of a reasoning chain, not just the final answer.

A process reward model (PRM) evaluates the intermediate steps of a solution, rewarding correct reasoning along the way, in contrast to an outcome reward model that judges only the end result. Step-level signal improves reasoning training and verification.

The input text you give a model to steer what it does.

A prompt is the instruction plus context handed to a model at inference time. Prompt engineering tunes wording, examples, and structure to elicit better output — but unlike training, it never changes the model's weights.

Provider-side cache that bills a repeated prompt prefix at a fraction of fresh-input cost on cache hit.

Prompt caching marks part of a request (typically the system prompt or a stable conversation prefix) with cache_control: ephemeral so the provider keeps a hashed copy. Subsequent requests with the same prefix bill as cache_read tokens — much cheaper than fresh input — while the volatile remainder is processed normally. It is API-side at the provider, distinct from the in-process KV-cache. Each model has a minimum cacheable prefix (e.g. 2048 tokens on Claude Sonnet 4.x); below that floor a well-behaved client omits the marker entirely.

An attack where untrusted input smuggles instructions that override the system's intended ones.

Prompt injection hides adversarial instructions in content the model ingests (a web page, a document, tool output) to hijack its behavior — exfiltrate data, ignore policy, or misuse tools. It is the defining security risk for agents that read untrusted data and is distinct from jailbreaks, which target the user-facing prompt.

Learn a small set of continuous 'soft prompt' vectors while freezing the model, to steer behavior cheaply.

Prompt tuning prepends a handful of trainable embedding vectors to the input and trains only those, leaving the model frozen. It is among the most parameter-light adaptations, storing just the soft prompt per task, though it is usually less expressive than LoRA.

A verifiable record of exactly what went into a model and how it was built.

Provenance is the chain of custody for a model: which teacher, which corpus version, which config and audits. QuKaiZen hashes each artifact and signs the chain so the lineage is tamper-evident.

The dominant deep-learning framework for research and much production, built on eager Python tensors.

PyTorch provides tensors, autograd, and neural-network building blocks with a define-by-run (eager) model that is easy to debug, plus torch.compile for speed. It is the framework most models are trained and released in.

Quantization-aware training: simulate low precision during training so the model learns to tolerate it.

Quantization-aware training inserts fake-quantization ops during training so weights and activations adapt to the eventual low-bit format, usually beating post-training quantization on accuracy at the cost of a training run. Used when the last points of quality matter.

LoRA on top of a 4-bit quantized base model — fine-tune big models on one consumer GPU.

QLoRA quantizes the frozen base to 4-bit (NF4) to shrink its footprint, then trains LoRA adapters on top in higher precision, with gradients flowing through the quantized weights via dequant-on-the-fly. Near-full-fine-tune quality at a fraction of the VRAM.

Storing weights/activations in fewer bits (FP16 to INT4) to shrink models and speed inference.

Quantization maps high-precision weights to a smaller numeric type (8-bit, 4-bit, ...) using a scale and zero-point, trading a little accuracy for big savings in memory and bandwidth. It is what lets frontier-scale models run on commodity hardware.

Train with simulated quantization so the model adapts to the reduced precision.

QAT inserts simulated quantization operations (fake quantization) during training, so the model learns to be robust to the quantization error. The gradients flow through the fake-quantize operations via the straight-through estimator. QAT recovers quality lost in PTQ at the cost of an additional training pass, and is preferred when deployment quality matters more than conversion speed.

Fine-tuning that teaches a model to reason over retrieved docs while ignoring distractors.

RAFT trains on a question plus a mix of oracle (relevant) and distractor (irrelevant) documents, teaching the model to cite the right source and ignore noise. The result reasons through imperfect retrieval rather than memorizing — domain-specific RAG baked into the weights.

Fetch relevant documents at query time and feed them to the model as context.

RAG retrieves passages from a knowledge store and injects them into the prompt, so the model answers from fresh, specific data rather than memory. It's the opposite of distillation — knowledge stays external and looked-up.

An agent pattern that interleaves reasoning steps ('thoughts') with actions ('tool calls') in a loop.

ReAct prompts a model to alternate between reasoning traces and concrete actions: think, act (call a tool or query the environment), observe the result, think again. Interleaving reasoning with grounded actions lets the agent plan, gather information, and correct course mid-task — the backbone pattern of most tool-using agents.

A model working through a problem in intermediate steps instead of answering in one leap.

Reasoning is a model's ability to chain intermediate inferences — premises, rules, constraints, cross-references — toward a conclusion, rather than pattern-matching a final answer. Chain-of-thought elicits it; distillation transfers and sharpens it into small models.

Continuously closing the gap between the team you declared and the team that's running.

Borrowed from infrastructure (Kubernetes-style control loops), reconciliation compares desired state to observed state and converges them; a watcher fixes drift forever after. PaperAgents applies it to agent teams.

Deliberately probing a model with adversarial inputs to surface harmful, unsafe, or broken behavior.

Red-teaming stress-tests a model by actively trying to make it fail — eliciting harmful content, jailbreaks, leaks, or unsafe tool use — so the gaps can be fixed before deployment. It can be manual, automated (one model attacking another), or continuous, and feeds both training data and guardrail design.

Lower the peak LR (and/or lengthen warmup) to restabilize.

Reduce peak LR by 2–10× and/or extend the warmup period; re-run from the last good checkpoint to confirm loss resumes its downward trend. This is the primary intervention for learning-rate-too-high producing divergence or oscillation. The new LR should be confirmed by observing a stable descent for at least a few thousand steps before committing to the full run.

An agent reviews its own past actions or outputs and writes higher-level lessons or corrections.

Reflection is an internal action where the agent examines its recent trajectory — outcomes, errors, retrieved memories — and produces a higher-level insight, critique, or revised plan that feeds future decisions. It turns raw episodes into reusable lessons and is a core self-improvement loop in agent frameworks.

An agent loop that converts failure feedback into written self-reflection stored in memory for the next attempt.

Reflexion is an agent method where, after a failed attempt, the agent generates a verbal self-reflection on what went wrong and stores it in episodic memory. On the next attempt that reflection is added to the context, so the agent improves over trials without updating any weights — reinforcement via language, not gradients.

Any technique that constrains a model to generalize better rather than memorize the training set.

Regularization covers methods that trade a little training-set fit for better generalization: weight decay, dropout, data augmentation, early stopping, and label smoothing among them. The goal is to reduce overfitting so the model performs on unseen data, not just the data it saw.

Sample many answers, keep only the ones that pass a check, then fine-tune on the survivors.

Rejection-sampling fine-tuning generates many candidate completions per prompt, filters them with a verifier, reward model, or ground-truth check, and trains the model on the accepted ones. It is a simple, stable alternative to RL for self-improvement and underpins much self-distillation.

Rectified Linear Unit — max(0, x). The most common hidden-layer activation.

ReLU (Rectified Linear Unit) applies max(0, x) element-wise, outputting zero for negative inputs and the input itself for positive inputs. It is computationally cheap and empirically effective for many architectures. Its main failure mode is 'dead neurons' — units that always receive negative input and therefore always output zero, ceasing to learn. GELU has largely replaced ReLU in transformer feed-forward layers.

A decoding adjustment that lowers the probability of tokens already generated, reducing loops.

Repetition (and the related frequency/presence) penalties down-weight tokens that have already appeared, discouraging the model from looping or echoing itself. They are post-logit adjustments applied at sampling time, tuned to avoid both repetition and unnatural avoidance.

Systematic inquiry — forming hypotheses, running experiments, and measuring results.

Research is the disciplined loop of asking a question, forming a hypothesis, experimenting, and measuring. ARAIL is built to run that loop with AI: autoresearch agents gather sources, probe ideas, and surface what's interesting.

Add a layer's input to its output so gradients and signal can flow straight through deep stacks.

A residual (skip) connection routes a sublayer's input around it and adds it back to the output, so each block learns a delta on top of identity. This keeps gradients from vanishing in very deep networks and is, with layer normalization, what makes 100+-layer transformers trainable.

Roll back to a saved state before the failure and restart with corrected hyperparameters.

After a divergence or NaN, the corrupted model weights must be discarded. Save checkpoints frequently during training (e.g., every 500–1000 steps) so that the last-good checkpoint is a short rollback away. Restore the checkpoint, fix the root cause (LR, clipping, precision settings), and resume. HF Trainer handles checkpoint save and resume automatically when `save_steps` and `resume_from_checkpoint` are set.

When a model maximizes the reward signal in unintended ways that don't reflect true quality.

Reward hacking (specification gaming) happens when the policy finds shortcuts that score high under an imperfect reward model without actually being good — verbosity, flattery, or exploiting reward-model blind spots. It is the central failure mode that KL penalties and better reward models try to contain.

A model trained to score outputs by human preference, providing the reward signal for RLHF.

A reward model is trained on human comparisons (A is better than B) to predict a scalar quality score for any output. In RLHF this learned reward stands in for expensive human feedback, guiding the policy model via PPO or similar. Its accuracy and robustness to gaming bound the quality of the aligned model.

Like RLHF, but the preference labels come from an AI judge instead of (or alongside) humans.

Reinforcement Learning from AI Feedback replaces human preference labels with judgments from a capable model, often guided by a written set of principles. It scales alignment data far beyond what human annotation allows and is the mechanism behind constitutional approaches; quality hinges on the judge model and the principles it follows.

Align a model to human preferences via a reward model trained on human rankings, then RL.

RLHF collects human comparisons of model outputs, trains a reward model to predict which response people prefer, then fine-tunes the policy with reinforcement learning (usually PPO) to maximize that reward. It is how raw pretrained models became helpful, harmless assistants.

A lighter normalization that scales activations by their root-mean-square, without subtracting the mean.

RMSNorm normalizes a vector by its root-mean-square and a learned scale, skipping LayerNorm's mean-centering and bias. It is cheaper and empirically as effective, so most recent large models use it in place of LayerNorm.

Encodes token position by rotating query/key vectors — the dominant positional scheme in modern LLMs.

Rotary Position Embeddings inject position by rotating query and key vectors by an angle proportional to their position, so attention naturally depends on relative distance. RoPE extrapolates to longer contexts better than learned absolute embeddings and underlies most current LLMs.

The evolving scoring criteria AutoResearch uses to probe and grade the student.

Rubrics are structured criteria that drive the Interrogator's probes, the Adversary's traps, and the Evaluator's scoring. AutoResearch evolves them over time as new failure modes are discovered.

A safe, fast, zero-copy tensor file format — the modern replacement for pickle-based checkpoints.

SafeTensors stores weights in a simple, memory-mappable layout with no arbitrary code execution (unlike Python pickle, which can run malicious code on load). It loads fast via zero-copy and is now the default for sharing weights on the Hub.

Drawing the next token randomly from the model's probability distribution rather than always taking the top one.

Sampling selects each next token by drawing from the model's predicted distribution (often after temperature, top-k, or top-p shaping), introducing controlled randomness. It produces more diverse, natural text than greedy decoding and is the basis for generating multiple candidate answers in self-distillation and best-of-N methods.

Empirical power-law curves showing model loss falls predictably as parameters, data, and compute grow.

Scaling laws are power-law relationships found empirically: a model's loss drops smoothly and predictably as parameters, training tokens, and compute increase together. They let labs forecast a model's capability before training it, and they later motivated training smaller models on far more data (compute-optimal, Chinchilla-style).

Distill a teacher's step-by-step reasoning into a small model via many symbolic CoT traces.

Symbolic Chain-of-Thought Distillation samples multiple chain-of-thought rationales from a large teacher and trains a small student on them, so even a 1-3B model learns to reason in explicit steps rather than pattern-match. It is a key reason small QuKaiZen students can think.

A cryptographic signature certifying a model's provenance — what it was distilled from and that it is untampered.

A seal is a cryptographic signature (QuKaiZen uses Ed25519) bound to a finished model, certifying its provenance: which teacher and corpus it came from, which certification gates it passed, and that its weights have not changed since. Anyone can verify the seal offline, so an owned model carries proof of exactly what it is — the Nucleus Seal.

Each token attends to all other tokens in the sequence to build context-aware representations.

Self-attention computes a weighted sum of value vectors, where weights are derived from the compatibility (dot-product) of query and key vectors for each token pair. It allows every position to directly attend to every other position, capturing long-range dependencies without the vanishing-gradient path lengths of RNNs. Scaled by 1/√d_k to prevent large dot products.

Sample several reasoning chains and take the majority answer, trading compute for accuracy.

Self-consistency improves chain-of-thought by sampling multiple independent reasoning paths and selecting the most common final answer, since correct reasoning tends to converge while errors scatter. It is a simple, strong test-time scaling technique.

A model acts as its own teacher — its current outputs become training targets for a refined version of itself.

Self-distillation removes the separate large teacher: the model generates its own outputs, reasoning traces, or soft labels and then trains on the best of them, so a single network bootstraps a sharper copy of itself. Variants filter generations by a reward or verifier (keep only correct traces) or distill an ensemble of the model's own sampled answers back into its weights. It is how a model can keep improving without a bigger model to copy from.

Create the training signal from the data itself — e.g. predict the next token — needing no human labels.

Self-supervised learning generates supervision from the raw data: mask or hold out part of an input and train the model to predict it. Next-token prediction is the self-supervised objective behind LLM pretraining, which is why models can learn from trillions of unlabeled web tokens.

An agent's store of general world knowledge and facts, decoupled from any single experience.

Semantic memory holds the agent's general, context-free knowledge — facts, concepts, and learned domain knowledge — as opposed to specific episodes. In language agents it spans the model's parametric knowledge plus an external knowledge base (often a vector store) the agent reads from and writes distilled facts to.

A language-agnostic tokenizer toolkit that trains subword models directly on raw text.

SentencePiece tokenizes raw text without pre-tokenizing on whitespace, treating the input as a stream of Unicode and learning BPE or unigram subwords. Being whitespace-agnostic makes it work uniformly across languages, which is why many multilingual models use it.

Plain supervised training on curated input to output examples — the first step of post-training.

SFT fine-tunes a pretrained model on labeled prompt/response pairs so it learns to follow instructions in a target format or domain. It is the foundation step before preference alignment (RLHF/DPO) and the simplest way to specialize a base model.

Stochastic gradient descent: estimate the gradient from a small random batch instead of the whole dataset.

Stochastic gradient descent approximates the true gradient using one mini-batch at a time, making each step cheap and adding noise that can help escape poor minima. Modern training uses momentum and adaptive variants (AdamW) built on this idea.

Each token attends only to a fixed window of nearby tokens, making attention linear in length.

Sliding-window attention restricts each token to a fixed-size local neighborhood instead of the full sequence, reducing attention cost from quadratic to linear in context length. Stacking layers still propagates information globally (a token's window overlaps its neighbors'), so long-range signal survives at far lower cost — used in models built for long contexts.

Loss decreases, but far more slowly than expected for the compute budget.

Slow convergence is when the training loss improves, but the descent rate is so low that the run will not reach the target loss within its compute budget. Root causes include a learning rate that is too low, a poor optimizer choice, a cold-start (insufficient warmup), or inadequate data quality. Distinguished from a plateau by the fact that improvement is still occurring — just too slowly.

A language model small enough to run on consumer hardware — typically 1B–13B parameters.

Small language models (1B–13B parameters) are the frontier of consumer-hardware deployment: they run at useful token rates on M-series Apple Silicon and fit in 8–24GB VRAM. SLMs trained on a specialized domain (via fine-tuning + distillation) can outperform much larger general models in that domain, because domain depth compensates for reduced overall capacity.

Shift quantization difficulty from activations to weights so both can go to INT8 cleanly.

SmoothQuant addresses activation outliers (which wreck low-bit quantization) by mathematically migrating scale from activations into weights, smoothing the activation range so both can be quantized to INT8 with little loss. It enables efficient 8-bit inference of large models.

A teacher's full probability distribution used as the training target, not just the single correct label.

Soft targets are the teacher's softened output probabilities (often via a temperature) over all classes or tokens. They encode 'dark knowledge' — how the teacher rates the wrong answers relative to each other — which teaches the student far more than a one-hot label. Matching soft targets is the core signal in classic knowledge distillation.

Turns a vector of logits into a probability distribution that sums to 1.

Softmax exponentiates each logit and divides by the sum, producing positive values that add to 1 — a probability distribution. It picks the next token from logits and, inside attention, weights how much each token attends to others.

Compute attention over only a chosen subset of token pairs instead of all of them.

Sparse attention replaces the dense all-pairs attention matrix with a structured or learned subset — local windows, strided/dilated patterns, global tokens, or routed blocks — to cut the quadratic cost of long sequences. The pattern is designed so information can still flow across the whole sequence in a few hops.

A small draft model proposes several tokens; the big model verifies them in one pass — lossless speedup.

Speculative decoding runs a cheap draft model to guess the next few tokens, then the large target model verifies them all in a single forward pass, accepting the longest correct prefix. Output is identical to normal decoding, but throughput rises 2-3x because the expensive model runs less often.

QuKaiZen's pipeline that distills deep reasoning from frontier teacher models into small, owned Super Skill models.

The Super Skill Distillation Pipeline (SSDP) extracts deep domain reasoning from 400B+ frontier teacher models and crystallizes it into small 1-7B Super Skill models that run on commodity hardware, air-gapped, and owned forever. It is not RAG — a Super Skill knows its domain. Nucleus implements it: KICE/TICE knowledge extraction, RAFT, Symbolic Chain-of-Thought distillation, an adversarial swarm that trains the student to convergence, three certification gates, and an Ed25519 Nucleus Seal.

Loading a checkpoint whose architecture or tokenizer does not match the current code.

A stale checkpoint is saved from a different model version (different architecture, layer names, or config) than the one being loaded. Shape mismatches cause hard errors; silent mismatches (different normalization, different positional encoding) cause degraded performance. Always pin the model architecture version alongside the checkpoint and use `from_pretrained` with the matching config.

A sequence architecture that carries a recurrent hidden state, scaling linearly with length instead of attention's quadratic cost.

State-space models (and selective variants like Mamba) process sequences with a continuous-time-inspired recurrence: a compact hidden state is updated token by token, giving linear-time, constant-memory inference over long sequences. Selective SSMs make the state update input-dependent, recovering much of attention's content-routing ability without its quadratic blow-up.

A string that, once generated, halts decoding — used to bound output and separate turns.

A stop sequence is one or more strings that terminate generation when produced, so the model doesn't run on past the intended boundary. They mark turn ends, close structured fields, or cap output. Distinct from the model's learned end-of-sequence token, stop sequences are caller-specified at request time.

Forcing a model's response into a machine-parseable shape like JSON conforming to a schema.

Structured output makes a model return data in a defined format (typically JSON matching a schema) instead of free text, so programs can consume it reliably. It is usually enforced via constrained decoding and underpins tool use and agent pipelines.

The small model being trained to absorb the teacher's reasoning.

The student is the compact model (1–7B) that learns from the teacher's traces, RAFT data, and adversarial correction — ending as a sealed Super Skill that can beat its teacher in-domain.

A 1-7B model that durably knows a domain, distilled from a frontier teacher and owned forever.

A Super Skill Model is the output of QuKaiZen's pipeline: a small (1-7B) model that crystallizes a frontier teacher's deep reasoning for a domain, runs on commodity or air-gapped hardware, and keeps improving. It knows — it does not look things up like RAG.

Learning from labeled examples — inputs paired with the correct outputs.

Supervised learning trains a model on input-output pairs so it learns to predict the output for new inputs. It underpins classification and the SFT stage of LLM training. Its bottleneck is the cost of obtaining labels.

An orchestrating agent that routes work to specialist sub-agents and integrates their results.

A supervisor (orchestrator) agent decomposes a task, dispatches subtasks to specialist agents, and combines their outputs — the hub of a hierarchical multi-agent system. Clean handoffs and well-scoped specialists keep the system coherent.

A gated activation for the feed-forward block that tends to beat plain GELU/ReLU at equal size.

SwiGLU combines a Swish activation with a gating mechanism: the FFN computes two projections and uses one to gate the other. It consistently improves quality over ReLU/GELU FFNs and is standard in recent LLMs, usually with a widened hidden dimension to keep parameter count comparable.

Change the optimizer (e.g., SGD → Adam, Adam → AdamW) to better fit the problem.

Different optimizers make different trade-offs: SGD with momentum generalizes well but is sensitive to LR and requires careful tuning; Adam adapts per-parameter LR and handles sparse gradients but is prone to weight drift; AdamW decouples weight decay from the adaptive LR and is the standard for fine-tuning transformers. Switching can resolve convergence problems when hyperparameter tuning alone fails.

A model's tendency to tell users what they want to hear rather than what's true.

Sycophancy is the learned habit of agreeing with or flattering the user, often a side effect of preference training where agreeable answers got rated higher. It undermines honesty and is a target of careful reward design and evaluation.

Capturing a teacher's reasoning as reusable symbolic structure, not just imitated text traces.

Symbolic Chain-of-Thought captures the structure of a teacher's reasoning — the steps, rules, and relationships — in symbolic form rather than copying surface-level wording. Distilling that structure (SCoTD) teaches a small student to reason faithfully instead of mimicking phrasing, which is what makes a Super Skill robust rather than brittle.

A high-priority instruction block that sets a model's role, rules, and behavior before the user's turn.

The system prompt is a special leading message that establishes the assistant's persona, constraints, tools, and policies for the whole conversation. Models are trained to weight it above ordinary user turns, making it the primary lever for steering behavior without fine-tuning — and a key surface for both control and prompt-injection risk.

Treat the weight change from fine-tuning as a 'task vector' you can add or subtract.

Task arithmetic defines a task vector as fine-tuned-minus-base weights; adding it imparts the skill, subtracting it removes a behavior, and summing vectors composes skills. It is the conceptual basis for several merging methods.

The large frontier model whose reasoning is distilled into a small student.

In distillation the teacher is the big, capable model (400B+) that generates reasoning traces and judgments; the student learns to reproduce its competence in-domain. QuKaiZen uses two-tier teachers for breadth and depth.

A large teacher model guides a smaller student model's training.

The teacher-student framework uses a fixed, high-quality teacher model to provide training signal for a smaller student. The student is trained to minimize the difference between its predictions and the teacher's predictions (soft targets, intermediate representations, or both). A common pattern is to use a large frontier model as the teacher and a smaller, deployable model as the student.

A knob for randomness in generation — low is focused/deterministic, high is creative/diverse.

Temperature scales logits before softmax: below 1 sharpens the distribution (safer, more repetitive), above 1 flattens it (more diverse, more errors). At 0 the model is effectively greedy. It is the simplest lever for output style.

Split individual weight matrices across devices so one layer's math is computed in parallel.

Tensor parallelism partitions the weight matrices of a layer across devices, each computing part of the matmul and exchanging partial results. It lets a single layer too big for one device run across several, at the cost of heavy inter-device communication, so it's used within a fast-interconnect node.

NVIDIA's inference optimizer/runtime that compiles models into highly tuned GPU engines.

TensorRT compiles a model into a hardware-specific engine with fused kernels, quantization, and kernel auto-tuning for maximum GPU inference throughput and low latency. TensorRT-LLM specializes it for transformers.

Hugging Face's production inference server for high-throughput, low-latency LLM serving.

Text Generation Inference is a serving stack with continuous batching, tensor parallelism, and optimized kernels for deploying LLMs at scale, comparable in role to vLLM. It exposes a standard generation API.

[ROADMAP] The sealed domain-specialist SLM produced by the Nucleus pipeline — the one bet.

[ROADMAP] 'The bake' is QuKaiZen's term for the sealed, domain-specialist small language model produced at the end of the RAW→COMPILED→BAKED knowledge lifecycle. The baked model is the value-add: a gated, sourced World compiled to a corpus (via bake-corpus.mts), fine-tuned by Nucleus, and sealed for deployment on AeroLLM. 'The one bet' per CLAUDE.md — the bake is the moat, not the framework. ROADMAP because no bake has been produced yet.

How many tokens a system generates per unit time, across all requests.

Throughput measures total tokens/second a serving stack produces; it trades off against per-request latency. Speculative decoding and batching push it up.

QuKaiZen's agent for Layer-7 tacit knowledge — the unwritten expert know-how and gotchas.

TICE extracts implicit, tribal knowledge — folklore, gotchas, and esoteric patterns that are not formally documented. It is the highest-value extractor for user-data-enriched (Mode 2/3) Super Skills, capturing expertise that lives only in practitioners' heads.

A merge recipe that trims small changes and resolves sign conflicts between task vectors.

TIES-Merging improves naive averaging by keeping only the largest-magnitude parameter changes, electing a consistent sign per parameter across models, and then averaging the agreeing updates. Resolving interference yields merged models that retain more of each source's skill.

Tokenizer and model are mismatched — inputs are decoded/encoded incorrectly.

A tokenization mismatch occurs when the tokenizer used during training differs from the one used during inference, or when a tokenizer is applied to data outside its vocabulary distribution. Symptoms range from subtle (degraded performance on certain token sequences) to severe (completely corrupted outputs). Always use the tokenizer shipped with the model checkpoint and apply it consistently across train/val/test.

Splits text into tokens (subword units) the model actually reads, and back again.

A tokenizer converts raw text into integer token IDs (and back) using a learned vocabulary, usually via subword schemes like BPE or SentencePiece. Token count drives context limits and cost, and odd tokenization explains many model quirks.

A model invoking external tools — APIs, code, search — to act beyond text.

Tool use lets a model call functions (query a database, run code, hit an API) and fold the results back into its reasoning, turning a language model into an actor in real systems.

Restrict sampling to the k most probable next tokens, then renormalize and draw from those.

Top-k sampling truncates the distribution to the k highest-probability tokens before sampling, cutting off the long tail of unlikely (often nonsensical) options. It trades a little diversity for coherence; the right k depends on how peaked the distribution is at each step.

Sample from the smallest set of top tokens whose probabilities sum to p — an adaptive cutoff.

Top-p (nucleus) sampling keeps the smallest set of most-probable tokens whose cumulative probability reaches p, then samples from that set. Unlike fixed top-k, the cutoff adapts to the distribution's shape: wide when the model is uncertain, narrow when it's confident. It is a common default for open-ended generation.

PyTorch's just-in-time compiler that traces and optimizes a model into faster fused kernels.

torch.compile captures a model's operations into a graph and lowers it through a backend (e.g. Inductor/Triton) to fused, optimized kernels, often yielding speedups with a one-line change. It brings ahead-of-time-style optimization to otherwise eager PyTorch code.

Validation loss significantly worse than training loss — generalization failure.

A large gap between training and validation loss signals overfitting: the model has memorized training data rather than learning to generalize. The gap widens over epochs as the model fits noise. The severity of overfitting is proportional to the gap size. Common during full fine-tuning of large models on small datasets.

Reuse a model trained on one task as the starting point for another, instead of training from scratch.

Transfer learning takes the knowledge captured by a model pretrained on a broad task and adapts it to a narrower one with far less data and compute. The pretrain-then-fine-tune recipe behind every modern LLM is transfer learning at scale.

The attention-based neural architecture behind essentially every modern LLM.

The transformer stacks blocks of multi-head attention and feed-forward layers with residual connections and normalization, processing all tokens in parallel. Introduced in 'Attention Is All You Need' (2017), it scales beautifully and underpins GPT, Llama, and the rest.

Explore multiple reasoning branches as a search tree, evaluating and backtracking, instead of one chain.

Tree of Thoughts generalizes chain-of-thought into a search: the model generates several candidate next steps, scores them, and explores promising branches with backtracking. It trades more compute for better performance on problems needing exploration or planning.

Attention that adds an explicit third 'context' term to the usual query-key interaction, modeling three-way relationships instead of pairwise ones.

Standard ('bi-') attention scores pairs: a query against keys. Tri-Attention introduces a third element — typically an explicit context representation — so relevance is computed over (query, key, context) triplets rather than (query, key) pairs. By making the context a first-class factor in the score (e.g. via a tensor/trilinear interaction) it captures dependencies that pairwise attention folds away, which helps retrieval-augmented and context-conditioned models reason about how a query and a candidate relate *given* the surrounding context.

A Python-like language for writing fast GPU kernels without hand-writing CUDA C++.

Triton (from OpenAI) lets researchers write custom GPU kernels in a Python-like syntax that compiles to efficient code, making fused high-performance ops far easier to author. Many modern kernels, including FlashAttention implementations, are written in Triton.

Time to first token — how long after a request before the model emits its first output token.

Time to first token measures responsiveness: the delay covering queuing plus prefill before any output appears. It is the latency users feel most in streaming interfaces, distinct from overall throughput or per-token speed.

Finding structure in data with no labels — clustering, density, or representation.

Unsupervised learning discovers patterns in unlabeled data, such as clusters or low-dimensional structure, without told-correct answers. It contrasts with supervised learning and overlaps with self-supervised learning, which manufactures labels from the data itself.

Held-out data used to tune and monitor training, kept separate from the final test set.

A validation (dev) set is data the model never trains on, used to pick hyperparameters, trigger early stopping, and watch for overfitting during training. It must stay separate from the test set, which is touched only once for the final, unbiased estimate.

Gradients shrink toward zero in early layers — no useful learning signal.

In deep networks without skip connections or normalization, gradients can shrink exponentially as they are backpropagated, making early-layer weights effectively frozen. The symptom is that early-layer losses barely improve while later layers train. Addressed by architectural choices (residual connections, layer normalization) rather than hyperparameter tuning.

A generative model that learns a probabilistic latent space via the ELBO objective.

A VAE learns to encode inputs into a distribution over latent variables (not a fixed vector) and decode samples from that distribution. The ELBO (Evidence Lower BOund) objective balances reconstruction quality (decoder term) against KL divergence of the posterior from the prior. VAEs are the predecessor to diffusion models for continuous generative modeling and are used in multimodal embedding spaces. Posterior collapse is the principal failure mode.

The target-model pass that accepts or corrects speculatively drafted tokens.

In speculative decoding the verifier is the large model's single forward pass that checks the draft's proposed tokens — keeping the correct prefix and resampling the first mismatch — which guarantees the same distribution as decoding normally.

A transformer that processes images by splitting them into patches treated as tokens.

A Vision Transformer (ViT) cuts an image into fixed patches, linearly embeds each as a token, and runs a standard transformer over them. It brought the transformer recipe to vision and is the image encoder in many multimodal models.

A high-throughput LLM serving engine; its PagedAttention manages the KV-cache like virtual memory.

vLLM maximizes GPU throughput via PagedAttention — treating the KV-cache as paged memory to eliminate fragmentation — plus continuous batching of incoming requests. It is the enterprise-grade backend for serving teacher models on GPUs.

The fixed set of tokens a model knows; its size sets the width of the input and output layers.

A model's vocabulary is the complete set of tokens its tokenizer can produce, fixed at training time. Its size (often 32k-256k) sets the dimensions of the embedding table and the final softmax: every step the model produces a distribution over the whole vocabulary. Larger vocabularies pack more text per token but enlarge those layers.

Ramping the learning rate up from near zero over the first steps to avoid early instability.

Learning-rate warmup starts the LR small and increases it over the first few hundred or thousand steps before the main schedule (often cosine decay). Early gradients are noisy; warmup prevents large destabilizing updates while the optimizer's statistics settle.

A process that observes for changes and triggers reconciliation when state moves.

A watcher monitors a source — files, a repo, an event stream — and fires the reconcile loop whenever it detects a change, so the system converges toward desired state without manual prompting. It is the trigger half of a declarative control loop: watch, then reconcile.

A penalty that nudges weights toward zero each step, discouraging overly large parameters and overfitting.

Weight decay shrinks parameters by a small factor every update, regularizing the model toward simpler solutions and improving generalization. In AdamW it is applied decoupled from the gradient-based update (the 'W'), which is why AdamW is preferred over plain Adam for transformers.

How weights are set before training — a critical determinant of early convergence.

Poor weight initialization causes vanishing or exploding gradients before training even begins. Key insight: variance of activations should stay roughly constant across layers. He initialization (for ReLU) and Xavier/Glorot initialization (for tanh/sigmoid) are designed to achieve this. Modern large language models typically use a small normal distribution with std proportional to 1/√d_model, sometimes with scaled initialization for residual paths.

QuKaiZen's core metric: certified, permanently-owned reasoning capability per unit of lifetime energy to mint and run it.

Renting a frontier model burns full datacenter energy on every query, forever, and you own nothing. QuKaiZen spends energy once to distill that reasoning into a small model you keep — certified by three independent gates, sealed with cryptographic provenance, run locally at near-zero marginal cost. Capability only counts if it is proven and trustworthy: a model that scores high but fails the hallucination gate scores zero. The energy is not a running cost; it is capital spent on an asset you own forever.

A declared sequence of steps an agent or pipeline executes.

A workflow encodes the steps — download, analyze, decide, process — as configuration rather than ad-hoc code, so it's inspectable, versionable, and reproducible. PaperAgents declares them in TOML.

An agent's active scratchpad — the small, volatile state it holds for the current decision.

In the CoALA framing, working memory is the agent's transient state for the current cycle: the active goal, intermediate reasoning, recently retrieved facts, and the latest observation. It is what actually flows into the prompt at each step and is overwritten as the task proceeds — analogous to RAM, not disk. Its capacity is bounded by the context window.

A method to extend a model's usable context window by rescaling its rotary position frequencies.

YaRN (Yet another RoPE extensioN) interpolates and rescales RoPE frequencies, often with brief fine-tuning, so a model trained at one context length works well at a much longer one. It is a common way to stretch context windows without full retraining.

DeepSpeed's optimizer that partitions optimizer state, gradients, and params to remove memory redundancy.

ZeRO eliminates the memory redundancy of vanilla data parallelism by partitioning optimizer states (stage 1), gradients (stage 2), and parameters (stage 3) across GPUs. It is the idea FSDP also implements, enabling trillion-parameter training.

Asking a model to perform a task from instructions alone, with no examples.

Zero-shot prompting gives only a task description and the input, no demonstrations, relying on the model's pretrained and instruction-tuned knowledge. It is the simplest, cheapest prompting mode; modern instruction-tuned models are surprisingly strong zero-shot, though few-shot still helps on tricky formats.