Glossary

114 terms from Fine-tuning LLMs, in plain words, each with the chapter that teaches it.

Search the glossary

A

Activations: The intermediate results a model saves during the forward pass so the backward pass can use them. Their memory grows with batch size and sequence length, on top of the weights and optimizer memory.
AdamW: A widely used optimizer for training neural networks. It keeps two running averages (optimizer states) for every trained parameter, which is a big part of full fine-tuning's memory cost.
Adapter: The small set of trained LoRA matrices, saved as a separate file. It loads on top of the unchanged base model, can be swapped for another task, or can be merged into the weights.
Alpha (lora_alpha): LoRA's scale setting. The adapter's update is multiplied by alpha / r, so it controls how strongly the adapter speaks relative to the frozen weights.
Attention mask: A list of 1s and 0s that tells the model which tokens are real and which are padding. `apply_chat_template` returns it alongside `input_ids`.
Attention projections (q_proj, k_proj, v_proj, o_proj): The four linear layers in each transformer block's attention part (query, key, value and output). LoRA can adapt just some of them, such as q_proj and v_proj, or all linear layers.

B

Base model: A model that has only been pretrained, so it continues text instead of following requests. For small fine-tuning datasets it is usually better to start from the instruct version of the model instead.
Baseline: The alternatives you compare the tuned model against on the same test set. For Sam's orders these are the prompted base model and prompted Claude Haiku.
Batch size: The number of examples processed together in one forward and backward pass. Larger batches need more memory, so a batch of 1 combined with gradient accumulation is a way to train on limited hardware.
Beta: In DPO, the setting that controls how strongly the model is held close to the reference model. Higher beta means smaller moves; TRL's default is 0.1.
bfloat16: A 16-bit number format with float32's range but less precision, using 2 bytes per parameter. It is the usual format for storing a frozen base model during LoRA fine-tuning.
bitsandbytes: The library QLoRA needs to load and compute with 4-bit weights. It requires an NVIDIA GPU.

C

Cache (of API responses): Saved model responses that are reused instead of calling the API again. The cache lets the generated datasets be rebuilt for free and identically, and lets readers run the scripts without an API key.
Chat template: The rules stored with the tokenizer that turn a list of messages into one token sequence with special tokens marking each turn. Training, evaluation and serving must all use the same one.
ChatML: The chat template style used by SmolLM2 and Qwen, where `<|im_start|>` and a role open each turn and `<|im_end|>` closes it.
Checkpoint: A saved snapshot of the adapter and optimizer state taken during training. If a run stops you can resume from the latest one, and the trainer can reload the one with the lowest validation loss.
Chosen and rejected: The better and worse answers to the same prompt in a preference pair. For Priya's DPO run, the chosen answer is her real reply and the rejected one is the untuned model's draft.
Completion: The part of a training example the model should write, such as the assistant's JSON answer. The loss counts only these tokens.
Cross-entropy loss: The standard loss for next-token training: the average, over tokens, of minus the log of the probability the model gave the correct token. Taking e to the power minus the loss gives the geometric mean probability of the correct tokens.

D

difflib.get_close_matches: A Python standard-library function that returns the most similar strings from a list, if they are similar enough. It can snap "Espresso" to "House Espresso" and let code drop names that match nothing, such as unknown products.
Distillation: Training a small model on a bigger model's outputs. Check the provider's terms first, since some forbid using outputs to train other models.
DPO (direct preference optimization): Preference tuning done in one ordinary training loop. It pushes the model to make the chosen answer likelier than the rejected one, relative to a frozen reference model.
DPOTrainer: TRL's trainer for DPO, which expects data with a prompt plus chosen and rejected completions. With a PEFT adapter and no separate reference, it uses a frozen copy of the adapter as the reference.

E

End-of-sequence (EOS) token: The special token a model writes to end its turn; it is `<|im_end|>` for SmolLM2. Training text must include it so the tuned model learns to stop instead of running on to `max_new_tokens`.
Entropy (in the logs): A training-log value showing how spread out the model's predicted probabilities are. Low entropy means the model is confident.
Epoch: One pass through the training data. For LoRA on a few hundred examples, 2 to 4 epochs is typical; too many and the model memorizes.
Exact match: Scoring an answer as right only if it equals the label exactly. "Fernwood Cafe" is wrong if the order system says "Fernwood Café".

F

Few-shot prompting: Putting a few worked examples in the prompt so the model copies their style or format. It is worth trying before fine-tuning when a model's tone is off.
Fine-tuning: Continuing to train a pretrained model on your own examples, so it keeps its general skills and picks up a new task or style. It changes the model's weights, unlike prompting or RAG.
Full fine-tuning: Updating every weight of the model. It is the most flexible option but needs about 16 bytes of memory per parameter, plus activations.

G

Gated model: A model you can download only after accepting its terms on the Hugging Face Hub and logging in with a token (`hf auth login`). Meta's Llama 3.2 is an example.
Generation prompt: The text that cues the model's turn, such as `<|im_start|>assistant` and a new line. `add_generation_prompt=True` adds it.
Geometric mean: The nth root of a product of n numbers. Here, e to the power minus the loss gives the geometric mean of the probabilities the model assigned to the correct tokens, so a loss of 0.69 means about 0.5.
GGUF: The single-file model format used by llama.cpp and Ollama. You convert a merged model to it with llama.cpp's `convert_hf_to_gguf.py`.
grad_norm: A training-log value giving the size of the gradient. A sudden jump warns of instability.
Gradient: The values, computed in the backward pass, that say how to change each weight to reduce the loss. Each trained parameter needs memory for its gradient.
Gradient accumulation: Adding up the gradients of several small batches before one optimizer step. Batches of 1 accumulated 8 times train like batches of 8 with the memory of 1.
Gradient checkpointing: Recomputing activations in the backward pass instead of storing them, saving memory at the cost of speed. The small SmolLM2 model doesn't need it.
Greedy decoding: Always picking the likeliest next token, which makes output repeatable. In `generate` it is set with `do_sample=False`.
Grouped-query attention: Attention where several query heads share keys and values, so `k_proj` and `v_proj` are smaller than `q_proj`. SmolLM2 uses it.

H

Hosted fine-tuning: A service that trains a model for you on its GPUs from a JSONL file and serves the result. It saves GPU setup, but your data goes to the provider and some services won't let you download the weights.
Hugging Face Hub: The website that hosts open models, datasets and their model cards. Each model page lists its license and whether it is gated.

I

Instruct model: A model that has been instruction-tuned and preference-tuned to follow requests in a chat format. It is the usual starting point for fine-tuning on a few hundred examples.
ISO date format: The date format year-month-day with zero-padded numbers, like 2026-11-02. Sam's answers use it, and a date such as "2026-11-2" counts as an error.

J

Jaccard similarity: A measure of how alike two texts are: the number of words they share divided by the number of distinct words in both. It can be used to drop near-duplicate examples, such as emails with a similarity over 0.9 to an earlier one.
JSONL (JSON Lines): A text file with one JSON object per line. You can append to it, count the examples with `wc -l`, and read it one example at a time.

L

Leakage: Information from the test set reaching training or decisions, which inflates scores. Near-duplicates across train and test are one cause.
Learning rate: The size of each update during training. About 2e-4 is typical for LoRA, higher than for full fine-tuning; too high makes the loss jump, too low makes it barely move.
Left padding: Padding prompts at the start so every prompt in a batch ends exactly where generation begins. Batched generation gives garbled output without it.
License (open weights): The terms on a model's card that say what you may do with it. Apache 2.0 and MIT are permissive, some families use their own conditions, and a fine-tuned model inherits the base model's license.
Linear schedule: A learning-rate schedule that falls in a straight line to 0 by the end of training. It is the common default, applied after an initial warmup.
llama.cpp: A C++ program for running GGUF models fast on ordinary CPUs and GPUs. Its repository includes the converter script that makes GGUF files.
LLM judge: A strong language model given a policy, a customer's message and a reply, and asked a narrow yes-or-no question about the reply. It grades free-text outputs, such as whether Priya-style replies follow policy and answer the customer, when exact matching isn't possible.
LoRA (low-rank adaptation): Freezing the model's weights and training two thin matrices, A and B, per layer; their product is the update. It needs far less memory than full fine-tuning and produces a small adapter.
lora_dropout: A setting that randomly zeroes a fraction (5% here) of the adapter's inputs during training, a little protection against overfitting. `model.eval()` turns it off for generation.

M

Masking (masked tokens): Giving prompt tokens the label -100 so the loss ignores them. This way the model learns only to write the answer, not the email.
max_length: The training length limit in tokens; longer examples are cut off, possibly mid-answer. Measure your example lengths first so nothing important is truncated.
max_new_tokens: A `generate` setting that caps how many tokens the model may write. A model that never learned to stop keeps going until it hits this limit.
Mean token accuracy: A training-log value giving how often the model's likeliest next token was the right one on answer tokens. It is only a proxy: one wrong digit is a wrong order.
Merging (an adapter): Adding the LoRA update into the base weights (W becomes W + (alpha/r)BA) to make an ordinary standalone model. Merge into a 16-bit base, not a 4-bit QLoRA one.
Messages: A list of conversation turns, each with a `role` (system, user or assistant) and `content`. Training examples are stored in this form.
Meta device: A PyTorch device that builds layers with their shapes but no actual numbers, so it uses no memory. It lets you inspect a model's layers and count LoRA parameters without loading weights.
Mixed precision (mixed-precision): Running calculations in 16 bits while keeping a 32-bit copy of the trained weights for the small updates. It is the usual full fine-tuning setup.
Model card: A model's documentation page on the Hugging Face Hub, including its license and intended uses.
Modelfile: The small text file that tells Ollama which weights (`FROM`), system message and settings make up a model. Include the system message the model trained with.

N

Near-duplicates: Examples that differ by only a word or two. They make the model over-learn those phrasings, and if one copy lands in the test set the score is inflated.
NF4: The 4-bit number format QLoRA uses to store frozen base weights.

O

Ollama: A free tool that downloads, stores and runs GGUF models behind a local API. `ollama create name -f Modelfile` registers a model and `ollama run name "..."` sends it one message.
Optimizer states: AdamW's two running averages for each trained parameter, taking 8 bytes per parameter in float32. LoRA avoids most of this cost by freezing the base weights.
Overfitting: Fitting the training examples better and better while getting worse on new ones. The sign is validation loss rising while training loss keeps falling.

P

Packing: Joining several short examples into one `max_length` block so no time is spent on padding. It's useful for many short examples on a GPU.
Parameter: One learned number in a model. It takes 2 bytes in 16-bit formats, so weights need about 2 GB per billion parameters.
PEFT (parameter-efficient fine-tuning): Methods that train only a small number of extra parameters, and the Hugging Face library that implements them. It provides `LoraConfig` and `get_peft_model`.
Pipeline: The `transformers` function that wraps the tokenizer, model and generation into one object. It lets you run a model, such as a merged fine-tune, in a few lines.
Post-processing cleanup: Plain code that fixes what code can check in a model's output, such as snapping product names to a known list and zero-padding dates. For Sam's order extractor it lifted the tuned model from 72% to 92% fully correct on the test set.
Preference tuning: Training on pairs of better and worse answers to the same prompt instead of copying single good answers. DPO is a common method for it.
Prompt (in a dataset): The messages the model reads before its answer, such as the system message and Sam's order email. In training it is masked from the loss.
Prompt and completion format: Training data with the messages to read under `prompt` and the answer under `completion`. TRL applies the chat template and masks the prompt automatically for it.
Prompting: Changing the instructions and examples the model sees on every request. Nothing about the model changes, and it is the cheapest option to try first.
Pydantic: A Python library for describing data shapes as classes. It can define the schema for a model's structured output, such as a `Batch` of emails or a set of `Verdicts`.

Q

q8_0: A GGUF format that stores weights in 8 bits, about half the size of 16-bit with almost no quality loss.
QLoRA: LoRA on a base model stored in 4 bits (NF4), which fits much bigger models on small GPUs. It needs `bitsandbytes` and an NVIDIA GPU, and trains somewhat slower than plain LoRA.
Quantization: Storing weights in fewer bits, such as 4 or 8, to save memory and disk. QLoRA stores the frozen base in 4 bits, and GGUF files can be 8-bit (q8_0) or smaller.

R

RAG (retrieval-augmented generation): Looking up relevant facts from your documents or database and pasting them into the prompt. It suits facts that change, like prices, and leaves the model unchanged.
Rank (r): The inner size of LoRA's two thin matrices, meaning how many directions the update can use. 8 to 64 is typical, and 16 is a good start for a narrow task.
Reference model: The frozen model DPO compares against, usually the SFT model you started from. It keeps the tuned model from drifting far from what it already does well.
Regular expression: A pattern language for matching text. It is useful for automatic checks, such as confirming an email mentions a bag count, measuring casual product names, or validating a greeting.
Repetition loop: A model repeating the same line until `max_new_tokens` runs out. It is common in small, undertrained models, as in the tuned model's repeated "Kettle & Crane" sign-offs.
Repetition penalty: A `generate` setting that makes already-used tokens less likely. It failed here because it also punishes words a reply must reuse, such as customer names.
Rewards (in DPO logs): For each answer, beta times how much more likely the model makes it than the reference does. `rewards/margins` is chosen minus rejected, and `rewards/accuracies` is the fraction of pairs where the chosen reward is higher.
RLHF: An older preference-tuning approach that trains a separate reward model on comparisons and then uses reinforcement learning. DPO gets a similar result with one ordinary training loop.
Rule check: Code that tests an output against a written rule, such as Priya's required sign-off or no exclamation marks. It checks form cheaply but not factual truth.

S

Safetensors: The standard, safe file format for model weights, such as `adapter_model.safetensors`.
Seed (random seed): A fixed starting value for random choices so that shuffles, generated specs and training runs repeat the same way. Seeding data generation, shuffling and training makes results reproducible.
Self-preference bias: An LLM judge's tendency to rate text like its own more kindly. Haiku judging its own replies may therefore be generous.
SFTTrainer: TRL's trainer for supervised fine-tuning. It applies the chat template, masks the prompt, batches examples and runs the training loop, configured with `SFTConfig` and PEFT's `LoraConfig`.
Special tokens: Reserved tokens such as `<|im_start|>` and `<|im_end|>` that mark where each turn begins and ends. The chat template inserts them, so the model trains on them.
Step: One optimizer update. With 250 examples and 8 examples per step, an epoch is 32 steps.
Structured output: An API feature that makes a model's reply match a JSON schema. It returns things like batches of emails or judge verdicts as typed data.
Supervised fine-tuning (SFT): Fine-tuning on inputs paired with the answers you want. It teaches by imitation and is the step that turned base models into chat assistants.
Synthetic data: Training examples written by a model. For Sam's data, code makes the labels and Claude Haiku only writes the emails.
System message: The first message in a conversation (role `system`), giving the model its instructions. Use the same system message in training and in use, because the model sees it as part of the text.

T

Target modules: The layers of a model that get LoRA adapters. `"all-linear"` adapts every linear layer and usually learns better than adapting only the query and value projections.
Temperature: A sampling setting. `temperature 0` in the Modelfile makes Ollama pick the likeliest token, like `do_sample=False`.
Test set: Labeled examples kept aside until the end to report final quality. If you use it to make choices, it stops being a fair test.
Training loss: The average loss on the batches the model just learned from, logged every few steps.
Training set: The examples the model learns from: 250 of Sam's 320.
TRL: Hugging Face's library for SFT, DPO and related training methods (the name stands for Transformer Reinforcement Learning). It is most often used for supervised fine-tuning through `SFTTrainer`.

U

Unsloth and Axolotl: Fine-tuning tools that add faster kernels and ready-made configurations on top of the usual training libraries. They are options for speeding up and simplifying larger fine-tuning runs.

V

Validation loss: The loss on the validation set, which the model never trains on. If it rises while training loss falls, the model is overfitting.
Validation set: Examples checked during training to choose settings and checkpoints: 30 of Sam's 320. You can also read its errors without spoiling the test set.
vLLM: A server for running models on GPUs for many users at once, with an OpenAI-compatible API. It can serve a merged model or a base model with LoRA adapters loaded alongside.

W

Warmup: Starting the learning rate near 0 and raising it over the first steps, so the first noisy updates are small. A value below 1 is a fraction of all steps (0.1 means the first 10%).
Weights: A model's trained parameters, stored in files such as safetensors or GGUF. An open model's weights can be downloaded and changed, and fine-tuning changes them.