Glossary

98 terms from RAG Beyond the Basics, in plain words, each with the chapter that teaches it.

Search the glossary

#

95% interval: A range of scores that the system's true score very likely falls inside. For Kettle & Crane's baseline, 0.64 on 120 questions comes with [0.56, 0.72], which shows how much the score could move on different questions.

A

Adaptive transformation: Search with the original question first, and call a query transformation only when the top rerank score is weak. This spends the extra model call on a minority of questions.
Agentic RAG (iterative retrieval): The model gets the search function as a tool and decides whether to search again with different words before answering. It helps when a second search depends on the first, but costs extra time and tokens.
Allow-list: An access rule where nothing is visible unless a document's metadata explicitly lets the user in. A document with an unknown audience is therefore visible only to staff, which is the safe way to fail.
Answer correctness: A RAG evaluation metric for whether an answer agrees with a reference answer. A model judge usually decides it, and it differs from faithfulness, which checks the answer only against the retrieved context.
Answer error: A failure type where the evidence was in the prompt but the model still answered wrongly, partially, or said "I don't know". Fixes are in the prompt, the user context or the checks.
Answered the unanswerable: A failure where the documents can't answer the question, or the user may not see the answer, and the assistant answers anyway. The customer may act on something false.
Apache 2.0 / MIT license: Permissive open-source licenses that allow a model to be used commercially. A model's card lists its license, so check it before shipping the model.
Asked-by line: One line added to the prompt telling the model who is asking (for example Bloom Café), so words like "our" make sense. The identity must come from your login system, and the line never replaces the access filter.
asyncio / async and await: Python's way of running tasks that spend time waiting, like model calls. An `async def` function pauses at `await` so others run meanwhile, and a Semaphore caps how many run at once (four for RAGAS).

B

BM25: A keyword-matching scoring method that ranks chunks by how well their words match the query's words. It works well on exact codes like "INV-2026-0919" and is combined with vector search in hybrid search.
Bootstrap: A way to estimate uncertainty by redrawing the questions at random with replacement many times (2,000 here) and rescoring each pretend set. The middle 95% of those scores is the 95% interval.
Budget: The maximum number of characters of sources the prompt may hold (4,000 by default). It keeps prompts cheap and focused, but chunks that don't fit are cut.

C

Cache key: The label under which a cached result is stored, made from everything that affects it. If it leaves something out, you get stale or leaked results, such as an answer cached for one café being served to another.
cache_control: A marker on part of an API request that makes the API store that part for prompt caching. Later requests that begin with identical text read it at a tenth of the input price. `"ttl": "1h"` asks for the 1-hour cache.
CCPA: The California Consumer Privacy Act, a privacy law that gives customers rights over their data, including the right to have it deleted. It is one reason to delete stored logs on a schedule.
Chunk: A piece of a document, at most 800 characters in RAG Basics, that is indexed and retrieved on its own. The chunker decides where to split and what label each piece gets.
Claim-level grounding: Splitting an answer into sentences and asking a model, for each one, whether the sources it cites actually state it. It catches a real source cited for something it doesn't say.
Cohen's kappa: A measure of how much two labelers agree beyond what chance would produce: 0 means no better than chance and 1 means perfect. It stops a judge that always says "correct" from looking good when most answers are correct.
Confusion matrix: A table counting how often each label from one labeler (a person) met each label from another (the judge). Its diagonal shows agreement.
Context coverage (cov_pool, cov_k, cov_kept): The share of a question's evidence strings found in the candidate pool, the top k chunks and the final prompt. Comparing them shows where evidence was lost.
Context overflow: A failure where the evidence ranked in the top k but didn't fit within the prompt's size budget, so it was cut. The `cov_kept` check catches it.
Context precision: A RAG evaluation metric for how much of the retrieved context is relevant, with relevant chunks ranked first. Libraries such as RAGAS compute it with model prompts.
Context recall: A RAG evaluation metric for how much of the information an answer needs was actually retrieved. A free approximation checks whether the expected evidence strings appear in the top `k` retrieved chunks.
Contextual chunk headers (contextual retrieval): A model writes one sentence that places each chunk within its whole document, and that sentence is added to the chunk before indexing. It helps most when chunks lose their context, as in long or unstructured documents.
Continuous integration: A system that automatically runs checks on every change. A script that exits with an error status, as the gate does with `sys.exit`, makes the check show as failed.
Cross-encoder: A model that reads the question and a chunk together and scores how well they match. It is more accurate but slower than comparing precomputed vectors, which is why it only reorders a small pool.

D

Dataclass (frozen): A Python class that just holds named values. "Frozen" means it can't be changed by accident, so `Config.but(...)` returns a modified copy instead, and `dataclasses.replace` does the same.
Demo mode: Setting `RAG_DEMO=1` replaces every model call with a stand-in so the code runs for free without an API key. The scores it produces are meaningless.
Deny-list: An access rule that shows everything except what is explicitly forbidden. It is quick to write, but it leaks any document nobody remembered to forbid, such as a handbook meant for cafés or one café's private page shown to other cafés.
Dev and test (split): The two halves of the evaluation set. Every tuning decision uses dev, and the final choice is measured once on test, which nothing was tuned on, so the result isn't flattered by luck.
Drift: A slow change over weeks in a monitored number such as decline rate. It matters as much as a sudden jump, so each week is compared with the last four.

E

Early refusal: Answering "I don't know" without calling the model when the best rerank score is below a threshold (-10 by default). It saves a model call but only catches clearly off-topic questions.
Embedding model: A model that turns text into a list of numbers (a vector) so that texts with similar meaning land near each other. It is used for vector search, and `all-MiniLM-L6-v2` is a small example.
Entities, relationships: In a knowledge graph, the entities are the things named (cafés, products, policies) and the relationships say how they connect, such as "Bloom Café orders House Espresso".
Evaluation set: A collection of questions with reference answers, evidence strings and users, used to measure the assistant. Kettle & Crane's has 120 questions chosen to expose weaknesses.
Evidence string: A short string that must appear in the retrieved text for a question to be answerable from it; "a OR b" accepts either. Matching strings is crude but free and repeatable.

F

Faithfulness (groundedness): Whether every claim in an answer is supported by the context the model was given. It needs no reference answer, so it can run on live traffic.
FastAPI / uvicorn: A Python web framework and the server that runs it, used only if you extend RAG Basics' web API with the shipped configuration.
Fixes and breaks: The count of questions a change turned from wrong to right (fixes) and from right to wrong (breaks). Reading both shows more than the net change, which can hide a 6-fixed, 4-broken swap.
Front matter: The `key: value` lines between `---` markers at the top of a Markdown file, which `loaders.py` turns into metadata. The `audience:` key there says who may read the document.

G

Global question: A question about a whole collection rather than one passage, such as the most common reasons cafés complain. Top-k retrieval handles these poorly.
GPU: Graphics hardware that runs neural models much faster than a CPU. Running a reranker on a GPU is one way to cut its processing time.
GraphRAG: A method from Microsoft Research that has a model extract a knowledge graph, group it into communities and summarize them before any question, then answers global questions from those summaries. Indexing is expensive, so it suits large, stable collections.

H

Heading path: A chunk's label made of every heading above it, such as "Wholesale Partner Handbook > Seattle and Tacoma > Delivery charge". It lets a chunk say which of three "Delivery charge" sections it is.
Hugging Face: A website hosting open models. Each model's card there lists details such as its license and whether it needs a query prefix.
Hybrid search: Search that combines vector search and BM25 keyword search by fusing their ranked lists. The `weight` setting gives the vector list's share, and the rest goes to keywords.
HyDE: Hypothetical Document Embeddings: a model writes a short made-up passage that answers the question and you search with that. The fake answer sits closer to real answers than the question does, and it is never shown to the user.

J

JSONL: A file with one JSON object per line. The evaluation questions, human labels and saved results are stored this way.

K

k: The number of chunks put into the prompt after reranking (5 by default, 8 in the shipped configuration). More chunks catch more evidence but add tokens and noise.
Knowledge graph: Entities and the relationships between them, extracted from the documents by a model. GraphRAG builds one before answering questions.

L

Ladder: A sequence of configurations in which each rung is the previous one plus one change. Measuring cumulatively shows that gains don't simply add up, since two fixes may repair the same question.
Leak: A source in the prompt that the asking user isn't allowed to see. Every run checks for leaks, and any leak counts as a failed build rather than a lower score.
Least privilege: A security principle of giving each user the least access they need. RAG Basics' per-café rule followed it but didn't scale beyond one file.
Long context: Skipping retrieval and putting every document the user may see into the prompt, cached for reuse. It scored as well as the shipped configuration here but cost over three times as much.
Lost in the middle: A finding that models use information in the middle of a long prompt less reliably than at its start or end. It's one reason to keep prompts short and put the best chunk first.

M

Metadata filter: A rule on chunk metadata (date, version, audience) that limits search to chunks that pass. It must run before ranking so each user gets their best chunks among those they may see.
Model judge: A model call that grades another model's output, here as correct, partial, wrong or declined. It's a measuring instrument and has to be checked against human labels.
Multi-query: A query transformation where the model writes three differently worded queries. You search with each plus the original and fuse the four lists.
Multihop and paraphrase questions: Two of the evaluation set's question types: a multihop question needs evidence from two passages, and a paraphrase question uses different words from the documents. Both are hard for single-search retrieval.

N

Noise: Score changes that come from chance, such as model variation, rather than from your change. A net change of one or two questions is within noise.

P

p50, p95: The median time (half of answers are faster) and the 95th percentile (95% of answers beat it). p95 shows the slow tail that the median hides.
Paired comparison: Comparing two systems question by question on the same questions by bootstrapping each question's score difference. It detects gains that the two systems' separate overlapping intervals would hide.
Pool: The candidate chunks retrieved by search before reranking (20 by default). Evidence missing from the pool is a retrieval miss.
process_time: A Python timer that counts only your own program's CPU work, not time spent waiting. Unlike a wall clock, it isn't distorted by a busy shared machine.
Prompt caching: The API stores a request's long, unchanging beginning for 5 minutes and later reads it at a tenth of the input price; writing costs 1.25 times. Claude 5.5 models only cache prompts of at least 512 tokens.
Prompt injection: An attack where instructions hidden in a document or question try to override the model's rules. A rule that exists only in the prompt, such as "don't reveal private data", can be bypassed this way, so access control belongs in code.
Pseudonym: A salted hash standing in for a user name in logs. The secret salt stops anyone from hashing the few possible names like `cafe:bloom` to identify users.

Q

Query prefix: Text that some embedding models (such as `bge-small-en-v1.5`) expect before each query. Without it they work worse, and documents get no prefix.
Query transformation: Using a model to turn the user's words into text closer to the documents' words before searching, for example by rewriting, multi-query or HyDE. It costs an extra model call per question.

R

RAGAS: An open-source Python library of RAG evaluation metrics, including faithfulness, context precision, context recall and answer correctness. Each metric is implemented with model prompts.
Ranking miss: A failure where the evidence is among the candidates but the reranker left it out of the top k.
Reciprocal rank fusion: A way to merge several ranked lists by giving each chunk 1 / (60 + rank) from every list it appears in and adding up. A chunk fairly high in several lists beats one that tops a single list.
Reference answer: The correct answer written for each evaluation question. The model judge compares the assistant's answer against it.
Regression gate: An automatic check run before every release that blocks it if it leaks or is clearly worse than the shipped version. It runs cheaply with cached replies.
Regular expression: A pattern for matching text, used here to find headings (`#{1,6} (.*)`) and citations like `[S2]`.
Reranker: A model, here a cross-encoder, that rescores the candidate pool by reading the question and each chunk together, then keeps the best k. Different rerankers use different score scales.
Retrieval miss: A failure where the evidence is not among the candidates at all. It can come from a permissions problem as well as from search.
Rewriting: A query transformation that replaces the question with a single model-written search query in the words the documents would use.

S

Semantic cache: Reusing the answer to an earlier question whose vector is similar enough. It is risky because "opened coffee" and "unopened coffee" questions have near-identical vectors but opposite answers.
Sentinel: A fake value added to the end of data so a loop handles the last item. The `"# END"` fake heading makes `sections_with_paths` save the final section.
SHA-256 hash: A function that turns any text into a 64-character fingerprint: the same input always gives the same fingerprint, and any change gives a different one. It is used for cache file names and for pseudonyms that stand in for user names in logs.
Small-to-big (parent-child, parent document): Search small child chunks but put each match's larger parent section in the prompt. That gives precise matching plus surrounding detail, though a huge parent can use up the budget.
Structured output: Asking the model to reply in a fixed JSON shape defined by a schema so the reply always parses, as the claim checker's verdicts do.
sys.argv / sys.exit: `sys.argv` is the list of words typed after `python`, so `ask.py` reads the user and question from it. `sys.exit` with a message ends the program with an error status.

T

Table row chunks: Extra small chunks, one per table row, written as a sentence that repeats the column headers (for example "Kenya Nyeri: Shop price $17.00"). They help long tables that get split; here they changed nothing.
Tenant: One customer organization whose data must stay separate from others, here a café. Separate indexes per tenant are the simplest guarantee when tenants must never mix.
ThreadPoolExecutor: A Python tool that runs several tasks at once in threads. It is useful for making many model API calls in parallel, such as eight at a time, which speeds up a batch several times.
Threshold: A cut-off score, such as -10 for early refusal. It must be tuned again when you change the reranker, because scales differ.
Token: The unit of text a model reads and writes, roughly a short word or word piece. Prices and caching limits are counted in tokens, and output tokens cost five times as much as input here.
Tool use: A feature where the model asks your code to run a function, here `search_docs`, and you send back a `tool_result` with the same `tool_use_id`. `tool_choice` controls whether it may or must call the tool.
Trace: A log record of one answer with everything that shaped it: configuration version, pseudonymous user, question, answer, sources, best score, checks, time and cost.

U

Unsupported-claim rate: The share of answers containing a claim that its cited sources don't support. It is a label-free signal to watch in production.

V

Vector search: Search that embeds the query and returns the chunks whose vectors are nearest, matching on meaning rather than exact words.
Version question (superseded documents): A question type where an old document would give the wrong answer. The `current_only` setting drops archived and older versions of a title so only current documents are searched.
Virtual environment: A separate set of installed Python packages for a project. RAGAS gets its own to avoid version conflicts with the other packages.

W

Weak retrieval: A best rerank score so low that the right chunks probably weren't found. A rising share signals that people are asking about things the documents don't cover.
Wrong refusal: Saying "I don't know" when the documents do answer the question. The customer emails Priya, which costs time but not trust.