Glossary

164 terms from System Design Basics, in plain words, each with the chapter that teaches it.

Search the glossary

A

ACID (atomicity, consistency, isolation, durability): Four promises a database makes about transactions: all of a transaction happens or none of it, rules still hold, concurrent transactions don't see each other's half-done work, and committed changes survive a crash.
Alert: A message sent to a person when a metric crosses a set line, such as an error rate above 1% for 5 minutes.
API key: A long secret string a caller sends with each request to prove who it is.
Asynchronous work: Work done a moment later by something else, so the customer's request doesn't wait for it. Kettle & Crane sends the confirmation email this way.
At-least-once delivery: A queue promise that every message arrives, but sometimes twice, for example when a worker finishes the job and crashes before telling the broker.
Atomic operation: One step that can't be split, so nothing can happen between the check and the change. A single `UPDATE ... WHERE qty >= ?` checks and subtracts stock at once.
Authentication: Proving who the caller is.
Authorization: Deciding what a known caller is allowed to do, such as customers reading only their own orders.
Auto-increment ID: A number the database hands out in order (1, 2, 3...) to each new row. The URL shortener turns it into a short code.
Autoscaling: Automatically adding servers when load rises and removing them when it falls, by a rule such as keeping average CPU under 60%.
Availability: The share of time a system answers, as a percentage such as 99.9%.

B

Back pressure: Pushing back on producers when a queue is full, so the queue doesn't grow until memory runs out. A bounded queue makes `put` wait or `put_nowait` fail.
Back-of-the-envelope estimate: A rough calculation, such as requests per second or storage, whose goal is the right order of magnitude rather than exact numbers.
Backups: Copies of data kept in a separate place so it can be restored after a disaster such as a failed disk, a bad deploy or accidental deletion. A backup is only trustworthy once you have tested it by actually restoring from it.
Base62: Writing numbers with 62 symbols (0-9, a-z, A-Z) instead of 10, so codes stay short. The shortener builds it with Python's `divmod(a, b)`, which returns the whole-number result and the remainder at once (`divmod(17, 5)` is `(3, 2)`).
Batching: Grouping many writes into one, such as one `commit()` after many inserts, so the slow wait for disk happens once.
Body: The data part of an HTTP request or response, after the headers and a blank line, such as the JSON of a new order.
Broker: The program that stores a message queue and hands messages to workers. RabbitMQ and Amazon SQS are common brokers.

C

Cache: A small, fast store that keeps copies of data that is slow to get, such as a product page read thousands of times between price changes.
Cache hit / cache miss: A hit means the data was found in the cache; a miss means it wasn't and must be fetched from the slow source such as the database.
Cache invalidation: Deleting or updating a cached copy when the underlying data changes, for example deleting a product's key right after Jo changes its price.
Cache stampede: What happens when a hot cached key expires and many requests all miss at once and hit the database together.
Cache-aside: A caching pattern where the app checks the cache first and, on a miss, reads the database and stores the result in the cache.
Cache-Control header: An HTTP header telling browsers and CDNs how long to keep a file; `max-age=86400` means keep it for a day.
CAP theorem: The idea that during a network partition a system must choose between consistency (refuse rather than be wrong) and availability (answer, maybe with stale data).
CDN (content delivery network): Servers in many cities that hold copies of files such as product photos, so a customer in Miami gets them from nearby instead of Portland.
Celebrity problem: In a feed, a source with huge numbers of followers makes fan-out on write very expensive, and the last followers see posts late.
Circuit breaker: A guard that notices repeated failures calling another service and stops calling for a while. It is closed (calls go through), open (calls fail at once) or half-open (calls allowed again; the first success closes it).
Client: A program that asks for something, such as a browser or the `requests` library; a server answers it.
Clock (injectable clock / fake clock): A function that returns the current time, passed in as a parameter so tests can use a fake clock moved by hand instead of waiting. The default `time.monotonic` counts seconds from a fixed point and never jumps backward, unlike `time.time`.
Commit: Saving a transaction's changes to the database; a rollback undoes them all instead. Each commit waits for data to reach disk, so many writes per commit is faster.
Component: One part of a system, such as a web server, database, cache or queue.
Consensus: The process of getting several servers to agree on a single decision, such as which one is the new primary. It must work even when some servers or network links fail.
Consistency: Whether all readers see the latest data. Strong consistency means every read sees the latest write, while eventual consistency lets copies be briefly out of date until they catch up; in ACID transactions, it means rules such as "qty is never negative" still hold.
Consistent hashing: A sharding method that moves only a small share of keys when shards are added, instead of nearly all as with `%`. It is covered in the next book.
Content-Type: An HTTP header saying what kind of data the body holds, such as `application/json`.
Cross-shard query: A question that doesn't include the shard key, so it must ask every shard and combine the answers, like "best sellers this week".

D

Daemon thread: A Python thread that stops automatically when the main program ends, so a worker's endless loop doesn't keep the program alive.
Database: A program that stores data safely and finds it again, such as SQLite, PostgreSQL or MySQL.
Dead-letter queue: Where messages go after failing too many times, so a person such as Priya can inspect them instead of the system retrying forever.
Denormalization: Storing a copy of data in the shape a page needs, even though it repeats data held elsewhere, to make reads cheap.
Design document: A short shared description of what you'll build and why, reviewed by the team before anyone writes code.
DNS (Domain Name System): The internet's phone book: it turns a name like shop.kettleandcrane.com into an IP address.
Document store: A NoSQL database that stores JSON-like documents, good for products with varied fields. MongoDB is an example.
Downtime: The time a system is not answering. Each extra "nine" of availability cuts yearly downtime by ten.
Durability: A promise that saved data is never lost; once an order is confirmed or a transaction committed, it survives crashes.

E

Edge location: One of a CDN's servers in a city near users, which hold copies of files.
Endpoint: One method and path pair that an API accepts, such as `GET /v1/products`.
Error rate: The share of requests that fail, a common metric to watch and alert on.
Eventual consistency: Reads may be briefly out of date, but once writes stop every copy catches up, as with read replicas.
Eviction: Throwing entries out of a full cache to make room for new ones.
EXPLAIN QUERY PLAN: An SQLite command placed before a query that describes how the database would run it, such as `SCAN` or `SEARCH ... USING INDEX`, without running it.
Exponential backoff: Doubling the wait after each failed retry (0.1 s, 0.2 s, 0.4 s...) to give a struggling server room to recover.

F

Failover: Switching to a standby when the main part fails, for example promoting a replica to primary or a spare load balancer taking over an address.
Fallback: A simpler answer shown when a part fails, such as a cached "Popular this week" list when recommendations are down.
Fan-out: Copying one post or message to many readers.
Fan-out on write / read: Two ways to build feeds: on write, a new post is copied into every follower's feed list (slow post, fast read); on read, posts are stored once and gathered when a customer opens the app (cheap post, costly read).
Feed: A list of posts from the cafés a customer follows, newest first.
Functional requirement: A statement of what the system does, such as "customers can check out".

G

Graceful degradation: Offering less instead of nothing when a part fails, such as a cached list when recommendations are down.

H

Hash sharding: Choosing a shard by turning the key into a number with a stable hash and taking the remainder after dividing by the shard count.
Header: A `Name: value` line in an HTTP request or response carrying extra information, such as `Content-Type` or `Idempotency-Key`.
Health check: A regular test, such as `GET /v1/health`, that a server is answering. A load balancer marks servers that fail it as down.
Hit ratio: Cache hits divided by all lookups; 0.9 means nine lookups in ten are served by the cache and only one in ten reaches the database.
Horizontal scaling: Adding more machines that share the work. It has no ceiling, but data or requests must be split between them.
Hot key: One key, such as a very busy café, that gets so much traffic it makes its shard busier than the rest.
Hot spot: A shard or key that gets far more traffic than the rest, as when range-sharding by date sends every new order to the newest shard.
HTTP (Hypertext Transfer Protocol): The plain-text protocol browsers and servers use: a request with a method, path, headers and maybe a body goes out, and a response with a status code comes back.
HTTP method: The verb on an HTTP request's first line saying what to do: `GET` reads, `POST` creates, `PUT` replaces, `DELETE` removes.
HTTPS: HTTP encrypted with TLS so nobody in between can read things like card numbers or tokens.
Hybrid fan-out: Fan-out on write for ordinary cafés, but fetching very popular cafés' posts at read time and merging them in.

I

Idempotency key: A unique ID the app sends with a request, in an `Idempotency-Key` header, so the server can return the first result for a retry instead of, say, charging twice.
Idempotent: An operation is idempotent if doing it twice has the same effect as doing it once. `GET`, `PUT` and `DELETE` are; a plain `POST` that creates an order is not.
Idempotent consumer: A queue worker that records which messages it has handled and skips repeats, so a duplicate delivery has the same effect as one.
Index: A sorted list of one column's values, each pointing at its rows, so the database can jump to them instead of scanning the table. It slightly slows writes.
IP address: A number such as 203.0.113.7 that identifies a machine on the network.

J

Jitter: Randomness added to retry waits so many clients don't all retry at the same instant.
Join: Combining rows from several tables in one SQL query, a feature of relational databases.
JSON: A text format for structured data, written like Python dicts and lists, that APIs use for bodies.

K

Kafka: A popular message broker built around topics, used for event streaming; covered in the next book.
Key-value store: A NoSQL database holding one value per key, like a Python dict. It suits sessions and carts; Redis and DynamoDB are examples.

L

Latency: How long one request takes, measured in units such as milliseconds.
Layer 4 / Layer 7: A Layer 4 load balancer looks only at addresses and ports and passes bytes on unread (very fast); a Layer 7 balancer reads the HTTP request so it can route by path and handle HTTPS.
Least connections: A balancing rule that sends each request to the server with the fewest requests still open, which helps when requests differ in cost.
Load balancer: A server in front of the app servers that takes every request and passes it to one of them, so customers see only one address. nginx and HAProxy are common ones you run yourself.
Logs: Records of events, such as each failed charge, used to see what happened.
LRU (least recently used): An eviction rule that, when the cache is full, throws out the entry that hasn't been read for the longest time. Python's `collections.OrderedDict` makes it easy to build.

M

Message: One item of work placed on a queue, such as an order ID for the email worker.
Message queue: Holds messages (such as order IDs) from producers until consumers take them off, so slow work like emails doesn't delay the reply.
Metrics: Numbers measured over time, such as the error rate or latency.
Microservices: A design that splits a system into small programs that talk over APIs; the opposite of a monolith.
Monitoring: Watching what a running system is doing through logs, metrics and alerts.
Monolith: A system built as one program that does everything, in contrast to microservices, which split a system into small programs that talk over APIs. A monolith is simpler to build and run, and suits a small team.

N

Network partition: A network failure where some servers can't reach others, for example replicas losing contact with the primary.
Nines: A way of describing availability by counting 9s: 99.9% is "three nines". Each extra nine cuts allowed downtime by ten.
Non-functional requirement: A statement of how well the system does its job: latency, availability, durability and scalability.
NoSQL: Databases that aren't relational tables, giving up some features for simple, very large scale: key-value, document and wide-column stores.

O

Object storage: A service that stores any number of files by name, cheaply and durably. Kettle & Crane's photos live there, and the CDN fetches them from it.
Order of magnitude: The rough size of a number in powers of ten, such as 2, 200 or 20,000 requests a second; estimates aim for it.
Outbox: A queue of work (such as order IDs to email) filled only after an order is safely committed.

P

p99 (99th percentile): The time that 99% of requests beat; the slowest 1% take longer. An average of 80 ms can hide a p99 of 3 seconds.
Pagination: Returning a long list in pieces, using something like `?page=2` or `?after=K-1040`.
Peak factor: How many times the average load the busiest moment reaches; the shop's sale hour is about 10.
Polling: The client asking again and again whether anything is new, for example every few seconds in a chat.
Port: A numbered door on a machine that lets it run several network services at once, with each connection addressed to a specific number. For example, 443 is used for secure web traffic and 8121 for a local example API.
Precompute: Calculating a heavy result ahead of time, such as best sellers every 10 minutes, and storing it so each page view is a cheap read.
Presence: Whether a person is online. In chat it is tracked with a key that has a short TTL and expires when the person stops sending "I'm here".
Primary: The database copy that takes every write and copies changes to the read replicas.
Producer / consumer: The producer is the code that puts messages on a queue (e.g. checkout); the consumer, or worker, takes them off and does the work (e.g. sending email).
Pub/sub (publish/subscribe): A messaging style where one message published to a topic is delivered as a copy to every subscriber, unlike a queue where one consumer gets it.
pytest: A Python testing tool that runs test functions; `pytest -q` prints a short summary like `3 passed`. A fixture is a helper function pytest runs before each test that asks for it, such as one that makes a fresh database.

R

Race condition: A bug whose result depends on how two actions overlap in time, such as two checkouts both reading 1 bag in stock and both selling it.
Range sharding: Giving each shard a range of keys, such as January to March on shard 0. Range queries hit one shard, but a hot spot can form.
Rate limiting: Capping how many requests each client may make in a given time. It protects a service from being overloaded by a misbehaving script or abusive caller, which is then usually answered with a 429 Too Many Requests status.
Read replica: A copy of the database that receives each change from the primary and answers reads.
Read-heavy / write-heavy: A read-heavy system is read far more than changed (the shop is about 100 reads per order); a write-heavy one is the reverse. The ratio tells you which side to scale first.
Read-your-writes: A promise that a user always sees their own latest changes, for example by reading their data from the primary for a short time after they write.
Redirect: A response that tells the browser to go to another address, given in a `Location` header. 301 means moved for good (browsers remember it); 302 means moved for now (every click returns to you).
Redis / Memcached: Popular in-memory servers: Redis is a key-value store often used as a shared cache, and Memcached is a shared cache server for many app servers.
Redundancy: Having two or more of each important part, so one failing doesn't stop everything.
Relational database: A database of tables with fixed columns, queried in SQL, where tables can be combined in a query. Good for orders, stock and money.
Replication: Keeping copies of a database on several servers.
Replication lag: How far a replica is behind the primary, usually milliseconds but sometimes seconds, so a read just after a write can return the old value.
Requirement: Something the system must do or be. Functional requirements say what it does; non-functional ones say how well.
Resharding: Moving rows to new shards when shards are added; with `% shard_count` most keys move.
Resource: A thing an API names with a noun in its URL, such as `orders`; the HTTP method says what to do with it.
REST: A common API style where URLs name resources (nouns like `orders`) and HTTP methods say what to do with them.
Retry: Trying a failed call again, which helps with brief failures but should be done only for idempotent work, with backoff and jitter.
Round robin: Giving requests to servers in turn: app1, app2, app3, then app1 again.
Router: App code that sends writes to the primary database and reads to a replica.
RPS (requests per second): The key load number: average RPS is daily requests divided by 86,400 seconds, and peak RPS is that times the peak factor.

S

Scalability: Keeping working as the load grows.
Scan: The database reading every row of a table to answer a query, which is slow on big tables; an index avoids it.
Server: A program or machine that answers requests from clients.
Session: What a server knows about one visitor, such as who is logged in and what is in their cart.
Shard: One piece of a table that has been split across several databases.
Shard key: The column that decides which shard a row lives on; `customer_id` keeps a customer's orders together.
Sharding: Splitting one table's rows across several databases. It scales writes but adds costs such as hot keys, cross-shard queries and resharding, so shard only when you must.
Short code: The few symbols after the slash in a short link, such as `abc12`.
SLO (service level objective): A team's target for a service, such as "checkout is available 99.9% of the time".
SPOF (single point of failure): A part whose failure stops everything, such as one database or one load balancer.
SQL: The language used to ask questions of a relational database, such as `SELECT` and `UPDATE`.
Stable hash: A hash giving the same number for the same key on every machine and run, like `zlib.crc32`. Python's `hash()` for strings changes between runs, so it can't pick shards.
Stale: Cached or copied data that no longer matches the source, such as an old price.
Stateless server: A server that keeps no session data in its own memory, so any server can answer any request and adding servers is easy.
Status code: The number that starts an HTTP response: 2xx success, 4xx the client's mistake, 5xx the server's problem. Common ones: 200 OK, 201 Created, 400 Bad Request, 401 Unauthorized (not logged in), 403 Forbidden (not allowed), 404 Not Found, 409 Conflict, 429 Too Many Requests, 500 Internal Server Error, 503 Service Unavailable.
Sticky session: A load balancer sending one customer to the same server every time because it holds their cart in memory; if it dies, the cart is lost.
Strong consistency: Every read sees the latest write, as when reading from the primary.
Synchronous work: Work the reply waits for, such as saving the order at checkout.
System design: Deciding the parts of a system and how they talk to each other, before or while you build it.

T

TCP (Transmission Control Protocol): The connection protocol that delivers bytes in order and resends lost ones; HTTP runs on top of it.
Thread: A separate line of execution inside one program, letting it work on several things at once. For example, a server can handle each request in its own thread so a slow call doesn't block the others.
Timeout: The longest you will wait for a reply to a network call, such as `requests.get(url, timeout=2)`. Without one a hung server can tie up every thread.
TLS (Transport Layer Security): The encryption that HTTPS uses so nobody between browser and server can read the traffic.
Token: A long secret string sent in a header such as `Authorization: Bearer <token>` to prove who is calling. The server answers 401 if it's missing or wrong, and 403 if the caller isn't allowed.
Transaction: A group of statements whose changes are saved all together or not at all; in Python's sqlite3 a `with con:` block commits on success and rolls back on error.
TTL (time to live): How many seconds a cached entry stays valid before it expires, so the next lookup misses and fetches fresh data.

V

Versioning: Putting a version such as `/v1/` in API paths so a `/v2/` can appear while old apps keep working.
Vertical scaling: Moving to a bigger machine. It's simple, but there is a largest machine you can buy.

W

WebSocket: A connection that starts as an HTTP request and then stays open, so either side can send messages at any time. The support chat uses it to push messages.
Weighted round robin: Round robin where bigger servers appear more often in the rotation and so get more requests.
Whiteboard sketch: A first design drawing: boxes for components and arrows for the messages between them.
Wide-column store: A NoSQL database with huge tables spread over many machines, suited to billions of events. Cassandra is an example.
Write-through cache: A cache that is written together with the database at the same time, instead of being invalidated.