Glossary

176 terms from System Design Basics in C++, in plain words, each with the chapter that teaches it.

Search the glossary

A

ACID (Atomicity, Consistency, Isolation, Durability): The four promises a database makes about transactions: all-or-nothing changes, rules still hold afterward, concurrent transactions don't see each other's half-done work, and committed changes survive a crash.
Alert: An automatic message to a person when a metric crosses a line, such as an error rate above 1% for 5 minutes.
Amazon S3: A well-known object storage service that stores files by name. Kettle & Crane's CDN could fetch product photos from it.
API (Application Programming Interface): The set of requests a program accepts from other programs, such as the shop's endpoints for products and orders.
API key: A secret string a caller sends, usually in a header, to prove who it is.
Asynchronous work: Work done a moment later by something else, so the reply doesn't wait for it. Kettle & Crane's confirmation email is a typical example.
At-least-once delivery: A promise most message brokers make: every message arrives, but some may arrive twice. Consumers must therefore handle duplicates.
atomic: A single step that can't be split, so nothing can happen between the check and the change. It prevents two checkouts from selling the last bag.
Authentication: Proving who the caller is, usually with an API key or token.
Authorization: Deciding what an already-identified caller is allowed to do, such as customers reading only their own orders.
auto-increment ID: A number the database hands out in order (1, 2, 3...) for each new row. The URL shortener turns it into a short code.
Autoscaling: Automatically adding servers when load rises and removing them when it falls, by a rule such as keeping average CPU under 60%.
Availability: The share of time a system answers, as a percentage such as 99.9%.

B

Back pressure: Pushing back on producers when a queue is full, so that memory doesn't run out. For example, a bounded queue's `put` can return `false` when it is full, so the producer can slow down, retry later or tell the customer.
back-of-the-envelope estimate: A rough calculation, such as requests per second or storage, done to find the order of magnitude before designing.
Backups: Copies of the data kept elsewhere so it can be restored after a disaster. They should be tested by actually restoring one.
Base62: Writing numbers with 62 symbols (0-9, a-z, A-Z) instead of 10, so a short code stays short. Six symbols cover about 56.8 billion IDs.
Batching: Grouping many writes into one, such as one commit or flush after many inserts. It waits for the slow step once instead of many times.
Bearer token: A token sent in the `Authorization: Bearer <token>` header to prove who is calling.
body (HTTP body): The data that follows the headers and a blank line in an HTTP request or response, such as a new order's JSON.
Broker: The program that stores a message queue, so workers can run on other machines and messages survive a restart. RabbitMQ and Amazon SQS are examples.

C

Cache: A small, fast store of copies of data that is slow to get, such as a product page read thousands of times between price changes.
Cache hit / cache miss: A hit means the data was found in the cache. A miss means it wasn't, so you fetch it from the slow source.
Cache invalidation: Removing or updating a cached copy when the underlying data changes, so readers don't see stale data. A common approach is to write to the database and then erase the cached key.
Cache stampede: A hot key expires and all requests miss at once, flooding the database. The fix is to let one request reload it while the others wait.
Cache-aside: A pattern where the app checks the cache first and, on a miss, reads the database and stores the result in the cache.
Cache-Control header: An HTTP header telling browsers and CDNs how long to keep a file. `max-age=86400` means keep it for a day.
CAP theorem: During a network partition, a system must choose between consistency (refuse rather than be wrong) and availability (answer, maybe stale). Without a partition you can have both.
CDN (content delivery network): Servers in many cities, called edge locations, that hold copies of files so a customer in Miami gets Kettle & Crane's photos from nearby instead of Portland.
celebrity problem: A very popular source with huge numbers of followers makes fan-out on write slow, so late followers see posts late. Hybrid fan-out is the usual fix.
Circuit breaker: A guard that notices repeated failures of a service and stops calling it for a while. It is closed while calls go through, open when calls fail at once, and half-open when a trial call is allowed.
client: A program that asks for something, like a browser. A server answers.
clock (injectable / fake clock): A function returning the current time, passed into a class so tests can move time by hand instead of sleeping.
commit and rollback: In a transaction, commit makes all the changes permanent, and rollback discards them. For example, a working copy of the stock either replaces the real stock (commit) or is thrown away (rollback).
Component: One part of a system, such as a web server, database, cache or queue.
Consensus: Getting several servers to agree on something, such as which server is the new primary. It is a hard problem in distributed systems, because servers and networks can fail while they are trying to agree.
Consistency: Whether all readers see the latest data. Strong consistency means every read sees the latest write, while eventual consistency means reads may be briefly out of date but all copies catch up once writes stop.
Consistent hashing: A sharding method that moves only a small share of keys when shards are added. Going from four to five shards moves about a fifth instead of about 80%.
Content-Type / Content-Length: HTTP headers that say what kind of data the body holds, such as JSON, and how many bytes it has.
cross-shard query: A question that doesn't include the shard key, so it must ask every shard and combine the answers, such as best sellers this week.

D

data race: Two threads using the same data at once with at least one writing and no lock. In C++ this is undefined behavior, which a mutex prevents.
Dead-letter queue: Where messages go after failing too many times, so a person such as Priya can look at them instead of the system retrying forever.
deadlock: A hang where two threads each hold one mutex and wait for the other. Locking in the same order everywhere, or using `std::scoped_lock`, avoids it.
Denormalization: Storing a copy of data in the shape a page needs, even though it repeats data held elsewhere. It trades some staleness and storage for fast reads.
design document: A short shared description of what you'll build and why, which the team reviews before anyone writes code.
DNS (Domain Name System): The internet's phone book. It turns a name like shop.kettleandcrane.com into an IP address.
Document store: A NoSQL database that holds JSON-like documents, good for products with varied fields. MongoDB is an example.
downtime: The time a system is not answering. Each extra nine of availability cuts it by ten.
Durability: Saved data is never lost. A committed order survives a crash.

E

edge location: A CDN server in one of many cities that holds copies of files close to users.
Endpoint: One method-and-path pair that an API accepts, such as `GET /v1/products`.
error rate: The share of requests that fail, a typical metric to watch and alert on.
Eventual consistency: Reads may be briefly out of date, but once writes stop, every copy catches up. Read replicas behave this way.
eviction: Throwing entries out of a full cache to make room.
EXPLAIN: An SQL command that shows how the database plans to run a query, for example by scanning or by using an index.
Exponential backoff: Doubling the wait after each failed try (0.1 s, 0.2 s, 0.4 s...) to give a struggling server room to recover.

F

Failover: Switching to a standby when the main part fails, such as promoting a replica to primary.
Fallback: A simpler answer shown when a part fails, such as a cached Popular this week list when recommendations are down.
Fan-out: Copying one post or message to many readers.
fan-out on write / fan-out on read: On write, a post is copied into every follower's feed when posted, so reading is fast. On read, posts are stored once and gathered when the customer opens the app.
feed: A list of posts from the cafés a customer follows, newest first.
five-step design method: A routine for working through any system design: requirements, estimates, API, data, then a sketch and the hard part. It keeps a design driven by what the system must do and how big it is, and puts most of the effort into the one difficult part.
flush: Pushing buffered output out to the operating system. `std::endl` flushes each time, while `'\n'` just buffers.
FNV-1a: A small, well-known hash function that gives the same result on every machine and run, used for stable sharding.
fsync: An operating-system call that waits until data is truly on disk. A real database commit waits on it, which takes milliseconds.
Functional requirement: What the system does, such as customers can check out or Priya can look up an order by ID.

G

GB (gigabyte): A decimal gigabyte, 1,000,000,000 bytes (a KB is 1,000 bytes). For estimates the difference from 1,024-based units doesn't matter.
Graceful degradation: Offering less instead of nothing when a part fails, such as a cached list when recommendations are down.

H

Hash sharding: Picking a shard by turning the key into a number with a stable hash and taking the remainder after dividing by the shard count.
header: A `Name: value` line in an HTTP request or response that carries extra information, such as Content-Type or Authorization.
Health check: A regular test, such as `GET /v1/health`, that a server is answering. Servers that fail are marked down and get no requests.
hit ratio: Cache hits divided by all lookups. A ratio of 0.9 means one lookup in ten reaches the database.
Horizontal scaling: Adding more machines to share the work. It has no ceiling, but the data must be split between them.
Hot spot: A shard or key that gets far more traffic than the rest, such as the newest shard when sharding by date.
HTTP (HyperText Transfer Protocol): The plain-text language browsers and servers speak: a request goes out with a method, path and headers, and a response comes back with a status code and body.
HTTP request and response: A request has a method, path, headers and maybe a body; the response sent back has a status code, headers and a body.
HTTP status codes: The three-digit number that starts an HTTP response: 2xx means success, 4xx is the client's mistake and 5xx is the server's problem. Common codes are 200 OK, 201 Created, 301 moved for good, 302 moved for now, 400 bad request, 401 caller not identified (missing or wrong token), 403 caller known but not allowed, 404 not found, 409 conflict with current state, 429 too many requests, 500 server bug or crash, and 503 service unavailable.
HTTPS: HTTP encrypted with TLS, so nobody in between can read card numbers or tokens.
hybrid fan-out: Fan out on write for ordinary sources, but fetch the few very popular sources' posts at read time and merge them in.

I

Idempotency key: A unique ID the app sends with a request so the server can recognize a retry and return the first result instead of acting twice.
Idempotent: Doing it twice has the same effect as doing it once. `GET`, `PUT` and `DELETE` are idempotent, while a plain `POST` is not.
idempotent consumer: A queue worker that records which messages it has handled and skips repeats, so a duplicate delivery has no extra effect.
Index: A sorted list of one column's values pointing at their rows, so the database can jump straight there instead of scanning every row. It costs some time and memory on writes.
IP address: A number such as 203.0.113.7 that identifies a machine on the network.

J

Jitter: Randomness added to retry waits, picking a random delay up to the backoff limit, so many clients don't all retry at the same instant.
join (SQL): Combining tables in one query, one of the features relational databases offer.
JSON: A text format for data using braces, names and values, such as `{"name": "Speckled blue mug", "price": 18}`. Web APIs commonly use it for request and response bodies.

K

Kafka: A popular message broker built around topics, used for event streaming. The next book covers it.
Key-value store: A NoSQL database holding one value per key, like a hash map. Good for sessions, carts and the URL shortener; Redis and DynamoDB are examples.

L

Latency: How long one request takes, measured in milliseconds or smaller units.
Layer 4 / Layer 7: A Layer 4 balancer looks only at addresses and ports and passes bytes on unread, so it's fast. A Layer 7 balancer reads the HTTP request, so it can route by path and handle HTTPS.
Least connections: A balancing rule that sends each request to the server with the fewest requests still open. It copes better than round robin when requests differ in cost.
Load balancer: A server in front of the app servers that takes every request and passes it to one of them. Customers only see its address.
Logs: Records of events, such as each failed charge, kept for later investigation.
LRU (least recently used): A rule for a full cache: evict the entry that has gone unread for the longest time. In C++ it can be built from a `std::list` that keeps entries in order of use plus a `std::unordered_map` that finds any entry's place in the list at once.

M

Memcached: A shared cache server, like Redis, that several app servers can use.
Message queue: A holding area where producers put messages until consumers take them off, so slow work doesn't delay the reply.
method (HTTP method): The first word of a request saying what to do: `GET` reads, `POST` creates, `PUT` replaces and `DELETE` removes.
metrics: Numbers measured over time, such as the error rate or latency.
microservices: A style that splits a system into small programs that talk over APIs, in contrast to a monolith.
Monitoring: Watching what the system is doing through logs, metrics and alerts.
Monolith: One program that does everything, as opposed to microservices, which split a system into small programs that talk over APIs. A monolith with a few helpers suits a small team such as Kettle & Crane's.
mutex (std::mutex): A lock that lets only one thread at a time use shared data. It is used to guard shared data such as queues and stock counts, often through a `std::lock_guard` that unlocks automatically.

N

Network partition: A failure where some servers can't reach others, such as replicas losing contact with the primary.
nginx / HAProxy: Two common load balancers that you run yourself.
nines: A shorthand for availability: 99.9% is three nines. Each extra nine cuts allowed downtime by ten and costs much more to build.
Non-functional requirement: How well the system does its job: latency, availability, durability and scalability.
NoSQL: Databases that aren't relational tables, giving up some features for simple, very large scale. Examples are key-value, document and wide-column stores.

O

Object storage: A service that stores any number of files by name, cheaply and durably. Product photos live there and a CDN fetches them.
order of magnitude: The rough size of a number: is it 2, 200 or 20,000? Each answer needs a very different design.
outbox: A thread-safe queue of work to do once an order is safely saved, such as the email worker's order IDs.

P

p99 (percentile): The time that 99% of requests beat, so the slowest 1% take longer. An 80 ms average can hide a p99 of 3 seconds.
Pagination: Returning a long list one page at a time, with `?page=2` or `?after=K-1040`.
path: The part of a request saying which thing you want, such as `/v1/products/MUG-BLUE`.
Peak factor: How many times the average load the busiest moment reaches. For the holiday sale hour it's about 10.
Polling: Asking again and again whether anything is new, such as a browser checking for chat messages every few seconds.
port: A numbered door on a machine, such as 443 for secure web traffic.
Precompute: Calculating a heavy result ahead of time and storing it for fast reads, such as best sellers saved every 10 minutes.
presence: Whether a user is online. In the chat design it's a key with a 30 second TTL refreshed every 10 seconds.
Primary: The database copy that takes every write and copies changes to the replicas.
Producer / consumer: The producer is code that puts messages on a queue, such as checkout. The consumer or worker takes them off and does the work.
Pub/sub (publish/subscribe): Delivering a copy of each message to every subscriber of a topic, unlike a queue, which gives each message to one consumer.

R

Race condition: A bug where the result depends on how two operations overlap in time, such as two checkouts both selling the last bag.
Range sharding: Giving each shard a range of keys, such as months of the year. It makes range queries easy but risks hot spots.
rate limiting: Capping how many requests each client may make in a given time. It protects a service from being overwhelmed by a script or a misbehaving client.
Read replica: A copy of the database that receives the primary's changes and answers reads, so more servers share the read load.
read-heavy / write-heavy: A read-heavy system reads far more than it changes, as the shop does at about 100 to 1. A write-heavy one, such as one logging every click, is the reverse.
Read-your-writes: A promise that a user always sees their own latest changes, for example by reading their data from the primary for a short time after they write.
Redirect: A 301 (moved for good) or 302 (moved for now) response with a `Location` header that tells the browser to go to another URL. Browsers remember a 301 and may skip your server, so use 302 if you must count every click.
Redis: A key-value store often used as a shared cache server.
redundancy: Having two or more of each part, so one failure doesn't stop everything.
relational database: A database of tables with fixed columns, queried with SQL, where tables can be combined. Best for orders, stock and money; SQLite, PostgreSQL and MySQL are examples.
Replication: Keeping copies of a database on several servers.
Replication lag: How far a replica is behind the primary. A read just after a write may return the old value.
requirement: Something the system must do or be. Functional ones say what it does and non-functional ones say how well.
resharding: Moving rows to their new shards after adding a shard. With the `%` rule about 80% of keys move.
resource: A thing an API names with a noun in the URL, such as `orders`, while the method gives the verb.
REST: A common style for web APIs where URLs name resources with nouns and HTTP methods say what to do with them.
retry: Trying a failed call again. It should be limited to idempotent work and use backoff with jitter.
Round robin: Handing requests to servers in turn: app1, app2, app3, then back to app1.
router: The part of the app that sends writes to the primary and reads to a replica.
RPS (requests per second): The key sizing number: the average is daily requests divided by 86,400, and the peak is that times the peak factor.

S

Scalability: Keeping working as the load grows.
Scan: The database reads every row of a table to answer a query, which gets slower as the table grows.
Session: What a server knows about one visitor, such as who is logged in and what's in their cart.
Shard: One piece of a table that has been split across several databases.
Shard key: The column that decides which shard a row lives on, such as `customer_id` for orders.
Sharding: Splitting one table's rows across several databases, each holding a shard. Do it only when one well-indexed database isn't enough.
Short code: The few symbols after the slash in a short link, such as `abc12`.
SKU (stock keeping unit): A product's unique code, such as `MUG-BLUE`. It is used as a key to look up the product in data such as prices and stock.
SLO (service level objective): A team's target for a service, such as checkout available 99.9% of the time.
SPOF (single point of failure): One part whose failure stops everything, such as a lone database. Redundancy removes it.
SSD (solid-state drive): A disk with no moving parts, much faster than a spinning hard disk. Reading 4 KB takes about 0.1 ms.
stable hash: A hash giving the same number for the same key on every machine and run. `std::hash` isn't guaranteed stable, so sharding uses FNV-1a instead.
stale: Cached or copied data that no longer matches the source of truth.
Stateless server: A server that keeps no session data in its own memory, so any server can answer any request and adding servers is easy.
Sticky session: A load balancer sending a user to the same server every time because that server holds their cart. If it dies, the cart is lost.
Strong consistency: Every read sees the latest write, as when you read from the primary.
synchronous work: Work the reply must wait for, such as saving the order at checkout.
system design: Deciding the parts of a system and how they talk to each other, before or while you build it.

T

TCP (Transmission Control Protocol): The connection protocol that delivers bytes in order and resends lost ones.
time units (ms, µs, ns): A millisecond is a thousandth of a second, a microsecond a millionth and a nanosecond a billionth; latencies are quoted in them.
Timeout: The longest you will wait for a reply to a network call. Without one, a hung server can tie up every thread.
TLS (Transport Layer Security): The encryption HTTPS uses so nobody in between can read the traffic.
token: A long secret string sent in a header, such as `Authorization: Bearer <token>`, to prove who the caller is.
topic: A named channel in pub/sub. Every subscriber gets a copy of each message published to it.
Transaction: A group of changes saved all together or not at all, such as reserving every product in a cart or none.
TTL (time to live): How many seconds a cached entry stays valid. After that it expires, so the next lookup misses and fetches a fresh copy.

U

URL shortener: A service that turns long links into short ones like `go.kettleandcrane.com/abc12` and redirects visitors.

V

Versioning: Putting a version such as `/v1/` in API paths so a `/v2/` can appear while old apps keep working.
Vertical scaling: Buying a bigger machine. It's simple, but there's a largest machine you can buy.

W

WebSocket: A connection that starts as an HTTP request and then stays open, so either side can send a message at any time. The chat server pushes messages down it.
Weighted round robin: Round robin where bigger servers appear more often and get more requests, such as listing app1 twice.
whiteboard sketch: A first design drawn as boxes for components and arrows for the messages between them.
wide-column store: A NoSQL kind with huge tables over many machines, good for billions of events. Cassandra is an example.
Write-through cache: A cache written together with the database at the same time, instead of being erased on change.