Glossary

161 terms from System Design Basics in Java, in plain words, each with the chapter that teaches it.

Search the glossary

A

ACID (atomicity, consistency, isolation, durability): The four promises a database transaction makes: all of it happens or none, rules still hold, concurrent transactions don't see each other's half-done work, and committed changes survive a crash.
Alert: An automatic message to a person when a metric crosses a line, such as an error rate above 1% for 5 minutes.
API key: A long secret string a caller sends to prove who it is, so the server can look it up and decide whether to answer.
assert: A Java statement that does nothing when its condition is true and throws an AssertionError when false, but only if you run the program with `java -ea`.
Asynchronous work: Work done a moment later by something else, so the request doesn't wait for it, like Kettle & Crane's confirmation email after checkout.
At-least-once delivery: A queue promise that every message arrives, but sometimes twice, for example when a worker did the job and crashed before telling the broker.
Atomic operation: One step that can't be split, so nothing can happen between the check and the change, such as an UPDATE ... WHERE qty >= ? that sells the last bag only once.
Authentication: Proving who the caller is, usually with an API key or token.
Authorization: Deciding what an already-identified caller is allowed to do, such as a customer reading only their own orders.
Auto-increment ID: A number the database hands out in order (1, 2, 3...) to each new row, used by the shortener to make unique codes.
Autoscaling: Automatically adding servers when load rises and removing them when it falls, by a rule such as keeping average CPU under 60%.
Availability: The share of time a system answers, given as a percentage such as 99.9%; the time it doesn't answer is downtime.

B

Back pressure: Pushing back on producers when a queue is full, for example with a bounded queue where offer returns false, so the queue doesn't grow until memory runs out.
Back-of-the-envelope estimate: A rough calculation, such as requests per second or storage, whose goal is the right order of magnitude rather than an exact number.
Backups: Copies of data kept in a separate place so it can be restored after a disaster, such as a lost disk or a deleted table. A backup is only trustworthy once you have tested it by actually restoring one.
Base62: Writing a number with 62 symbols (0-9, a-z, A-Z) instead of 10, so a link ID becomes a short code like ww8.
Batching: Grouping many writes into one, such as a single commit or disk sync after many inserts, so you wait once instead of many times.
BlockingQueue: A Java queue that threads can share safely: put adds, take removes and waits while empty, and an ArrayBlockingQueue has a size limit.
Broker: The program that stores a message queue, so workers can run on other machines and messages survive a restart.

C

Cache: A small, fast store holding copies of data that is slow to get, such as a product page read thousands of times between price changes.
Cache hit / miss: A hit means the data was found in the cache; a miss means it wasn't and must come from the slow source. The hit ratio is hits divided by all lookups.
Cache invalidation: Deleting or updating a cached copy when its data changes, such as writing Jo's new price to the database and then deleting the cache key.
Cache stampede: What happens when a hot cached key expires and all requests miss at once and hit the database together; the fix is to let one request reload it while others wait.
Cache-aside: The usual caching pattern: the app checks the cache first, and on a miss reads the database and stores the result in the cache.
Cache-Control header: An HTTP header that tells browsers and CDNs how long to keep a file; `max-age=86400` means keep it for a day.
CAP theorem: During a network partition a system must choose consistency (refuse rather than be wrong) or availability (answer, maybe stale); you can't rule partitions out.
CDN (content delivery network): Servers in many cities that keep copies of files such as product photos, so customers get them from nearby instead of from Portland.
Celebrity problem: When one source has huge numbers of followers, fan-out on write means tens of thousands of writes per post and late delivery; hybrid fan-out is the usual fix.
Circuit breaker: A guard that notices repeated failures of a service and stops calling it for a while: closed lets calls through, open fails them at once, half-open lets a trial call through after a wait.
Client / server: A client is a program that asks for something, like a browser; a server is the program that answers.
Clock (injectable / fake clock): A function that returns the current time; passing it as a parameter lets a test use a fake clock moved by hand instead of sleeping.
Commit / rollback: In a transaction, committing keeps all the changes; rolling back undoes them all. A commit also waits until data is safely on disk.
Component: One part of a system, such as a web server, database, cache or queue.
Consensus: The problem of getting several servers to agree on one answer, for example which replica becomes the new primary after the old one fails. It is needed because servers and networks can fail, and the servers must still not disagree.
Consistency: Whether all readers see the latest data; it can be strong or eventual.
Consistent hashing: A sharding method that assigns keys to shards so that adding a shard moves only a fraction of the keys (about a fifth when going from four to five shards), instead of most of them as with a simple `%` remainder. It makes resharding much cheaper.
Content-Type / Content-Length: HTTP headers saying what kind of data the body holds (such as application/json) and how many bytes it has.
CRC32: A built-in Java checksum class that turns bytes into a large whole number; its published definition makes it a stable hash usable for choosing a shard.
Cross-shard query: A question that doesn't include the shard key, so it must ask every shard and combine the answers, such as best sellers this week.

D

Dead-letter queue: A place where messages go after failing too many times, waiting for a person such as Priya to look at them instead of being retried forever.
Denormalization: Storing a copy of data in the shape a page needs, even though it repeats data held elsewhere, trading some staleness and storage for fast reads.
Design document: A short shared description of what you'll build and why, reviewed by the team before anyone writes code.
DNS (Domain Name System): The internet's phone book: it turns a name like shop.kettleandcrane.com into an IP address.
Document store: A NoSQL database that stores JSON-like documents, good for products with varied fields.
DoubleSupplier: A ready-made Java interface for a function that takes nothing and returns a double (called with getAsDouble); Supplier<T> does the same for any type T. Used here to pass in a clock or a piece of work to retry.
Downtime: The time a system is not answering; for example 99.9% availability allows about 8.76 hours of downtime a year.
Durability: Saved data is never lost, even after a crash.

E

Edge location: One of the CDN's servers in a city that holds copies of files close to customers.
Endpoint: One method-and-path pair an API accepts, such as `GET /v1/products`.
Error rate: The share of requests that fail, a common metric to watch and alert on.
Eventual consistency: Reads may be briefly out of date, but once writes stop every copy catches up, as with read replicas.
Eviction: Throwing entries out of a full cache to make room.
EXPLAIN: A SQL command that shows how the database will answer a query, such as whether it scans the table or uses an index.
Exponential backoff: Doubling the wait after each failed retry (0.1 s, 0.2 s, 0.4 s...) to give a struggling server room to recover.

F

Failover: Switching to a standby when the main part fails, such as promoting a replica to primary.
Fallback: A simpler answer shown when a part fails, such as a cached 'Popular this week' list when recommendations are down.
Fan-out: Copying one post or message to many readers.
Fan-out on write / read: On write, a post is pushed into every follower's feed list when posted (slow post, fast read); on read, posts are stored once and gathered and sorted when a customer opens the feed.
Feed: A list of posts from the cafés a customer follows, newest first.
Functional requirement: A statement of what the system does, such as 'customers can check out'.

G

Graceful degradation: Offering less instead of nothing when a part fails, such as queuing emails when the mail service is down.

H

Hash sharding: Picking a shard by turning the key into a number with a stable hash and taking the remainder after dividing by the shard count.
Health check: A regular test, such as GET /v1/health, that a server is answering; a server that fails is marked down and gets no requests until it passes again.
Horizontal scaling: Adding more machines that share the work; it has no ceiling, but data and requests must be split between them.
Hot key: A single key, such as a café with many orders or a popular cached entry, that gets far more traffic than the rest and can overload its shard.
Hot spot: A shard or key that gets far more traffic than the rest, as with range sharding by date where all new orders hit the newest shard.
HTTP (HyperText Transfer Protocol): The plain-text protocol browsers and servers speak: a request goes out, a response comes back.
HTTP request parts (method, path, headers, body): A request's first line has the method (GET read, POST create, PUT replace, DELETE remove) and path; headers follow as Name: value lines, then maybe a body such as order JSON.
HTTPS and TLS: HTTPS is HTTP encrypted with TLS so nobody in between can read things like card numbers.
HttpServer / HttpClient: JDK classes: HttpServer runs a small web server in Java and HttpClient is a web client like a browser without a window.
Hybrid fan-out: Fanning out on write for ordinary cafés but fetching the few very popular cafés' posts at read time and merging them in.

I

Idempotency key: A unique ID the app makes for each checkout and sends in an Idempotency-Key header, so the server returns the first order for a repeat instead of creating a second.
Idempotent: Doing it twice has the same effect as doing it once; GET, PUT and DELETE are idempotent, POST is not unless you add an idempotency key.
Idempotent consumer: A queue worker that records which messages it has handled and skips repeats, so a duplicate delivery has no extra effect.
Index: A sorted list of one column's values pointing at their rows, so the database can jump straight to matches instead of scanning every row, at a small cost to writes.
IP address: A number such as 203.0.113.7 that identifies a machine on the internet.

J

JDBC: Java's standard API for connecting to a SQL database through a driver, so programs can send queries and read results. The JDK itself ships no database, so a driver for the chosen database must be added.
Jitter: Randomness added to retry waits (a random wait up to the backoff limit) so many clients don't retry at the same instant.
JSON: A text format for data using braces and key-value pairs, such as {"name": "Speckled blue mug", "price": 18}, used for API bodies.

K

Kafka: A popular message broker built around topics, used for event streaming; covered only in the next book.
Key-value store: A NoSQL database holding one value per key, like a Map; good for sessions and carts (examples: Redis, DynamoDB).

L

Latency: How long one request takes, measured in units such as milliseconds.
Layer 4 / Layer 7: A Layer 4 balancer looks only at addresses and ports and passes bytes unread, so it's very fast; a Layer 7 balancer reads the HTTP request so it can route by path and handle HTTPS.
Least connections: A balancing rule that sends each request to the server with the fewest requests still open, which suits requests of unequal cost.
LinkedHashMap: A Java map that remembers the order keys were added, used to build an LRU cache and to keep printed maps in a fixed order.
Load balancer: A server in front of the app servers that takes every request and passes it to one of them; customers only see its address.
Logs / metrics / monitoring: Monitoring shows what's happening: logs record events such as failed charges, and metrics are numbers over time such as the error rate.
LRU (least recently used): A cache eviction rule that throws out the entry that has gone unread the longest.

M

Message queue: Holds messages (such as order IDs) from producers until consumers take them off, so slow work can happen after the reply.
Microservices: A style of building a system as many small programs that each do one job and talk to each other over APIs. It is the alternative to a monolith, where one program does everything.
Monolith: One program that holds all the features.

N

Network partition: A network failure in which some servers can't reach others, such as replicas losing contact with the primary.
nginx / HAProxy: Two common load balancer programs that you run yourself.
Nines: A way to describe availability by counting 9s: 99.9% is 'three nines'; each extra nine cuts allowed downtime by ten.
Non-functional requirement: How well the system does its job: latency, availability, durability and scalability.
NoSQL: Databases that aren't relational tables, giving up some features for simple, very large scale, such as key-value and document stores.

O

Object storage: A service that stores any number of files by name, cheaply and durably; the CDN fetches product photos from it. Amazon S3 is the best known.
Order of magnitude: A rough size such as 2, 200 or 20,000 requests a second, which is what an estimate aims for.
Outbox: A queue of work (here order IDs) to do only after the order is safely saved.

P

p99: The latency that 99% of requests beat, so the slowest 1% take longer; an average of 80 ms can hide a p99 of 3 seconds.
Pagination: Returning a long list one page at a time, with something like ?page=2 or ?after=K-1040.
Peak factor: How many times the average load the busiest moment reaches; the shop's holiday sale hour is about 10.
Poison pill: A special message, such as "STOP", that tells a queue worker to shut down.
Polling: Asking again and again whether anything is new; simple but slow to notice messages and wasteful when there are none.
Port: A numbered door on a machine, such as 443 for secure web traffic.
Precompute: Calculate a heavy result ahead of time, like best sellers every 10 minutes, and store it so each page view reads a few rows.
Presence: Who is online; the chat design tracks it with a TTL cache key refreshed every 10 seconds.
Primary: The database copy that takes every write and copies changes to replicas.
Producer / consumer: The producer is the code that puts messages on a queue (checkout); the consumer, or worker, takes them off and does the work.
Pub/sub (publish/subscribe): One message published to a topic is delivered as a copy to every subscriber, such as email, shipping and sales-report services all hearing 'order placed'.

R

RabbitMQ: Common message brokers (RabbitMQ and Amazon SQS) that store queues on their own server.
Race condition: A bug whose result depends on how two operations overlap in time, such as two checkouts both reading 1 bag and both selling it.
Range sharding: Giving each shard a range of keys, such as January to March on shard 0; range queries hit one shard but a hot spot can form on the newest shard.
Rate limiting: Capping how many requests each client may make in a given time, so that one client or script cannot overload a service. Requests over the limit are typically refused, often with a 429 status code.
Read replica: A copy of the database that receives the primary's changes and answers reads.
Read-heavy / write-heavy: A read-heavy system is read far more often than changed (the shop's ratio is about 100 to 1); a write-heavy one is the reverse. The ratio tells you which side to scale first.
Read-your-writes: A promise that a user always sees their own latest changes, for example by reading their data from the primary for a short time after they write.
Redirect: A 301 or 302 response that sends the browser to another address given in a Location header; 301 is moved for good (browsers remember), 302 moved for now (every click returns to you and can be counted).
Redis / Memcached: Two common shared cache servers that keep hot data in memory; Redis is also a key-value store.
Redundancy: Having two or more of each part so one failure doesn't stop everything.
Relational database: A database made of tables with fixed columns, queried in SQL, where tables can be combined in a single query (a join). It is a good default for data that must add up, such as orders, stock and money.
Replication: Keeping copies of a database on several servers.
Replication lag: How far a replica is behind the primary: right after Jo's price change a replica still returns the old price.
Requirement: Something the system must do or be; functional requirements say what it does, non-functional ones say how well.
Resharding: Moving rows to their new shards after changing the number of shards; with % shardCount most keys move.
REST: A common API style where URLs name resources (nouns like orders) and HTTP methods say what to do with them.
Retry: Trying a failed call again; worth it for brief failures, but only for idempotent work and with backoff and jitter.
Round robin: Giving requests to servers in turn: app1, app2, app3, then app1 again.
Router (read/write): App code that sends writes to the primary database and reads to the replicas.
RPS (requests per second): The key estimate for load: daily requests divided by 86,400 gives the average, and times the peak factor gives the peak.

S

Scalability: Keeping working as the load grows.
Scan: The database reads every row of a table to answer a query, which is slow on big tables.
Seed (random): A starting value for a random generator; with a fixed seed it produces the same numbers on every run, which makes demos repeatable.
Session: What a server knows about one visitor, such as who's logged in and what's in their cart.
Shard / sharding: Splitting one table's rows across several databases; each piece is a shard.
Shard key: The column that decides which shard a row lives on, such as customer_id for orders.
Short code: The few symbols after the slash in a short link, such as abc12.
SKU (stock keeping unit): A product's unique ID code in the shop, such as MUG-BLUE.
SLO (service level objective): A team's target for a service, such as checkout being available 99.9% of the time.
SPOF (single point of failure): One part whose failure stops everything, such as a lone database.
Stable hash: A hash that gives the same number for the same key on every machine and every run, with a published definition so any language can match it; Java's String.hashCode isn't suitable for sharding.
Stale: Cached or replicated data that no longer matches the source of truth.
Stateless server: A server that keeps no session data in its own memory, so any server can answer any request and adding servers is easy.
Status code: The number that starts an HTTP response and says how the request went: 2xx means success, 4xx the client's mistake, 5xx the server's problem. Common codes are 200 OK, 201 Created, 301 moved for good, 302 moved for now, 400 Bad Request, 401 not identified, 403 known but not allowed, 404 Not Found, 409 Conflict, 429 Too Many Requests, 500 Server Error and 503 Service Unavailable.
Sticky session: A load balancer sending one customer to the same server every time because it holds their cart in memory; if that server dies the cart is lost.
Strong consistency: Every read sees the latest write, as when reading from the primary.
synchronized: A Java keyword on a method that lets only one thread run it at a time, making a check-then-change atomic within one program.
Synchronous work: Work the reply must wait for, such as saving the order before replying to checkout.
System design: Deciding the parts of a system and how they talk to each other, before or while you build it; it often starts as a whiteboard sketch of boxes and arrows.

T

TCP: The connection protocol that delivers bytes in order and resends lost ones.
Timeout: The longest you will wait for a reply to a network call, so a hung server doesn't tie up every thread.
Token: A long secret string sent in a header such as `Authorization: Bearer <token>` to prove who the caller is.
Transaction: A group of steps that succeed or fail together: if all finish they are committed, otherwise they are rolled back, like reserving every product in a cart or none.
TTL (time to live): How many seconds a cached entry stays valid before it expires and the next lookup misses and fetches a fresh copy.

U

URL shortener: A service that turns long shop addresses into short links like go.kettleandcrane.com/abc12 and redirects visitors to the long URL.

V

Versioning: Putting a version such as /v1/ in API paths so a /v2/ can appear while old apps keep working.
Vertical scaling: Buying a bigger machine; simple, but there's a largest machine you can buy.

W

Warm-up: Untimed runs of code before measuring, because Java starts slowly and speeds up code it runs often.
WebSocket: A connection that starts as an HTTP request and then stays open, so either side can send a message at any time; the support chat uses one per person.
Weighted round robin: Round robin where bigger servers appear more often in the ring and so get more requests.
Write-through cache: A cache that is written together with the database at the same time, instead of being deleted on change.