Glossary

108 terms from Linear Algebra for ML, in plain words, each with the chapter that teaches it.

Search the glossary

#

@ operator (matrix multiplication): NumPy's sign for matrix multiplication. Between two vectors it gives the dot product, like `qty @ price` for an order total.

A

Activate: Tell your current terminal to use a virtual environment, so `python3` and `pip` work inside the project's private folder. You do it again in each new terminal.
Activation function: A simple bend applied after a neuron's dot product, such as ReLU or sigmoid. Without it, stacked layers would collapse into a single matrix.
allclose: A NumPy function that says whether two arrays are equal to within a tiny tolerance. Use it instead of `==` for decimals.
argmax / argmin: Array methods that give the position of the largest (or smallest) value, not the value itself. It's how you find which product sold most.
argsort: An array method that gives the positions that would sort the array from smallest to largest. Reversed with `[::-1]`, it ranks items from highest to lowest.
Array: NumPy's container of numbers, all of one type, that does math on every element at once. A vector is a 1D array and a matrix is a 2D array.
astype: An array method that converts every number to another type. Converting decimals to `int` chops off the decimal part.
Axis: One direction of an array: axis 0 runs down the rows and axis 1 across the columns. The axis you name in `sum(axis=0)` is the one that gets collapsed.

B

Batch: Several examples processed together, stacked as the rows of one matrix. Five cafés fed through a layer at once are a batch.
Bias: The number a neuron adds after its dot product. It shifts the result up or down whatever the inputs are, like the intercept of a line.
Boolean mask: An array of `True`/`False` values used inside brackets to pick elements, such as `sales[sales > 8]`. Combine conditions with `&`, `|` and `~`.
Broadcasting: NumPy stretching a smaller array to match a bigger one's shape. Line shapes up from the right; each pair of sizes must be equal or be 1.

C

Column vector: A matrix with one column, shape (n, 1); a row vector has shape (1, n). A plain 1D array of shape (n,) holds the same numbers but broadcasts differently.
Component: One number in a vector, such as Maya's 6 espresso bags. Math's v₁ is NumPy's `v[0]`.
Condition number: A measure of how much a matrix magnifies small errors. Near 1 is safe; above about 1e10 you shouldn't trust the answer.
Correlation: How closely two standardized features rise and fall together, from −1 to 1. Coffee and seats at the cafés have a correlation of 0.984.
Cosine: A function of an angle: 1 at 0° (same direction), 0 at 90° (right angle) and −1 at 180° (opposite).
Cosine similarity: The dot product divided by both vectors' lengths, comparing direction only. It's 1 for the same direction, 0 for unrelated and −1 for opposite, so it compares the mix rather than the volume.
Covariance matrix: A square matrix saying how each pair of features varies together, built as `Z.T @ Z / n`. PCA finds its eigenvectors.
CSV file (comma-separated values): A plain text file where each line is a row and values are separated by commas. The project's `purchases.csv` holds each customer's purchase counts.
Cumulative sum: A running total where each element is the sum of everything up to that point. It shows how much of a matrix's energy you keep as layers are added.

D

Design matrix: The matrix of inputs X in a fitting problem, one row per observation. A column of ones gives the line an intercept.
Determinant: A single number saying how much a square matrix scales area. A determinant of 0 means the matrix is singular and has no inverse.
Diagonal: The entries from top left to bottom right of a square matrix. `np.diag(s)` builds a matrix with `s` along it and zeros elsewhere.
Dimension: The number of components of a vector, so Maya's vector is 4-dimensional. It's also the number of axes of an array (`.ndim`), which is 1 for a vector and 2 for a table.
Dot product: Multiply matching components of two vectors and add the results, giving a single number. It's large and positive when vectors point the same way, 0 when orthogonal and negative when opposite.
dtype: The type shared by every number in an array, such as `int64` or `float64`. Mixing whole numbers and decimals converts everything to floats.

E

Eigenvalue: The factor by which a matrix stretches (or shrinks) one of its eigenvectors, written λ in A v = λ v. For example, with A = [[2, 1], [1, 2]], the vector (1, 1) has eigenvalue 3.
Eigenvector: A direction a matrix only stretches and never turns: Av = λv. Any multiple of an eigenvector is also one.
Element-wise: Done to each pair of matching elements separately. `*`, `/` and `**` on arrays work this way, unlike `@`.
Embedding: A vector an AI model uses to represent text, an image or a product by its meaning. Similar things get vectors pointing in similar directions, compared with cosine similarity.
Energy: A layer's squared singular value, measuring how much of the matrix that layer carries. The running total tells you how much you keep when you cut off layers.
Explained variance ratio: The share of the data's spread captured by one principal component, found by dividing its eigenvalue by the total. The first café component explains 86.7%.

F

Fancy indexing: Picking elements with a list of positions, like `maya[[0, 3]]`. It always makes a copy, not a view.
Feature: One number describing a thing, such as a café's seats or weekly coffee. In an ML dataset each column is a feature.
Float: A number with a decimal point; NumPy's usual decimal type is `float64`. Arrays that mix whole numbers and decimals become all floats.
Floating-point rounding: Tiny errors from storing decimals in binary, such as `-5.55e-17` where you expected 0. Compare decimals with `np.allclose`, never `==`.

I

Identity matrix: The matrix that changes nothing: 1s on the diagonal and 0s elsewhere. IA = A, like multiplying a number by 1.
Ill-conditioned: Describes a matrix with a huge condition number, so tiny errors in the input are magnified enormously and the solution of a linear system is unreliable. A nearly singular matrix with a condition number around 1.9e+16 is an example.
Intercept: The value of a line when its input is 0: c in y = mx + c. For Priya's ads it's the bags sold with no ads.
Inverse: The matrix that undoes another: A⁻¹A = I. Only square, non-singular matrices have one, and in code `solve` is usually better.

K

keepdims: An option on summaries that keeps the collapsed axis with size 1, so row totals stay a column that broadcasts correctly.

L

Layer: A group of neurons that all read the same inputs: `relu(X @ W + b)`. It transforms every data point and then bends the result.
Least squares: Choosing coefficients that make the sum of squared residuals as small as possible. `np.linalg.lstsq` does it and copes with repeated columns.
Library: A large collection of ready-made modules you install and import. NumPy is one.
Linear algebra: The math of lists and tables of numbers: how to add, compare and multiply them and solve equations with them. Machine learning runs on it.
Linear combination: Vectors each multiplied by a number and then added. A matrix times a vector is a linear combination of the matrix's columns.
Linear regression: Predicting a number as a weighted sum of inputs, fitted by least squares. Priya's ad spend plus newsletter model is one.
Linear system: A set of equations where unknowns are only multiplied by numbers and added, writable as Ax = b. Jo's blend problem is one.
Linear transformation: A move of space that keeps lines straight, parallel lines parallel and evenly spaced, and the origin fixed. Every one is a matrix.
Linearly independent: A set of vectors where none is a linear combination of the others. If row 3 = row 1 + row 2, the rows are dependent and tell you nothing new.
loadtxt: A NumPy function that reads a text file of values straight into an array. Options like `delimiter`, `skiprows` and `usecols` pick what to read.
Low-rank approximation: Keeping only the top k layers of the SVD to get the best rank-k stand-in for a matrix. It saves storage and removes noise while keeping the main structure.

M

Machine learning: Getting a computer to learn patterns from data instead of following rules a person wrote. ML models store data and knowledge as lists and tables of numbers.
Matrix: A rectangular table of numbers in rows and columns, like the cafés' orders. In ML, each row is usually a sample and each column a feature.
Matrix multiplication: Each entry of the result is a row of the first matrix dotted with a column of the second. Shapes follow (m × n) @ (n × p) → (m × p), and the order matters.
matrix_rank: A NumPy function that computes a matrix's rank, handling rounding sensibly.

N

Neural network: A model made of layers of neurons, which does matrix multiplies with weights and biases and is trained to give useful outputs.
Neuron: A unit that computes a dot product of weights and inputs plus a bias, then applies an activation. Without the activation, a single neuron is a linear model.
Norm: The length of a vector: square each component, add them up and take the square root. The distance between two vectors is the norm of their difference.
Normal equations: XᵀX c = Xᵀy, the square system that least squares solves. It fails when columns repeat, so `lstsq` is safer.
Normalize: Divide a vector by its length to get a unit vector, keeping only its direction.
np.linalg.eigh: The NumPy function that returns a symmetric matrix's eigenvalues (smallest first) and unit eigenvectors as columns. The h is for Hermitian, a mathematician's term that includes symmetric; `eig` handles other matrices but may return complex numbers.
np.linalg.solve: The NumPy function that solves Ax = b for x. It's faster and more accurate than multiplying by the inverse.
NumPy: Python's library for arrays and linear algebra, imported as `np`. Pandas, scikit-learn and PyTorch are built on it or work the same way.

O

Origin: The point (0, 0) where the axes cross, where a vector arrow starts.
Orthogonal: At right angles. Two orthogonal vectors have a dot product of exactly 0.
Outer product: A column vector times a row vector, giving a whole rank-1 matrix. It's the opposite of the dot product, which squashes two vectors into one number.

P

Parameters: The weights and biases of a neural network, which training adjusts to reduce its error. A network with 3 inputs, 4 hidden neurons and 1 output has 3 × 4 + 4 + 4 × 1 + 1 = 21 parameters.
PCA (principal component analysis): Finding new axes for data, ordered by how much the data spreads along each, so keeping the first two gives a 2D map that loses little. It standardizes, builds the covariance matrix and takes eigenvectors.
pip: Python's package installer. It downloads packages like NumPy from PyPI, the Python Package Index at pypi.org.
Projection: Placing data on new axes by dotting each row with the chosen eigenvectors, like `Z @ vectors[:, :2]` in PCA. It gives each sample fewer coordinates.
PyTorch: A Python library for building and training neural networks. Its `nn.Linear` layer stores its weight matrix as (outputs × inputs) and computes `X @ W.T + b`, the transpose of the (inputs × outputs) layout used with `X @ W`.

R

Random number generator: Something that makes random numbers; `np.random.default_rng(42)` makes one that repeats the same numbers every run. Networks start with random weights from one.
Rank: The number of independent rows (or columns) of a matrix; full rank means as many as possible. Lower rank means singular: no answer or infinitely many.
Recommender system: A program that suggests items by comparing vectors, for example describing each product by who bought it and ranking other products by cosine similarity to the one being viewed. It powers "customers who bought this also liked" suggestions.
ReLU (rectified linear unit): An activation that keeps positive numbers and turns negative ones into 0: `np.maximum(0, z)`.
Reshape: An array method that rearranges the same numbers into a new shape; `-1` means "work this size out". `.reshape(-1, 1)` makes a column vector.
Residual: Actual value minus predicted value. Least squares picks the line with the smallest sum of squared residuals.

S

Sample: One row of a dataset: a customer, a café or a photo.
Scalar: A single number, as opposed to a vector or matrix. Multiplying a vector by one scales every component.
Seed: A starting number given to a random number generator, such as `np.random.default_rng(42)`, so it produces the same sequence of "random" numbers on every run. It makes results reproducible.
Shape: An array's size along each axis as a tuple, such as `(4, 3)` for 4 rows and 3 columns. A vector's shape looks like `(4,)`.
Shear: A move that slides the top of a shape sideways while the bottom stays put, like pushing the top of a deck of cards.
Sigmoid: A function, 1 / (1 + e⁻ᶻ), that squashes any number into the range 0 to 1. It's used for a network's output when it should be a chance.
Singular: A square matrix with no inverse (determinant 0, rank too low). It squashes space flat so different inputs land in the same place.
Singular value: The weight of one layer in the SVD, largest first. The number of non-zero singular values is the matrix's rank.
Slicing: Picking a range of elements with `a[start:stop]`, or `A[rows, cols]` for a matrix. In NumPy a slice is a view, not a copy.
Slope: How much a line rises when its input goes up by 1: m in y = mx + c. For Priya, extra bags per extra ad dollar.
Square matrix: A matrix with the same number of rows and columns. Only square matrices can have an inverse or determinant, and `solve` needs one.
Standard deviation: A measure of how spread out numbers are around their mean. PCA divides each column by it so units don't dominate.
Standardize: Subtract each column's mean and divide by its standard deviation, so every column has mean 0 and the same spread. Without it PCA just follows the biggest numbers.
SVD (singular value decomposition): Any matrix written as a sum of rank-1 layers, A = U Σ Vᵀ, sorted by singular value. Keeping only the top layers compresses the matrix.
Symmetric matrix: A square matrix equal to its own transpose. Covariance matrices are symmetric, so `eigh` works on them.

T

Terminal commands (mkdir, cd): `mkdir` makes a new directory (folder) and `cd` changes the directory the terminal works in. Together they set up the project folder.
Third-party: Written by others and installed separately from Python itself, as NumPy is.
Tip-to-tail addition: Adding vectors as arrows: start the second arrow at the tip of the first, and the sum goes from the origin to where you end up. It matches adding components.
Training: Adjusting a model's weights to reduce its error on known examples. It starts from random weights and nudges them thousands of times.
Transpose: A matrix with its rows and columns swapped. A 4 × 3 matrix becomes 3 × 4, and `A.T` often fixes shape mismatches.

U

Unit vector: A vector of length 1, made by dividing a vector by its own length. It keeps only the direction.

V

Vector: An ordered list of numbers, like Maya's purchases (6, 1, 0, 4). In 2D it's also an arrow from the origin with a direction and a length.
Vectorized: Code that works on whole arrays at once instead of looping. It's shorter and much faster because NumPy runs the loop in compiled code.
venv: Python's built-in module that creates a virtual environment in a folder, here called `.venv`.
View: A slice that shares its numbers with the original array, so changing it changes the original. Use `.copy()` for an independent array.
Virtual environment: A private folder of installed packages for one project, so two projects can't break each other with different versions.

W

Weight: A number a neuron multiplies one input by. Positive weights push the output up and negative ones push it down.
Well-conditioned: Describes a matrix with a small condition number, so tiny errors in the input are magnified only slightly and the solution of a linear system is reliable. A matrix with a condition number of about 5.7 is an example.