Glossary

100 terms from Regression, in plain words, each with the chapter that teaches it.

Search the glossary

A

Accuracy: The share of cases a yes/no model gets right at a chosen threshold. It is a poor guide when one outcome is rare: flagging nobody would be right for 73.6% of subscribers.
Adjusted R²: A version of R² that subtracts a penalty for each predictor, so it only rises when a new predictor helps more than random noise would. Use it to compare models with different numbers of predictors.
anova_lm: A statsmodels function that runs an F-test comparing a smaller model with a bigger one that contains it. It reports whether the extra coefficients improve the fit more than chance would.
AUC (area under the ROC curve): A single score for how well a yes/no model ranks cases: the chance that a random churner gets a higher predicted probability than a random stayer. 0.5 is guessing and 1 is perfect.
autocorr: A pandas method giving the correlation between a series and a copy of itself shifted by some number of rows. `.autocorr(7)` checked whether each day's residual resembles the one a week earlier.

B

bse: The statsmodels attribute holding the standard error of each coefficient. The "b" is a traditional name for coefficients.

C

C(): A formula wrapper that marks a column as categorical, so each value gets its own dummy even if it is a number like month. With `Treatment(reference="Mon")` it also lets you pick the reference level.
Categorical variable: A variable that puts each row into a group, such as weekday, city or subscription plan. Regression turns each category into dummy variables.
Causal claim: A claim about what would happen if you changed something, such as "spending $100 more on ads will bring 5 more orders". Regression on everyday data shows association, and only an experiment reliably supports causal claims.
Churn: Customers leaving, such as Priya's coffee subscribers canceling. In the data, `churned` is 1 for someone who canceled.
Coefficient: A number in the fitted model: the intercept or a predictor's slope. For example, 0.0679 is how many more orders each extra ad dollar goes with.
Condition number: A statsmodels warning figure measuring how hard the predictors are to separate numerically. It can flag multicollinearity but can also be large just because predictors are on very different scales.
Confidence interval: A range built so that 95% of such ranges, across datasets like this one, contain the true value. Roughly the coefficient plus or minus 2 standard errors; it shows how big an effect plausibly is.
Confidence interval for the mean: A narrow range for the average outcome at given predictor values, such as the average $400 ad day. It is much narrower than a prediction interval for one single day.
Confounder: A variable related to both a predictor and the outcome that distorts their simple relationship. Email days also had higher ad spend, so email confounded the ad slope until it was added to the model.
Confusion table: A count of predicted yes/no flags against what actually happened. It shows how many churners Priya's model caught and missed at a chosen threshold.
Cook's distance: A measure of how much one row changes the fitted model when it is removed. Values far above the rest, like the March 14 outage day, deserve investigation.
Correlation (r): A number from −1 to 1 measuring how closely two variables follow a straight line. Ad spend and orders have r = 0.715, but r alone doesn't say how many orders $100 buys.
Cross-validation (k-fold): A way to compare models using only the training rows: split them into k folds, fit on k−1, measure error on the fold left out, repeat for each fold and average. It estimates prediction error without touching the test set.

D

Degrees of freedom: The number of rows minus the number of fitted coefficients. It's why the residual standard error divides by n − 2 rather than n.
Diminishing returns: Each extra unit of x adds less to y than the one before, like extra ad dollars buying fewer orders when Sam already spends a lot. A log transformation models it.
Dummy variable: A 0/1 column that is 1 when a row is in a group and 0 otherwise, such as `email` or `weekend`. A category with k levels needs k − 1 dummies.
Durbin-Watson: A statistic in the full summary checking whether neighboring residuals are correlated. Near 2 means no correlation; lower values mean positive correlation, which breaks the independence assumption.

E

Elasticity: When both y and x are logged, the coefficient gives the percentage change in y for a 1% change in x. For example, 10% more ad spend goes with about 5.3% more revenue.
Extrapolation: Predicting outside the range of the data the model was fitted on. Two models that agree inside the data can disagree wildly at, say, $3,000 of ads.

F

F-statistic: A summary-table test of whether all the predictors together are useless. Its p-value is almost always tiny, so it is usually skippable.
F-test: A test of whether extra coefficients together improve a model more than chance would. It requires the smaller model to be nested in the bigger one.
Fishing for significance: Trying many predictors, interactions or outlier rules until something gets p < 0.05. Even pure random columns will look significant by chance, so decide what to test beforehand and report everything.
Fitted value: The model's prediction for a row it was fitted on, stored in `model.fittedvalues`. Residuals are the actual values minus these.
Formula (formula API): Text such as `"orders ~ ad_spend"` describing a model, with the outcome left of `~` and predictors on the right. statsmodels' formula API, imported as `smf`, reads it using the patsy package.

G

get_influence: A statsmodels method whose summary_frame gives one row per observation with studentized residuals, leverage (`hat_diag`) and Cook's distance (`cooks_d`). It is how you find unusual rows.
get_prediction: Statsmodels methods for predicting on new rows. `.predict()` returns plain predictions; `.get_prediction(new).summary_frame(alpha=0.05)` adds confidence and prediction intervals.

H

HC3: A widely recommended kind of robust standard error, requested with `.fit(cov_type="HC3")`. Coefficients stay the same, but standard errors, p-values and intervals change.
Homoscedasticity: Residuals spreading about equally widely at every fitted value; unequal spread, shown as a funnel in the residual plot, is heteroscedasticity. Unequal spread makes prediction ranges wrong in places.

I

Influential point: A row whose removal noticeably changes the coefficients, usually because it combines high leverage with a large residual. Black Friday was one in the full data.
Interaction: When one predictor's effect depends on another's value, such as the newsletter adding more orders on weekdays than on weekends. In a formula `a * b` fits a, b and their interaction `a:b`.
Intercept: The line's predicted outcome when every predictor is 0, where it crosses the y-axis. It is often a guess beyond the data, so treat it as an anchor rather than a forecast.

L

Least squares: Choosing the line (coefficients) with the smallest possible sum of squared residuals. Squaring stops positive and negative misses from cancelling.
Leverage: How unusual a row's predictor values are, from 0 to 1. A high-leverage point is far from other rows in x but only moves the line if it also has a large residual.
LINE: The four regression assumptions: Linearity, Independence, Normal residuals and Equal spread. You check them by looking at residual plots.
Log transformation: Using the logarithm of a variable in the model. A logged predictor models diminishing returns, and a logged outcome makes coefficients percentage changes. `np.log` works inside a formula.
Log-odds (logit): The natural log of the odds. It runs from minus to plus infinity, with 0 meaning 50/50, which is why a straight line can model it safely.
Logistic regression: A model for the probability of a yes/no outcome, like whether one of Priya's subscribers cancels. It models log-odds as a straight line so probabilities stay between 0 and 1.

M

MAE (mean absolute error): The average size of a miss with signs ignored, in the outcome's units. It's the typical error and easy to explain, such as "4.7 orders off on average".
Maximum likelihood: A way of finding coefficients by searching for the ones that make the observed outcomes most probable. statsmodels uses it for logistic regression, which is why it prints iteration messages.
Model: An equation, simplified from the data, that you can use to predict and explain, like "orders = 20 + 0.068 × ad spend".
Multicollinearity: Predictors so closely related that the model can't separate their effects, like ad spend and clicks (r = 0.992). Coefficients become unstable with wide intervals, though predictions are barely hurt.
Multiple regression: Regression with several predictors, joined with `+` in the formula. Each coefficient is the effect of that predictor holding the others constant.

N

Natural logarithm: The power that e (about 2.718) must be raised to in order to give a number. Each doubling adds the same 0.693 to the log, so logs turn "times 2" into "plus 0.693".
Nested model: A smaller model that is contained in a bigger one, such as the weekend flag model inside the full weekday model. F-tests compare nested models.
No. Observations: The summary line showing how many rows the model actually used. If it is smaller than expected, statsmodels silently dropped rows with gaps in the formula's columns, so check it every time.
Normal distribution: The symmetric bell-shaped spread of values. Regression assumes residuals are roughly normal, which prediction intervals rely on.
Null hypothesis: The assumption that a coefficient's true value is 0, meaning the predictor has no relationship with the outcome. The p-value is computed assuming it is true.
NumPy: The Python numbers library that pandas is built on, usually imported as `np`. It supplies math functions such as `np.log`, `np.exp` and `np.sqrt`.

O

Odds: p / (1 − p): the chance something happens compared with the chance it doesn't. A probability of 0.75 is odds of 3, or three to one.
Odds ratio: The factor the odds are multiplied by for one more unit of a predictor, found with `np.exp` of the coefficient. An odds ratio of 1.93 for late deliveries means 1.93 times the odds of churning, not 1.93 times as likely.
OLS (ordinary least squares): The standard way to fit a linear regression by least squares. In statsmodels, `smf.ols(formula, data=df).fit()`.
Outcome: The number you want to predict or explain, such as daily orders. Also called the response or dependent variable.
Outlier: A row with a large residual: its outcome is far from the model's prediction, like the 4-order checkout-outage day. It isn't automatically influential.
Overfitting: Fitting the noise in the training data rather than the real pattern, so predictions on new data get worse. It shows as training error falling while cross-validated error rises.

P

p-value: How surprising an estimate would be if the true coefficient were 0: the probability of an estimate at least this far from 0 under the null. It is not the chance the null is true.
pd.crosstab: A pandas function that counts rows for each combination of two columns. It is used to check that there is data for every combination before fitting an interaction, and to build a confusion table of predicted against actual outcomes.
pd.cut: pandas functions that sort numbers into bins. `pd.cut` uses edges you give; `pd.qcut` picks edges so each bin has the same number of rows.
Poisson regression: A regression model for count outcomes, such as the number of orders per day, that models counts directly rather than treating them as ordinary numbers. In statsmodels it is fitted with `smf.poisson`.
Prediction grid: A small table of predictions, one per combination of predictor values, used to explain interactions and categories in real units instead of raw coefficients.
Prediction interval: A range for a single new value, such as one $400 ad day, which is much wider than the confidence interval for the average. It is the honest answer to "how many orders tomorrow?".
Prediction vs explanation: Prediction means making a good guess for an unknown number; explanation means understanding how one thing changes with another. Regression can do both.
Predictor: A value used to predict or explain the outcome, such as ad spend. Also called a feature or independent variable.

R

Randomized experiment: Changing something (like the ad budget) on randomly chosen days or units and comparing the results. It is the reliable route to a causal answer.
Reference level: The category every dummy is compared against, by default the first alphabetically (Friday for weekdays). The intercept then reflects the reference group.
Regression: Fitting an equation that describes how an outcome changes with one or more predictors. It is used to predict a number and to explain what drives it.
Residual: Actual value minus predicted value; on a chart, the vertical gap between a dot and the line. Plotting residuals shows what the model is missing.
Residual standard error: Roughly the typical size of a residual, in the outcome's units, computed as √(SSE/(n − 2)) for one predictor. For the ad spend model, a typical day is about 11 orders from the line.
Residuals vs fitted plot: A scatter of residuals against fitted values. A shapeless band around zero looks fine; a curve means non-linearity, a funnel means unequal spread, and far-off dots are outliers.
Reverse causation: The outcome causing the predictor rather than the other way round. If Sam raises ad spend on days he expects to be busy, high orders partly cause high spend.
RMSE (root mean squared error): Square the errors, average them, then take the square root. It is in the outcome's units and punishes big misses heavily.
Robust standard errors: Standard errors that don't assume residuals have equal spread. Use them when residuals fan into a funnel; requested with `cov_type="HC3"`.
R² (R-squared): The share of the outcome's variation the model accounts for, 1 − SSE/SST, from 0 to 1. A high R² doesn't mean the model is correct or causal.

S

Scatter plot: A chart with one dot per row, the predictor on the x-axis and the outcome on the y-axis. Every regression should start with one.
Scientific notation: A way of writing very large or small numbers: `3.546124e-58` means 3.5 × 10⁻⁵⁸. It appears for tiny p-values, which the summary rounds to 0.000.
scikit-learn: Python's main machine learning library, imported as `sklearn`. It provides tools such as `train_test_split`, `KFold`, error metrics like RMSE and MAE, and AUC for evaluating models.
Seed: A number that makes random results, like a shuffled train/test split, come out the same every run.
Simple linear regression: Regression with one predictor, fitting a straight line.
Slope: How much the outcome changes for one more unit of a predictor, on average. About 0.068 more orders per extra ad dollar, or 6.8 per $100.
Squared term: Adding the square of a predictor to fit a curve. Unlike a log, a squared curve eventually turns around, so use it when there really is a peak.
SSE (sum of squared errors): The sum of each residual squared; least squares makes it as small as possible. The best ad spend line had an SSE of 44,218.
SST (total sum of squares): The SSE of a flat line at the mean of the outcome: how much the outcome varies when you ignore the predictors. R² compares SSE with it.
Standard error: The typical distance between an estimate and the true value. It shrinks with more rows and less noise.
Statistically significant: A p-value below a threshold chosen in advance, usually 0.05. It doesn't mean the effect is large or important.
statsmodels: The Python library that fits statistical models such as regression and prints detailed reports.
Studentized residual: A residual divided by its standard error, read like a z-score. Values beyond about ±3 are unusual; March 14's was −7.6.
summary(): The statsmodels method that prints a full regression report; `slim=True` gives a shorter version, and `.tables[1]` prints just the coefficient table.

T

t value: A coefficient divided by its standard error: how many standard errors the estimate is from zero. Beyond about ±2 gives p < 0.05.
Test set: Rows held back from fitting to measure prediction on unseen data. It should be used only once, at the end.
Threshold: The predicted probability above which a row is flagged "yes", such as calling customers with churn probability of 0.3 or more. Choose it from the costs of each kind of mistake.
train_test_split: A scikit-learn function that shuffles rows and splits them into a training set and a test set, e.g. `test_size=0.25` holds back a quarter.
Training set: Rows used to fit the model; the remaining rows form the test set.
Treatment coding: The default way statsmodels builds dummies, comparing each category with the reference level. Coefficient names like `weekday[T.Mon]` show it.

V

VIF (variance inflation factor): How many times larger a coefficient's variance is because its predictor overlaps with the others. 1 means no overlap, above 5 deserves attention and above 10 is serious.

Z

z-score: How many standard errors a value is from a reference value, usually zero. Logistic regression tables report z in place of t.