Linear Regression Explained: Equation, Worked Example and R²
Reviewed by Prof. Dr. Alexander Holtermann · Updated 16 October 2026 · 6 min read
Linear regression predicts a metric outcome from one or more predictors with a straight line: ŷ = a + b · x. R² shows how much variance it explains.
Key takeaways
- Simple linear regression fits the line
ŷ = a + b · xthat minimizes the sum of squared residuals (ordinary least squares). - The slope b is the predicted change in y for a one-unit increase in x; the intercept a is the predicted y when x = 0.
- R² is the proportion of variance in the outcome explained by the model; in simple regression it equals r².
- Multiple regression adds predictors and estimates each effect while holding the others constant; standardized β coefficients make them comparable.
- Check the assumptions with the residuals: linearity, independence, homoscedasticity, normally distributed residuals and, for multiple regression, no strong multicollinearity.
What does linear regression do?
Correlation tells you how strongly two variables are related. Regression goes a step further: it describes the relationship with an equation that predicts one variable (the outcome, dependent variable, criterion) from another (the predictor, independent variable).
In simple linear regression there is one predictor:
ŷ = a + b · x
ŷ (“y-hat”) is the predicted value, a the intercept and b the slope or regression coefficient. The difference between an observed and a predicted value is the residual: e = y − ŷ. Ordinary least squares (OLS) chooses a and b so that the sum of squared residuals is as small as possible.
b = Σ(x − Mx)(y − My) / Σ(x − Mx)²a = My − b · MxR² = 1 − SS_residual / SS_total
Worked example: predicting exam scores from study hours
We use the same ten students as in the correlation article. Study hours x: 2, 3, 5, 6, 7, 8, 10, 11, 12, 14; exam scores y: 55, 62, 58, 66, 61, 70, 68, 74, 65, 80. Mx = 7.80, My = 65.90, Σ(x − Mx)(y − My) = 222.80, Σ(x − Mx)² = 139.60.
| Quantity | Calculation | Result |
|---|---|---|
| Slope b | 222.80 / 139.60 | 1.60 |
| Intercept a | 65.90 − 1.596 · 7.80 | 53.45 |
| Regression equation | ŷ = 53.45 + 1.60 · x | |
| Standard error of b | from residual variance | 0.37 |
| t-test for b | 1.596 / 0.368, df = 8 | t = 4.34, p = .002 |
| 95% CI for b | 1.596 ± 2.306 · 0.368 | [0.75, 2.44] |
| R² (adjusted R²) | 1 − 151.31 / 506.90 | .70 (.66) |
| F-test of the model | (355.59 / 1) / (151.31 / 8) | F(1, 8) = 18.80, p = .002 |
| Standard error of the estimate | √(151.31 / 8) | 4.35 points |
Reading the regression output
Statistics software and R or Python libraries print a similar set of tables. These are the parts you need for your thesis:
- Model summary: R, R², adjusted R² (penalizes extra predictors), standard error of the estimate.
- ANOVA table: F-test of whether the model as a whole explains more than zero variance. In simple regression, F = t² of the slope (4.34² ≈ 18.80).
- Coefficients table: unstandardized b with SE, t, p and CI for every predictor, plus standardized β. In simple regression β = r (here .84).
- Residual diagnostics: plots of residuals against predicted values and a Q-Q plot of residuals.
Multiple linear regression
With several predictors the model becomes ŷ = a + b₁x₁ + b₂x₂ + … + b_kx_k. Each b now means: the predicted change in y for a one-unit change in that predictor, holding the other predictors constant. This is how regression “controls” for confounders, for example estimating the effect of study hours while holding prior grades constant.
Unstandardized b coefficients depend on the units of each predictor. Standardized coefficients (β) express all variables in SD units and let you compare the relative importance of predictors within one model. Categorical predictors enter as dummy variables (0/1), with one category as the reference.
- Rule of thumb for sample size: at least 10 to 15 cases per predictor, better a power analysis.
- Report adjusted R² when comparing models with different numbers of predictors.
- Hierarchical regression adds predictor blocks step by step and tests the change in R² (ΔR²).
Assumptions and diagnostics
Most assumptions refer to the residuals, not to the raw variables. Check them after fitting the model.
| Assumption | Check | If violated |
|---|---|---|
| Linearity | Scatter plot; residuals vs. predicted values show no curve | Add a quadratic term or transform variables |
| Independence of residuals | Design; Durbin-Watson test for time series | Multilevel or time-series models |
| Homoscedasticity (equal residual variance) | Residual plot without a funnel shape | Robust (heteroscedasticity-consistent) standard errors |
| Normality of residuals | Q-Q plot of residuals | Usually harmless in larger samples; bootstrap CIs |
| No strong multicollinearity | VIF below about 5 to 10 | Drop or combine highly correlated predictors |
| No overly influential cases | Cook’s distance | Check data; report results with and without the case |
Common mistakes
Regression is powerful, and therefore easy to over-interpret.
- Causal language without an experiment: “study hours increase scores” is only justified with random assignment. Say “are associated with”.
- Extrapolation: predicting a score for 20 study hours (85.4 points) goes beyond the observed range of 2 to 14 hours; the line may not hold there.
- Using linear regression for a binary outcome such as pass/fail; use logistic regression instead.
- Comparing unstandardized b across predictors with different units.
- Adding many predictors to a small sample, which inflates R² and produces unstable coefficients.
How to report a regression in APA style
Report the model fit (R², F with both df, p) and the coefficients (b, SE or CI, β, t, p). For multiple regression, a table is the norm.
Doing this in SPSS, R, Stata or JASP
The coefficients, standard errors, R² and F test are identical across programs. Request confidence intervals and diagnostics (residual plots, collinearity) in each.
| Program | How to run it |
|---|---|
| SPSS | Analyze → Regression → Linear; Statistics: Estimates, Confidence intervals, Model fit, Collinearity diagnostics; Plots: residuals |
| R | fit <- lm(y ~ x1 + x2, data = df), then summary(fit), confint(fit) and plot(fit) |
| Stata | regress y x1 x2 |
| JASP / jamovi | Regression → Linear Regression; add covariates, tick confidence intervals and residual plots |
Frequently asked questions
What is the difference between correlation and regression?
Correlation measures the strength of a relationship symmetrically. Regression models one variable as an outcome of another, gives a prediction equation and can include several predictors at once.
What is a good R²?
It depends on the field. In social science research with human behavior, R² values of .10 to .30 are common and can be meaningful; in physics, R² below .90 might be disappointing.
How do I interpret the regression coefficient b?
b is the predicted change in the outcome for a one-unit increase in the predictor, holding all other predictors constant. Its unit is outcome units per predictor unit.
What is the difference between b and β?
b is unstandardized and in the original units. β is standardized (all variables in SD units) and allows comparison of predictors within one model.
Do my variables need to be normally distributed for regression?
No. The assumption concerns the residuals, and even that matters mainly for small samples. Predictors can have any distribution, including binary dummy variables.
Sources and further reading
- Agresti, A., & Finlay, B. (2018). Statistical Methods for the Social Sciences (5th ed.). Pearson.
- Howell, D. C. (2013). Statistical Methods for Psychology (8th ed.). Wadsworth Cengage Learning.
- Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Lawrence Erlbaum.
Your university’s examination regulations and style guide always take precedence.
Related articles
- CorrelationA correlation coefficient measures the strength and direction of a relationship between two variables, from −1 to +1. Correlation is not causation.
- Logistic regressionLogistic regression predicts a binary outcome (yes/no) from one or more predictors. Coefficients are log-odds; exp(b) gives the odds ratio.
- Choosing a statistical testYour research question, the level of measurement of your outcome, the number of groups and whether samples are paired decide which statistical test fits.
- Effect sizeAn effect size tells you how large a difference or relationship is, independent of sample size. Cohen’s d of 0.2, 0.5 and 0.8 is small, medium, large.
- Reporting statistics in APAAPA style reports test statistic, degrees of freedom, exact p-value and effect size, e.g. t(13.46) = 3.42, p = .004, d = 1.71. Examples for every test.
- Statistics software comparedSPSS and Stata are menu- or command-based commercial packages, R and Python are free code, JASP and jamovi are free point-and-click. Choose by field and task.
