Type I and Type II Errors: Alpha, Beta and Statistical Power
Reviewed by Prof. Dr. Alexander Holtermann · Updated 16 October 2026 · 6 min read
A Type I error is a false positive (rejecting a true null hypothesis, risk α); a Type II error is a false negative (missing a real effect, risk β).
Key takeaways
- Type I error (α, false positive): you conclude there is an effect, but there is none. Its probability is the significance level, usually 5%.
- Type II error (β, false negative): you miss an effect that really exists. Its probability depends on effect size, sample size and α.
- Power = 1 − β is the probability of detecting a real effect. The convention is at least 80%.
- For a medium effect (d = 0.5) in a two-group t-test you need 64 people per group for 80% power; with 20 per group the power is only about 34%.
- Running many tests inflates the Type I error rate: with 10 independent tests at α = .05 the chance of at least one false positive is about 40%.
Two ways to be wrong
A significance test ends with a decision: reject the null hypothesis or not. Because the decision is based on a sample, it can be wrong in two ways. The table is the classic overview; it is worth memorizing.
| Decision | H₀ is actually true (no effect) | H₀ is actually false (effect exists) |
|---|---|---|
| Reject H₀ (“significant”) | Type I error, false positive, probability α | Correct decision, probability 1 − β (power) |
| Do not reject H₀ | Correct decision, probability 1 − α | Type II error, false negative, probability β |
Type I error and the significance level α
You choose α before the study. With α = .05 you accept that, if there is truly no effect, 5% of studies like yours would still produce a significant result just by chance. Lowering α to .01 reduces false positives but, with the same sample, increases false negatives.
Examples of Type I errors in student research: concluding that a study technique improves grades when the difference arose by chance; concluding that two survey variables are correlated when the correlation is a sampling fluke. Replications are the best protection.
Type II error, β and statistical power
β is the probability of missing a real effect. Its complement, power = 1 − β, is the probability of detecting it. Unlike α, you do not set β directly; it follows from four quantities:
- Effect size: large effects are easier to detect than small ones.
- Sample size: more data give more power.
- α: a stricter α (e.g., .01) reduces power.
- Variability: less noise (reliable measures, homogeneous samples, paired designs) gives more power.
How many participants do you need?
A power analysis before data collection answers this question. You specify the test, the expected effect size, α and the desired power (usually .80), and get the required sample size. The power analysis tool uses a proven normal approximation; its results match the standard tables.
| Design (α = .05, two-sided, power = .80) | Small effect | Medium effect | Large effect |
|---|---|---|---|
| Independent t-test, n per group | 394 (d = 0.2) | 64 (d = 0.5) | 26 (d = 0.8) |
| Paired t-test, n total | 199 (d_z = 0.2) | 34 (d_z = 0.5) | 15 (d_z = 0.8) |
| Correlation, n total | 783 (r = .1) | 85 (r = .3) | 30 (r = .5) |
| Chi-square 2 × 2, n total | 785 (w = .1) | 88 (w = .3) | 32 (w = .5) |
Multiple testing inflates Type I errors
Each test at α = .05 carries a 5% false-positive risk. Run several tests and the risks add up. With k independent tests, the familywise error rate is 1 − (1 − α)^k.
- Bonferroni correction: test each hypothesis at α / k. With 3 tests: .05 / 3 = .0167. Simple, but conservative.
- Holm procedure: a stepwise version of Bonferroni that is uniformly more powerful; a good default.
- Planned comparisons and post-hoc tests after ANOVA (for example Tukey HSD) control the familywise error by design.
- Best prevention: define a small number of main hypotheses in advance and label everything else as exploratory.
| Number of tests | Chance of at least one false positive |
|---|---|
| 1 | 5% |
| 3 | 14% |
| 5 | 23% |
| 10 | 40% |
| 20 | 64% |
Type III and other practical errors
Some textbooks add a Type III error: correctly rejecting H₀ but drawing the wrong conclusion about the direction, or answering the wrong question. More common in practice are errors that inflate α without anyone noticing, sometimes called “p-hacking”:
- Collecting data until p drops below .05 and then stopping.
- Trying several outcome variables, outlier rules or covariates and reporting only the significant version.
- Switching from a two-sided to a one-sided test after seeing the data.
- Pre-registering your hypotheses and analysis plan, even informally with your supervisor, prevents all three.
What to write in your thesis
Mention α in your methods section, justify your sample size with a power analysis, and discuss Type II errors when results are not significant. A non-significant result in an underpowered study is inconclusive, not evidence of no effect.
Frequently asked questions
What is the difference between a Type I and a Type II error?
A Type I error is a false positive: you reject a null hypothesis that is true. A Type II error is a false negative: you fail to reject a null hypothesis that is false, so you miss a real effect.
Which error is worse?
It depends on the consequences. In medicine, approving an ineffective drug (Type I) and missing an effective one (Type II) both have costs. Science conventionally guards more strictly against Type I errors (α = .05) than against Type II errors (β = .20).
How can I reduce both errors at the same time?
Increase the sample size or reduce measurement noise. For a fixed n, lowering α raises β and vice versa.
What is statistical power?
The probability that a test detects an effect of a given size if it exists: power = 1 − β. Studies usually aim for at least 80%.
Why was my result not significant even though the effect looked large?
Probably your sample was too small, so power was low. Report the effect size and confidence interval and discuss the possibility of a Type II error.
Sources and further reading
- Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Lawrence Erlbaum.
- Howell, D. C. (2013). Statistical Methods for Psychology (8th ed.). Wadsworth Cengage Learning.
- Gravetter, F. J., & Wallnau, L. B. (2017). Statistics for the Behavioral Sciences (10th ed.). Cengage.
Your university’s examination regulations and style guide always take precedence.
Related articles
- p-value and significanceThe p-value is the probability of a result at least as extreme as yours if the null hypothesis were true. p < .05 is usually called significant.
- Sample sizeFor a survey with 95% confidence and a ±5% margin of error you need about 385 responses. For group comparisons, a power analysis sets the size.
- Effect sizeAn effect size tells you how large a difference or relationship is, independent of sample size. Cohen’s d of 0.2, 0.5 and 0.8 is small, medium, large.
- Hypothesis testing in 7 stepsHypothesis testing in 7 steps: state H₀ and H₁, set α, choose the test, check assumptions, compute the statistic, decide with p, report the effect size.
- Writing a hypothesisA hypothesis is a testable prediction about the relationship between variables. Write a null (H₀) and an alternative hypothesis (H₁) for each test.
