To apply a Bonferroni correction, divide your original significance level by the number of tests you ran, then judge every p-value against that stricter threshold. With alpha at .05 and five comparisons, each test has to clear .01 before you call it significant. It takes about five minutes with a calculator, and once you know the formula it never changes.
The correction exists because every extra test buys you another chance at a false positive. Run 20 independent tests at .05 and you have roughly a 64% chance of at least one bogus result. That is the multiple comparisons problem, and it is why a reviewer who asks for “Bonferroni corrected” p-values is not being fussy.
Bonferroni correction: a method for controlling the family-wise error rate by dividing the significance level (alpha) by the number of tests or comparisons (m).
Family-wise error rate (FWER): the probability of making one or more Type I errors across an entire family of simultaneous statistical tests.
Adjusted alpha: the stricter per-test threshold equal to alpha divided by m.
Bonferroni-adjusted p-value: the raw p-value multiplied by m, capped at 1.00; an alternative way to express the same correction.
Table of Contents
- 1What You Need
- 2Step-by-Step: How to Apply a Bonferroni Correction in 6 Steps
- 31. Decide Which Tests Need Correcting
- 42. Set the Familywise Significance Level
- 53. Calculate the Adjusted Threshold
- 64. Compare Each P-Value With the Threshold
- 75. Run the Correction in SPSS, R, Stata, or SAS
- 86. Report the Corrected Results
- 9Common Mistakes
- 10Frequently Asked Questions
- 11What is a Bonferroni procedure?
- 12When not to use Bonferroni correction?
- 13How do you adjust for multiple comparisons in statistical analysis?
- 14How is the Bonferroni correction used in an ANOVA test?
- 15What is the difference between an adjusted alpha and an adjusted p-value?
- 16Is Bonferroni correction always required by APA?
- 17Conclusion
What You Need
Five things have to be on your desk before you touch the arithmetic. Most mistakes happen because one of them was missing.
- Your original significance level, usually .05 or .01.
- The number of planned comparisons (m) in the family you are correcting.
- The raw p-values from each test, before any correction.
- The test or output that produced them — an ANOVA post hoc table, a set of t-tests, a correlation matrix.
- A reporting standard, so the correction ends up described rather than buried.
Here is the running example used throughout. You run one-way ANOVA on three groups (control, treatment, placebo) and the omnibus test is significant, so you run post hoc pairwise t-tests: control versus treatment, control versus placebo, and treatment versus placebo. That gives m = 3, alpha = .05, adjusted alpha = .0167, and three p-values sitting in your SPSS output.
Everything below works the same for five comparisons or fifty. The formula does not care how many rows you have.
Step-by-Step: How to Apply a Bonferroni Correction in 6 Steps
This is the workflow for learning how to apply a Bonferroni correction, from deciding the family to writing the sentence that goes in your paper. Work through it in order, and do not skip straight to the division.
1. Decide Which Tests Need Correcting
Start by naming the family: the specific set of tests that share a single alpha budget. Planned comparisons count. Unplanned or exploratory ones usually do not, and lumping them in inflates m until nothing survives.
This is the hardest judgment call in the whole procedure, and it has to be made before you look at which results came out small. Define the family in writing, such as “all pairwise comparisons among the three treatment arms,” and stick to it. A family is usually a set of tests answering one question from the same data, run because the design called for them.
2. Set the Familywise Significance Level
Your family-wise alpha is the total probability of one or more Type I errors across the family. .05 is the standard. The Bonferroni correction spreads that budget across m tests so each one gets alpha/m.
adjusted alpha = original alpha / number of comparisons (m)Two equivalent routes give the same answer. Adjust alpha and keep raw p-values, or keep alpha at .05 and multiply each p-value by m. Pick whichever matches your output and your reviewer.
3. Calculate the Adjusted Threshold
Do the division in front of you. Three groups means three pairwise comparisons, so m = 3 and .05 / 3 = .0167.
A general shortcut saves time whenever you compare groups: k groups produce k(k-1)/2 pairwise tests. Four groups give 6 comparisons and an adjusted alpha of .0083. Ten groups give 45 comparisons and .0011.
Here is a quick reference for the values you will meet most often, all at alpha = .05.
| Number of comparisons (m) | Adjusted alpha |
|---|---|
| 3 | 0.0167 |
| 5 | 0.0100 |
| 10 | 0.0050 |
| 20 | 0.0025 |
| 25 | 0.0020 |
| 100 | 0.0005 |
4. Compare Each P-Value With the Threshold
Go comparison by comparison. A result is significant when its p-value is at or below .0167. One at .012 survives; one at .021 does not, even though it would have cleared .05 on its own.
Here is the same three-group ANOVA worked through with raw p-values, adjusted p-values (p × 3, capped at 1.00), and the decision under each method.
| Comparison | Raw p-value | Adjusted p-value | Significant at .0167? |
|---|---|---|---|
| Control vs Treatment | 0.004 | 0.012 | Yes |
| Control vs Placebo | 0.021 | 0.063 | No |
| Treatment vs Placebo | 0.048 | 0.144 | No |
Two things to keep straight. The correction never changes your raw p-values; it only changes the bar. And only valid comparisons belong in the count — drop a test you ran then abandoned, and m drops with it.
5. Run the Correction in SPSS, R, Stata, or SAS
Every major package does this for you. The output is identical in all four; only the syntax differs. Menu paths change between versions, so check your release notes if a dialog does not match the labels below.
R — the p.adjust function takes the raw p-values and the method name.
raw <- c(0.004, 0.021, 0.048)
p.adjust(raw, method = "bonferroni")
# pairwise post hoc comparisons, Bonferroni adjusted
pairwise.test(x = score, g = group, p.adjust.method = "bonferroni")SPSS — use the Post Hoc dialog in Analyze > Compare Means > One-Way ANOVA, tick Bonferroni under Significance, or write the syntax by hand.
ONEWAY score BY group
/EMMEANS=COMPARE(group) ADJ(BONFERRONI) ALPHA(.05)
/WELCH.Stata — the test command already reports a Bonferroni-adjusted p-value next to the raw one, so you can read both from a single output row.
oneway score group, tabulate
test Control = Treatment
test Control = PlaceboSAS — PROC MULTTEST handles the adjustment and can write adjusted p-values straight into your output dataset.
proc multtest data=results;
procedure holm;
pvalues obs_p = p unadjusted;
ods output PValues = adjusted;
run;Whichever route you take, save both the raw and the corrected column in your output. You will need the raw value for the results table and the corrected one for the decision.
6. Report the Corrected Results
Reporting is where most marks are lost, because a correction that is never named reads as an unverified adjustment. Three things belong in the write-up: how many tests were in the family, which correction was applied, and what each comparison showed against the adjusted threshold.
APA style puts this in the text and the table note. A model sentence: “Three pairwise comparisons were conducted with a Bonferroni correction, giving an adjusted alpha of .0167.” Then report each test with its statistic, degrees of freedom, raw p-value, and whether it survived correction.
Table notes follow the same pattern: Note. Significance tested with Bonferroni-adjusted alpha of .0167 across three comparisons.
Common Mistakes
Six errors come up again and again. Each has a direct fix.
Counting every possible comparison instead of your planned family. Six groups do not mean 36 tests unless you genuinely ran 36. Fix: count the tests in the output, not the pairs that exist in theory.
Correcting twice. If your software already applied Bonferroni in a post hoc table, do not divide by m again. Fix: check whether the output column is labelled “Bonferroni adjusted” before touching anything.
Choosing the family after seeing the results. Including or excluding tests based on their p-values invalidates the correction. Fix: write down the family in your analysis plan, before the numbers exist.
Conflating Bonferroni with Holm. Holm-Bonferroni is a step-down procedure that is uniformly at least as powerful and usually more so. If your field expects “Bonferroni corrected,” ask whether Holm satisfies the reviewer, because the answer is very often yes.
Reporting adjusted alpha without saying how many tests produced it. A bare “.0167” is not reproducible. Fix: always pair the number with m.
Reading a lost significance as a failed study. A p-value of .021 that stops clearing the bar is the expected cost of controlling Type I error, not evidence that your study broke. Fix: report it as non-significant after correction and treat it as a Type II error trade-off.
Ignoring correlated comparisons. Pairwise comparisons drawn from the same groups share data and are not independent, which makes Bonferroni overly harsh. Tukey’s HSD for equal variances or Games-Howell when they differ handle that correlation and dominate the plain Bonferroni. Fix: use the ANOVA-specific post hoc test rather than hand-rolling Bonferroni.
Two habits keep most of this at bay: decide the family first, and state m in the write-up. Where the literature matters, Perneger’s 1998 commentary in BMJ explains why the correction is more conservative than the math strictly requires, and Benjamini and Hochberg’s 1995 paper on controlling the false discovery rate covers the main alternative.
Frequently Asked Questions
What is a Bonferroni procedure?
A Bonferroni procedure controls the chance of at least one false positive across a family of statistical tests. It works by splitting your original significance level across the number of tests: adjusted alpha equals alpha divided by m. With alpha at .05 and five comparisons, each test must reach .01 or lower. It is the simplest correction to compute, needs no special assumptions beyond independent tests, and is easy to describe in a paper, which is why it remains the default when a reviewer asks for corrected p-values.
When not to use Bonferroni correction?
Skip it when the tests were run on separate data sets with no shared family, or when the tests were already adjusted by your software. It is a poor fit when you screened dozens of variables for follow-up analysis, because dividing alpha across all of them leaves little power for anything. In that situation Benjamini-Hochberg controls the false discovery rate instead, and for pairwise comparisons among groups Tukey HSD or Games-Howell handle the correlation between comparisons far better.
How do you adjust for multiple comparisons in statistical analysis?
Pick your family of tests, divide your significance level by the number of tests in that family, and compare each p-value with the result. The equivalent form multiplies every p-value by m and caps it at 1.00. For pairwise group comparisons, count them as k(k-1)/2 where k is the number of groups. Always record m and name the correction in your write-up so a reader can reproduce the threshold.
How is the Bonferroni correction used in an ANOVA test?
The omnibus ANOVA tells you whether group means differ at all. Only after that do post hoc comparisons make sense, and those comparisons are where the correction applies. With k groups you run k(k-1)/2 pairwise tests and divide alpha by that number: three groups give three comparisons and an adjusted alpha of .0167. Most software does this automatically through a Bonferroni option in the post hoc dialog, though Tukey HSD is usually more powerful for equal variances.
What is the difference between an adjusted alpha and an adjusted p-value?
They are two ways of expressing the same correction. Adjusted alpha divides alpha by m and leaves the raw p-values untouched, so you compare each p-value against the stricter threshold. Adjusted p-values multiply each p-value by m, capped at 1.00, so you still compare against .05. The final decision is identical either way. Report adjusted p-values when a reviewer asks for corrected values, and adjusted alpha when the original p-values matter for the results table.
Is Bonferroni correction always required by APA?
No. APA style requires you to report the statistical method, the alpha level, and enough detail for a reader to follow the decision. It does not mandate a Bonferroni correction for every analysis with several tests. A correction becomes necessary when more than one test runs on the same data and one false positive would matter, and journals in some fields expect it by default. What you must do is state clearly which procedure you used and the threshold it produced.
Conclusion
Five actions, in order: name the family of tests, set your family-wise alpha, divide it by the number of comparisons, compare every p-value with the result, and record which ones survive. Then write it down — m, the adjusted alpha or adjusted p-values, and the correction’s name in your table note.
If a result stops clearing the bar after correction, that is the procedure working as designed rather than a failure of your study. The trade is fewer false positives in exchange for more false negatives, and reporting it honestly is what keeps the correction defensible in review.
