A t-test compares the means of two groups. One-way ANOVA compares the means of three or more, testing whether the variation between groups is large compared with the variation inside them (the F ratio). Run both with the t-Test Calculator and the ANOVA Calculator; background in t-test and p-value.
The multiple-comparison problem
Each pairwise test has a 5 % chance of a false alarm, and the chance grows as tests are added (this calculation assumes independent tests, an approximation).
| Groups | Pairwise t-tests needed | Chance of at least one false positive at α = 0.05 |
|---|---|---|
| 3 | 3 | 14.3 % |
| 4 | 6 | 26.5 % |
| 5 | 10 | 40.1 % |
| 6 | 15 | 53.7 % |
Two groups: they are the same test
With exactly two groups a pooled-variance t-test and ANOVA give the same p-value, and F = t². For the samples 4, 5, 6, 7, 8 and 6, 7, 8, 9, 10, both give t² = F = 4.0 and p = 0.0805 (checked with SciPy).
Which to use
- Two groups: t-test (Welch's version when the variances differ).
- Three or more groups: ANOVA, then post hoc pairwise tests if it is significant.
- The same group measured repeatedly: use the paired or repeated-measures versions of both.
- Strongly skewed data or tiny samples: consider a rank-based alternative (Mann-Whitney, Kruskal-Wallis).