F-test for Equal Variances

Tests whether two samples have the same spread. The larger unbiased variance is divided by the smaller to give F, which is compared against the F distribution. The result guides the choice between the Welch and Student forms of the two-sample t-test.

When two machines produce the same part, the average dimension matters and so does the consistency. Testing whether two samples differ in spread is what the F-test does.

F = \dfrac{s_1^2}{s_2^2}

$s_1^2$ is the larger unbiased variance
$s_2^2$ is the smaller unbiased variance

If the two variances were equal, their ratio would sit near 1. The further from 1 it lands, the stronger the evidence that the spreads really differ, and the F distribution says how far is too far to blame on chance.

Putting the larger variance on top is deliberate: it forces $F$ to be at least 1, so only the upper tail needs examining. The price is that testing without having decided in advance which sample is more variable requires doubling the upper-tail probability to make it two-sided.

The shape of the F distribution is fixed by two degrees of freedom, one less than the count in the numerator sample and one less than the count in the denominator sample.

Example

Take the defaults, A as 12, 15, 18, 20, 25 and B as 10, 14, 15, 17, 19.

A has a mean of 18 and squared deviations summing to 98. Dividing by $5 - 1 = 4$ gives an unbiased variance of 24.5. B has a mean of 15, squared deviations summing to 46, and a variance of 11.5.

A is the larger, so $F$ is $24.5 \div 11.5$ , about 2.1304, with 4 degrees of freedom above and below.

The two-sided p-value is about 0.4819 against a critical value of about 9.6045. F falls far short, so the spreads cannot be said to differ. A better than two-to-one ratio of variances is still well inside what chance produces with only five values each.

Points to watch

This test leans heavily on the data being normally distributed. Modest departures from normality are known to distort it badly, which is why it is increasingly avoided in practice.

Textbooks describe running an F-test first to decide which form of the two-sample t-test to use. Current advice is to skip that and use the Welch form from the start. Welch assumes nothing about equal variances and loses almost no power when the variances happen to be equal anyway.

Failing to reach significance does not establish that the spreads are equal. As this example shows, five values per group cannot detect even a two-fold difference.