A Practical Rule for Residual Degrees of Freedom in ANOVA

 

Imagine you are conducting an experiment in an area with a fertility gradient. The land is on a slope and is therefore more fertile at the bottom than at the top. You want to compare four treatments, which we will call A, B, C, and D, and you decide to arrange them in five blocks. Each block can accommodate four plots. The experimental design could be the one shown in Figure 1.

            Figure 1: Layout of a randomized complete block design

Table 1 presents the Analysis of Variance (ANOVA) for this experiment.

             Table 1: Analysis of Variance (ANOVA)

This design is appropriate because the variation within each block has been minimized (by grouping similar fertility levels together), and the variation between blocks has been maximized. But what can be said about the number of residual degrees of freedom?

The most repeated criticism in experimental work is that the sample size is too small. Sometimes it is also argued that the number of residual degrees of freedom should be greater than 10 or 12. But why?

Remember that you want to compare four treatments. Therefore, the degrees of freedom for treatments are necessarily 3. If you increase the sample size, by how much does the residual degrees of freedom increase? Look at Table 2, which shows the increase in residual degrees of freedom as the sample size—more specifically, the number of blocks—increases.

   Table 2: Residual Degrees of Freedom for 4 Treatments and a Varying Number of Blocks

Now, observe Table 3 below. It provides some critical values of F for 3 degrees of freedom in the numerator (because you are comparing four treatments) and various degrees of freedom in the denominator (the residual). Notice that the critical F-values stabilize after the denominator has about 12 degrees of freedom. Therefore, increasing the number of blocks beyond this point does not help much in achieving statistical significance.

           Table 3: Critical F-values at the 5% significance level for 3 numerator df and various denominator df.

This becomes clearer by looking at Figure 2. The F-value is what determines significance. So, your ability to detect differences between the means of the four treatments improves if you organize five blocks instead of four (the critical F decreases from 3.86 to 3.49). However, it does not improve as much if you use six blocks instead of five (the critical F only decreases from 3.49 to 3.29).

        Figure 2: A graph plotting the data from Table 3, showing the critical F-value rapidly decreasing and then leveling off as the residual df increases.

This is the origin of the practical rule: aim for at least 12 residual degrees of freedom in the ANOVA. But note well: this is for 4 treatments. In agricultural sciences, it is common to compare 4 or even more treatments. Therefore, this rule is quite reasonable.

Summary

Here is a well-established and practical rule of thumb.

 

·        The power of an ANOVA F-test to detect differences between treatments depends on the critical F-value.

·        This critical F-value drops quickly as the residual degrees of freedom (df) increase from a low number but stabilizes after around 10-12 df.

·        Therefore, beyond a certain point (e.g., 12 residual df), adding more replicates (blocks) provides diminishing returns for the cost and effort involved.