Introduction
If you've ever presented research with a designed experiment, you've
likely heard the critique: 'Your sample size is too small.' But is there a
statistical reason behind the common rule of thumb that your residual degrees
of freedom should be at least 12? Let's break it down.
The Experimental Design: Blocking on a Fertility Gradient
Imagine
that you have an area with a fertility gradient to conduct an experiment. The
land is sloped and therefore more fertile at the bottom than at the top. You
want to compare four treatments, which we will indicate as A, B, C, and D, and
decide to design five blocks. Each block contains four treatments. The design
could be as shown in the Figure 1.
Figure 1
Design of an experiment in blocks
This design is appropriate because it groups similar experimental units
(plots) together into blocks, effectively accounting for the major fertility
gradient and reducing the unexplained variation (error) used to test the
treatments. But
what can be said about the number of degrees of freedom of the residual?
Understanding the ANOVA Table
See the breakdown of degrees of freedom (df)for this
experiment in Table 1.
Table 1
Degrees of Freedom Schema
|
Source |
df |
|
Treatments |
3 |
|
Blocks |
4 |
|
Error |
12 |
|
Total |
19 |
The most
common criticism of experimental work is that the sample size is too small.
Sometimes it is also argued that the number of degrees of freedom of the
residual should be greater than 10 or 12. But why?
How Adding Blocks Affects Degrees of Freedom
Remember
that you want to compare four treatments. Therefore, the number of degrees of
freedom for treatments must be 3. If you increase the sample size, by how much
does the number of degrees of freedom of the residual increase?
See the
Table 2, which shows the increase in the number of degrees of freedom of
the residual when the sample size is increased or, more specifically in the
case of the example, the number of blocks.
Table 2
Degrees of freedom of the residual as a function of
the number of blocks (4
treatments)
|
Statistics |
Number of blocks |
|||||
|
3 |
4 |
5 |
6 |
7 |
8 |
|
|
DF (Treatments) |
3 |
3 |
3 |
3 |
3 |
3 |
|
DF(Blocks |
2 |
3 |
4 |
5 |
6 |
7 |
|
DF (error) |
6 |
9 |
12 |
15 |
18 |
21 |
|
Number of plots |
12 |
16 |
20 |
24 |
28 |
32 |
The Key Insight: Looking at the F-Distribution
Look
at the F distribution in Table 3. Pay particular attention to the column for 3
degrees of freedom in the numerator (because you want to compare four
treatments). Note that the critical F values with 3 degrees of freedom in the
numerator tend to stabilize after 12 degrees of freedom in the denominator
(residual from the analysis of variance). Next, look at the Table 3, of
critical F values as a function of the number of degrees of freedom in the
denominator (number of degrees of freedom in the numerator = 3).
Table 3 F distribution table
Critical values at 5% F (number of degrees of freedom in the denominator and number of degrees of freedom in the numerator = 3)
Conclusion:
Why 12 is the Magic Number
This
becomes clearer when looking at the Figure 2. Since it is the value of F that
determines significance, your ability to detect differences between the means
of the four treatments improves if you arrange five blocks instead of four
(critical F decreases from 3.86 to 3.49). However, it does not improve much if,
instead of five blocks, you use six (critical F decreases from 3.49 to 3.29).
Figure
2
Critical values at 5% of F (number of degrees
of freedom in the denominator and number of degrees of freedom in the
numerator = 3)
So, the next time you're designing an experiment, remember that while
more replication is always better, there's a point of diminishing returns. Aim
for that sweet spot of 12-15 residual degrees of freedom to
maximize the power and reliability of your F-tests without wasting unnecessary
resources.
✨ Hence the rule of thumb: at least 12 degrees
of freedom in the residuals of the analysis of variance.
No comments:
Post a Comment