The Tukey-Kramer test

                                                                   Summary         

 When ANOVA indicates significant differences between groups, the next step is to identify which groups actually differ from each other. If the groups have unequal sizes (unbalanced samples), the Tukey-Kramer test is a reliable and robust choice. In this post, we explain how to apply this test step-by-step, with a real example and interpretation of the results.

Introduction

When a researcher obtains a significant result in ANOVA (Analysis of Variance) for an experiment involving three or more groups, there is a need to perform post-hoc tests to compare the means and identify which ones are statistically different.

Several tests are available for this purpose. This text covers the Tukey-Kramer test, which is recommended specifically for situations where the groups have unequal sizes. In these cases, it is necessary to adjust the procedure by replacing the common group size (n) with the individual sizes (n_i and n_j) of each pair being compared.

The Tukey-Kramer test

The Tukey-Kramer test assumes homoscedasticity, or homogeneous variances. Therefore, the mean square error (MSE) obtained from the analysis of variance (ANOVA) table is an estimate of the common variance of the variable.

The minimum significant difference (MSD) between the means of two groups of sizes n_i and n_j, denoted by d_ij is calculated using the following formula:

Where:

               ·       q (k, df, α) is the critical value from the studentized distribution.

               ·       k is the number of groups;

               ·       df is the degrees of freedom for the residual (error) in the ANOVA.

               ·       MSE is the mean square error.

               ·       α is the significance level (e.g. 0.05 for 5%).

Example

Table 1 presents data from an experiment with four groups (four brands of green tea). The means for each group are shown at the bottom of the table. The aim is to compare these means using the Tukey-Kramer test. First, an ANOVA must be performed, as shown in Table 2.

Next, pairwise comparisons of the group means are conducted. The test was applied using the value of q for a significance level of 5%, with k = 4 groups and df = n - k = 24 - 4 = 20 residual degrees of freedom.

Table 1: Folic acid (vitamin B) content in green tea leaves randomly selected from four brands (1)

Table 2: Analysis of variance for the data in Table 1

 For example:

To compare the mean of Brand 1 to Brand 2 (α = 5%), calculate:

To compare the mean of Brand 1 to Brand 3, use the same procedure, but with n₁ = 7 and n₃ = 6.

The same procedure is repeated for the remaining pairs. Table 3 shows the observed differences between the means, as well as the respective d_i,j values. If the absolute difference between two means is greater than the corresponding d_i,j, the null hypothesis of equality between those means (H₀: μi = μ_j) is rejected.

Table 3: Mean comparison using the Tukey–Kramer test                                                                                       

Interpretation

For example, the results in Table 3 indicate that Brand 1 has a significantly higher average folic acid content than Brand 4.

Approximation using the harmonic mean

Calculating all minimum significant differences for the Tukey-Kramer test is laborious if done by hand. Statistical software automates this process. In the past, when group sizes were approximately equal, a common approximation to simplify the calculation was to use the standard Tukey HSD formula, replacing n with the harmonic mean (H) of the sample sizes. The formula becomes:

This approach is an approximation and may not provide exact control of the significance level, but it can be found in older literature.

Using the data in Table 1, where the group sizes are 7, 5, 6 and 6, the harmonic mean H is calculated as follows:

                                                                                  

         Thus

Substituting these values yields a single d value for all comparisons. In this example, interpreting the results using this approximation remains consistent with the complete analysis.

Bibliography

1.    Chen. TS; Lui. CK; Smith. CH. Journal of the American Dietetic Association [1983.82(6):627-632].    Apud Devore. JL. Probability and Statistics for engineering and the sciences. Brooks Cole. 2015.On line books.  

2.    table of the studentized range - David Lane .             http://davidmlane.com/hyperstat/sr_table.html

3.    Multiple Comparisons With Unequal Sample Sizes

https://www.uvm.edu/~dhowell/gradstat/.../labs/.../Multcomp.html

            4 .  ANOVA & Tukey-Kramer test. https://www.youtube.com