Structured Testing of 2×2 Factorial Effects: An Analytic Plan Requiring Fewer Observations

In a randomized 2×2 factorial trial, more than one hypothesis is to be tested, so some method must be used to control the probability of falsely rejecting at least one true hypothesis. We contrast familiar elementary methods of controlling the family-wise error rate based on the Bonferroni–Holm procedure with a less familiar but equally elementary form of structured testing associated with the large class of procedures that descend from the closed testing approach of Marcus, Peritz, and Gabriel. In a range of plausible situations, giving priority to main effects in structured testing typically yielded greater power to detect main effects for a given sample size or reduced sample size for a given power; it also permitted testing for interaction when main effects are found.