Some Counterclaims Undermine Themselves in Observational Studies

Claims based on observational studies that a treatment has certain e§ects are often met with counterclaims asserting that the treatment is entirely without e§ect, that all associations with treatment are produced by biased treatment assignment. Some counterclaims undermine themselves in the following speciÖc sense: presuming the counterclaim to be true may strengthen the support that the original data provide for the original claim, so that the counterclaim fails in its role as a critique of the original claim. In mathematics, a proof by contradiction supposes a proposition to be true en route to proving that the proposition is false. Analogously, the supposition that a particular counterclaim is true may justify an otherwise unjustiÖed statistical analysis, and this added analysis may interpret the original data as providing even stronger support for the original claim. More precisely, the original study is sensitive to unmeasured biases of a particular magnitude, , but an analysis that supposes the counterclaim to be true may be insensitive to much larger unmeasured biases, 0 > . Illustrated using data from the US Fatal Accident Reporting System.