Assessing the Homogeneity of Three Odds Ratios: A Case Study in Small-Sample Inference

Abstract

In an experiment on the effects of different types of high-frequency ventilation on lung damage in rabbits, six groups of 6 to 8 animals each were compared, using a factorial treatment structure of 3 frequency values crossed with 2 amplitudes. The resulting data were reduced to binary outcomes for each animal, producing a 3 × 2 × 2 contingency table. Although the numbers were small, there appeared to be a large effect of amplitude at the two extreme frequency levels, but there were no failures at either amplitude in the middle frequency group. The question of interest was whether the data provided evidence that the effect of amplitude differed between the 3 frequencies and in particular whether the effect in the middle group was lower than in the two extreme groups. Various models were considered for the 3 odds ratios in question, all seeking to incorporate minimally informative prior assumptions. Because of the small numbers, sensitivity to prior distribution specifications was considerable, and we compared the effect of assuming independent prior distributions on each cell in the 3 × 2 factorial with that of using a more structured prior distribution incorporating exchangeable row, column and interaction effects. Only under rather strong prior assumptions could it be concluded that there was substantial evidence of non-homogeneity. The analysis provides a case study of the sensitivity of inferences in small samples, in an example where the popular exact frequentist approach, based on a null hypothesis of equality of the odds ratios, breaks down.