Research Article: A discussion on significance indices for contingency tables under small sample sizes

Date Published: August 2, 2018

Publisher: Public Library of Science

Author(s): Natalia L. Oliveira, Carlos A. de B. Pereira, Marcio A. Diniz, Adriano Polpo, Mauro Gasparini.


Hypothesis testing in contingency tables is usually based on asymptotic results, thereby restricting its proper use to large samples. To study these tests in small samples, we consider the likelihood ratio test (LRT) and define an accurate index for the celebrated hypotheses of homogeneity, independence, and Hardy-Weinberg equilibrium. The aim is to understand the use of the asymptotic results of the frequentist Likelihood Ratio Test and the Bayesian FBST (Full Bayesian Significance Test) under small-sample scenarios. The proposed exact LRT p-value is used as a benchmark to understand the other indices. We perform analysis in different scenarios, considering different sample sizes and different table dimensions. The conditional Fisher’s exact test for 2 × 2 tables and the Barnard’s exact test are also discussed. The main message of this paper is that all indices have very similar behavior, except for Fisher and Barnard tests that has a discrete behavior. The most powerful test was the asymptotic p-value from the likelihood ratio test, suggesting that is a good alternative for small sample sizes.

Partial Text

We discuss indices for homogeneity, independence, and Hardy-Weinberg equilibrium hypotheses [1, 2] in contingency tables. We propose an exact evaluation of the Likelihood Ratio Test (LRT) as a benchmark significance index. Based on the work of [3], its idea is to evaluate the probability distribution of all possible tables on the sample space under the null hypothesis. Once the distribution for sampling contingency tables under the hypothesis is known, we are able to compute the exact distribution of the Likelihood Ratio Test (LRT) statistics. The main difficulty for this procedure is that it is computationally time-consuming, being only feasible for small sample sizes and/or for tables of small dimension.

After evaluating the indices for tables in different scenarios, we noticed that all of them had very similar behaviors, independently of the perspective (Bayesian or frequentist), sample size and table dimension. The exceptions are the p-values for Fisher and Barnard’s exact tests for the homogeneity hypothesis in 2 × 2 tables, and Barnard’s exact test for Hardy-Weinberg equilibrium, which show a discretized behavior. Studying the power functions considering homogeneity hypothesis in 2 × 2 tables and Hardy-Weinberg equilibrium hypothesis, the LRT presented itself as a powerful test when considering small sample sizes, while Fisher’s exact test was the least powerful one for the homogeneity hypothesis and the Barnard’s exact test was the least powerful for the Hardy-Weinberg equilibrium hypothesis. By enlarging sample sizes, the power of these tests increases accordingly.




0 0 vote
Article Rating
Notify of
Inline Feedbacks
View all comments