Efficient exact p-value computation for small sample, sparse, and surprising categorical data

Gill Bejerano*, Nir Friedman, Naftali Tishby

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

27 Scopus citations


A major obstacle in applying various hypothesis testing procedures to datasets in bioinformatics is the computation of ensuing p-values. In this paper, we define a generic branch-and-bound approach to efficient exact p-value computation and enumerate the required conditions for successful application. Explicit procedures are developed for the entire Cressie-Read family of statistics, which includes the widely used Pearson and likelihood ratio statistics in a one-way frequency table goodness-of-fit test. This new formulation constitutes a first practical exact improvement over the exhaustive enumeration performed by existing statistical software. The general techniques we develop to exploit the convexity of many statistics are also shown to carry over to contingency table tests, suggesting that they are readily extendible to other tests and test statistics of interest. Our empirical results demonstrate a speed-up of orders of magnitude over the exhaustive computation, significantly extending the practical range for performing exact tests. We also show that the relative speed-up gain increases as the null hypothesis becomes sparser, that computation precision increases with increase in speed-up, and that computation time is very moderately affected by the magnitude of the computed p-value. These qualities make our algorithm especially appealing in the regimes of small samples, sparse null distributions, and rare events, compared to the alternative asymptotic approximations and Monte Carlo samplers. We discuss several established bioinformatics applications, where small sample size, small expected counts in one or more categories (sparseness), and very small p-values do occur. Our computational framework could be applied in these, and similar cases, to improve performance.

Original languageAmerican English
Pages (from-to)867-886
Number of pages20
JournalJournal of Computational Biology
Issue number5
StatePublished - 2004


  • Branch and bound
  • Categorical data
  • Exact tests
  • Real extension
  • p-value


Dive into the research topics of 'Efficient exact p-value computation for small sample, sparse, and surprising categorical data'. Together they form a unique fingerprint.

Cite this