# Two-sided Exact Tests and Matching Confidence Intervals for Discrete Data

Disclaimer: The provided code links for this paper are external links. Science Nest has no responsibility for the accuracy, legality or content of these links. Also, by downloading this code(s), you agree to comply with the terms of use as set out by the author(s) of the code(s).

 Authors Michael P Fay Journal/Conference Name R Journal Paper Category Other Paper Abstract There is an inherent relationship between two-sided hypothesis tests and confidence intervals. A series of two-sided hypothesis tests may be inverted to obtain the matching 100(1-Î±)% confidence interval defined as the smallest interval that contains all point null parameter values that would not be rejected at the Î± level. Unfortunately, for discrete data there are several different ways of defining two-sided exact tests and the most commonly used twosided exact tests are defined one way, while the most commonly used exact confidence intervals are inversions of tests defined another way. This can lead to inconsistencies where the exact test rejects but the exact confidence interval contains the null parameter value. The packages exactci and exact2x2 provide several exact tests with the matching confidence intervals avoiding these inconsistencies as much as possible. Examples are given for binomial and Poisson parameters and both paired and unpaired 2Ã— 2 tables. Applied statisticians are increasingly being encouraged to report confidence intervals (CI) and parameter estimates along with p-values from hypothesis tests. The htest class of the stats package is ideally suited to these kinds of analyses, because all the related statistics may be presented when the results are printed. For exact two-sided tests applied to discrete data, a test-CI inconsistency may occur: the p-value may indicate a significant result at level Î± while the associated 100(1-Î±)% confidence interval may cover the null value of the parameter. Ideally, we would like to present a unified report (Hirji, 2006), whereby the p-value and the confidence interval match as much as possible. A motivating example I was asked to help design a study to determine if adding a new drug (albendazole) to an existing treatment regimen (ivermectin) for the treatment of a parasitic disease (lymphatic filariasis) would increase the incidence of a rare serious adverse event when given in an area endemic for another parasitic disease (loa loa). There are many statistical issues related to that design (Fay et al., 2007), but here consider a simple scenario to highlight the point of this paper. A previous mass treatment using the existing treatment had 2 out of 17877 experiencing the serious adverse event (SAE) giving an observed rate of 11.2 per 100,000. Suppose the new treatment was given to 20,000 new subjects and suppose that 10 subjects experienced the SAE giving an observed rate of 50 per 100,000. Assuming Poisson rates, an exact test using poisson.test(c(2,10),c(17877,20000)) from the stats package (throughout we assume version 2.11.0 for the stats package) gives a p-value of p = 0.0421 implying significant differences between the rates at the 0.05 level, but poisson.test also gives a 95% confidence interval of (0.024,1.050) which contains a rate ratio of 1, implying no significant difference. We return to the motivating example in the â€˜Poisson two-sampleâ€™ section later. Overview of two-sided exact tests We briefly review inferences using the p-value function for discrete data. For details see Hirji (2006) or Blaker (2000). Suppose you have a discrete statistic t with random variable T such that larger values of T imply larger values of a parameter of interest, Î¸. Let FÎ¸(t) = Pr[T â‰¤ t;Î¸] and FÎ¸(t) = Pr[T â‰¥ t;Î¸]. Suppose we are testing H0 : Î¸ â‰¥ Î¸0 H1 : Î¸ pm it is possible for pc Î±. To calculate the matching confidence intervals, we consider only regular cases where FÎ¸(t) and FÎ¸(t) are monotonic functions of Î¸ (except perhaps the degenerate cases where FÎ¸(t) = 1 or FÎ¸(t) = 0 for all Î¸ when t is the maximum or minimum). In this case the matching confidence limits to the central test are (Î¸L,Î¸U) which are solutions to: Date of publication 2010 Code Programming Language R Comment Copyright Researcher 2022