In many areas of science, models involve unseen latent variables. Often these variables are such that, were we able to observe them, the testing of scientific hypotheses would be straightforward. A classical example is that of Bernoulli trials (tosses of a fair coin) observed with error. If the number of successes (heads) is observed, testing that the coin is fair is straightforward, but how should uncertainty in observation be taken into account?
Recently, the notions of fuzzy p-values and confidence levels have been introduced into the statistics literature as a way to describe the uncertainty inherent in a randomized test. The same ideas can be applied to uncertainty about latent variables. The fuzzy p-value puts our uncertainty directly onto the p-value scale, and permits simultaneous expression of the strength of the evidence and the level of uncertainty.
Using the example of testing for association in a 2x2 table, we first discuss issues of power and robustness in conditional tests. The example is then extended to include latent variables, showing both the flexibility and the usefulness of the fuzzy p-value approach. The approach is compared to the analogue, in this simple context, of a standard approach of forming test statistics by averaging over latent variables conditional on the observed data. The fuzzy p-value approach has advantages both in computation and robustness.