Optimal test statistics for anytime-valid hypothesis tests

In this dissertation, we study hypothesis testing: evaluating whether sample data support a claim regarding a broader population. The approach is to assume that the claim is false and to examine whether this assumption, called the null hypothesis, holds up in light of the data. For example, researchers in a clinical trial wish to use the data to refute the hypothesis that their medication does not work. Hypothesis tests help guide decision-making in all walks of life, from inance to agriculture, by offering a structured framework to evaluate the strength of evidence against hypotheses. This dissertation is a contribution to the theory of anytime-valid hypothesis tests, which are tools that are compatible with lexible experimental design. That is, anytime-valid methods allow researchers to stop or continue their experiment based on observed data, which is not possible with traditional methods. Anytime-valid methods work by keeping track of a numerical measure of evidence—the e-process—against the null hypothesis. In particular, we consider e-processes obtained through the combination of e-statistics, which measure the evidence that can be derived from each data point separately. The hypothesis can be refuted if the combined evidence against it grows too large. The goal is therefore to construct e-statistics that give as much evidence as possible if the hypothesis is not true. These are called log-optimal e- statistics. In Chapter 3, we discuss the abstract problem of finding log-optimal e-statistics, which leads to a general recipe for their construction. In Chapters 4–7, we use this recipe to find the log-optimal e-statistics for specific hypotheses f interest (exponential families, conditional independence and group invariance). A key assumption underlying the optimality results in these chapters is that we know (or can learn) what exactly happens if the null hypothesis is not true. In chapter 8, we take a different approach and consider the worst case over a set of possible alternative hypotheses. That is, we study a setting where it is possible to compute the e-statistic that maximizes the rate at which evidence is accumulated in the worst case over the alternative.