Studies accumulate over time and meta-analyses are mainly retrospective. These processes define meta stopping rules that should receive attention in recommendations to reduce research waste. The resulting bias — Accumulation Bias — is inevitable, and even if it can be approximated and accounted for, no valid p-value tests can be constructed. Fortunately, tests based on likelihood ratios withstand Accumulation Bias: they provide bounds on error probabilities that remain valid despite the bias. From this follow two approaches to consider time in error control: either treat individual (primary) studies and meta-analyses as two separate worlds — each with their own timing — or integrate individual studies in the meta-analysis world. Taking up likelihood ratios in either approach allows for valid tests that relate well to the accumulating nature of scientific knowledge.

doi.org/10.7490/f1000research.1117473.1
Safe Bayesian Inference: A Theory of Misspecification based on Statistical Learning
O'Bayes 2019: Objective Bayes Methodology Conference
Machine Learning

ter Schure, J. (2019). Efficient science + meta-analysis – Bayes comes in, and p-values are out: especially for frequentists!. doi:10.7490/f1000research.1117473.1