Hidden Publication Bias in Epidemiologic Secondary Data Analysis

We highlight a particular type of publication bias unique to secondary data analysis, and particularly common in epidemiologic research. We begin by setting a reminder of the scientific method of inquiry, and—by analogy with the movement for full transparency in clinical trials—present arguments for reporting all results of secondary data analysis. We then describe the ways in which data dredging—a subtle form a p-hacking—can lead to a distorted scientific literature; we highlight prior research that has empirically demonstrated this. We conclude by arguing that in order to combat this bias, epidemiologists should move toward preregistering analyses, and epidemiologic journals should encourage this through the implementation of Registered Reports. Finally, we respond to some common criticisms of preregistration.