Analysis reproducibility in mental health research: challenges and solutions

Recent years have marked a renaissance in efforts to increase research reproducibility in psychology, neuroscience and related fields. Reproducibility is the cornerstone of a solid foundation of fundamental research – one that will support new theories built on valid findings and technological innovation that works. The increased focus on reproducibility has made the barriers to it increasingly apparent, along with the development of new tools and practices to overcome these barriers. Here, we review challenges, solutions, and emerging best practices with a particular emphasis on neuroimaging studies. We distinguish three main types of reproducibility, discussing each in turn. “Analytical reproducibility” is the ability to reproduce findings using the same data and methods. “Replicability” is the ability to find an effect in new datasets, using the same or similar methods. And “robustness to analytical variability” refers to the ability to identify a finding consistently across variation in methods. The incorporation of these tools and practices will result in a more reproducible, replicable, and robust psychological and brain research, and a stronger scientific foundation across fields of inquiry.