Support for Open Science is Equally High in Opt-In versus Representative Samples of Scientists

Many studies on open science attitudes and practice use opt-in samples. This strategy poses a problem: it is unclear whether and to what degree scholars who opt in to surveys about open science are more likely to support open science, compared to the whole population of scholars. Thus high levels of support for open science in prior surveys might reflect self-selection bias bias (Heckman, 1990). In the current study we compare attitudes toward open science in two samples of scientists, differing only in sampling strategy. The first sample was collected by emailing a survey to eligible researchers at a research focused institution, yielding a 27% participation rate. The second sample was collected by providing time to complete the survey at the beginning of regularly scheduled lab meetings at the same institution, yielding a 95% participation rate. Overall, support for open science practices was high, and there was positive evidence that the two samples answered the questions in the same way. This suggests that opt-in sampling strategies do not overestimate support for open science.