Background: Reproducible research includes sharing data and code. The reproducibility policy at the journal Biostatistics rewards articles with badges for data and code sharing. This study investigates the effect of badges at increasing reproducible research, specifically, data and code sharing, at Biostatistics.
Methods: The setting of this observational study is the Biostatistics and Statistics in Medicine (control journal) online research archives. The data consisted of 240 randomly sampled articles from 2006 to 2013 (30 articles per year) per journal, a total sample of 480 articles. Data analyses included: plotting probability of data and code sharing by article submission date, and Bayesian logistic regression modelling to test for a difference in the probability of making data and code available after the introduction of badges at Biostatistics.
Results: The probability of data sharing was higher at Biostatistics than the control journal but the probability of code sharing was comparable for both journals. The probability of data sharing increased by 3.5 times (95% credible interval: 1.4 to 7.4 times, p-value probability that sharing increased: 0.996) after badges were introduced at Biostatistics. On an absolute scale, however, this difference was only a 7.3% increase in data sharing (95% CI: 2 to 14%, p-value: 0.996). Badges did not have an impact on code sharing at the journal (mean increase: 1.1 times, 95% credible interval: 0.45 to 2.14 times, p-value probability that sharing increased: 0.549).
Conclusions: The effect of badges at Biostatistics was a 7.3% increase in the data sharing rate, 5 times less than the effect of badges on data sharing at Psychological Science (37.9% badge effect). Though the effect of badges at Biostatistics did not impact code sharing, and was associated with only a moderate effect on data sharing, badges are an interesting step that journals are taking to incentivise and promote reproducible research.