For Publishers
DiscoveryMetadataPeer reviewHostingPublishing
For Researchers
JoinPublishReviewCollect
Register
Blog
About
Search
Advanced search
My ScienceOpen
Search
For Publishers
For Researchers
Blog
About
49
views
59
references
 Top references
cited by
71
    
0 reviews
 Review
0
comments
 Comment
0
recommends
+1 Recommend
0 collections
    0
    shares
      162
      similar
       All similar
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Impact of interventions to improve the quality of peer review of biomedical journals: a systematic review and meta-analysis

      research-article

      Read this article at

      ScienceOpenPublisherPMC
      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Background

          The peer review process is a cornerstone of biomedical research. We aimed to evaluate the impact of interventions to improve the quality of peer review for biomedical publications.

          Methods

          We performed a systematic review and meta-analysis. We searched CENTRAL, MEDLINE (PubMed), Embase, Cochrane Database of Systematic Reviews, and WHO ICTRP databases, for all randomized controlled trials (RCTs) evaluating the impact of interventions to improve the quality of peer review for biomedical publications.

          Results

          We selected 22 reports of randomized controlled trials, for 25 comparisons evaluating training interventions (n = 5), the addition of a statistical peer reviewer (n = 2), use of a checklist (n = 2), open peer review (i.e., peer reviewers informed that their identity would be revealed; n = 7), blinded peer review (i.e., peer reviewers blinded to author names and affiliation; n = 6) and other interventions to increase the speed of the peer review process (n = 3). Results from only seven RCTs were published since 2004. As compared with the standard peer review process, training did not improve the quality of the peer review report and use of a checklist did not improve the quality of the final manuscript. Adding a statistical peer review improved the quality of the final manuscript (standardized mean difference (SMD), 0.58; 95 % CI, 0.19 to 0.98). Open peer review improved the quality of the peer review report (SMD, 0.14; 95 % CI, 0.05 to 0.24), did not affect the time peer reviewers spent on the peer review (mean difference, 0.18; 95 % CI, –0.06 to 0.43), and decreased the rate of rejection (odds ratio, 0.56; 95 % CI, 0.33 to 0.94). Blinded peer review did not affect the quality of the peer review report or rejection rate. Interventions to increase the speed of the peer review process were too heterogeneous to allow for pooling the results.

          Conclusion

          Despite the essential role of peer review, only a few interventions have been assessed in randomized controlled trials. Evidence-based peer review needs to be developed in biomedical journals.

          Electronic supplementary material

          The online version of this article (doi:10.1186/s12916-016-0631-5) contains supplementary material, which is available to authorized users.

          Related collections

          Most cited references59

          • Record: found
          • Abstract: not found
          • Article: not found

          Peer review: a flawed process at the heart of science and journals.

          Richard Smith (2006)
          Article 0 comments Cited 188 times – based on 0 reviews     Review now
            Bookmark
            • Record: found
            • Abstract: found
            • Article: not found

            Evidence b(i)ased medicine--selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications.

            B Beermann, J Rastad, Hans Melander … (2003)
            To investigate the relative impact on publication bias caused by multiple publication, selective publication, and selective reporting in studies sponsored by pharmaceutical companies. 42 placebo controlled studies of five selective serotonin reuptake inhibitors submitted to the Swedish drug regulatory authority as a basis for marketing approval for treating major depression were compared with the studies actually published (between 1983 and 1999). Multiple publication: 21 studies contributed to at least two publications each, and three studies contributed to five publications. Selective publication: studies showing significant effects of drug were published as stand alone publications more often than studies with non-significant results. Selective reporting: many publications ignored the results of intention to treat analyses and reported the more favourable per protocol analyses only. The degree of multiple publication, selective publication, and selective reporting differed between products. Thus, any attempt to recommend a specific selective serotonin reuptake inhibitor from the publicly available data only is likely to be based on biased evidence.
            Article 0 comments Cited 146 times – based on 0 reviews     Review now
              Bookmark
              • Record: found
              • Abstract: found
              • Article: found
              Is Open Access

              Effect of using reporting guidelines during peer review on quality of final manuscripts submitted to a biomedical journal: masked randomised trial

              E Cobo, J. A. Cortés, J M Ribera … (2011)
              Objective To investigate the effect of an additional review based on reporting guidelines such as STROBE and CONSORT on quality of manuscripts. Design Masked randomised trial. Population Original research manuscripts submitted to the Medicina Clínica journal from May 2008 to April 2009 and considered suitable for publication. Intervention Control group: conventional peer reviews alone. Intervention group: conventional review plus an additional review looking for missing items from reporting guidelines. Outcomes Manuscript quality, assessed with a 5 point Likert scale (primary: overall quality; secondary: average quality of specific items in paper). Main analysis compared groups as allocated, after adjustment for baseline factors (analysis of covariance); sensitivity analysis compared groups as reviewed. Adherence to reviewer suggestions assessed with Likert scale. Results Of 126 consecutive papers receiving conventional review, 34 were not suitable for publication. The remaining 92 papers were allocated to receive conventional reviews alone (n=41) or additional reviews (n=51). Four papers assigned to the conventional review group deviated from protocol; they received an additional review based on reporting guidelines. We saw an improvement in manuscript quality in favour of the additional review group (comparison as allocated, 0.25, 95% confidence interval –0.05 to 0.54; as reviewed, 0.33, 0.03 to 0.63). More papers with additional reviews than with conventional reviews alone improved from baseline (22 (43%) v eight (20%), difference 23.6% (3.2% to 44.0%), number needed to treat 4.2 (from 2.3 to 31.2), relative risk 2.21 (1.10 to 4.44)). Authors in the additional review group adhered more to suggestions from conventional reviews than to those from additional reviews (average increase 0.43 Likert points (0.19 to 0.67)). Conclusions Additional reviews based on reporting guidelines improve manuscript quality, although the observed effect was smaller than hypothesised and not definitively demonstrated. Authors adhere more to suggestions from conventional reviews than to those from additional reviews, showing difficulties in adhering to high methodological standards at the latest research phases. To boost paper quality and impact, authors should be aware of future requirements of reporting guidelines at the very beginning of their study. Trial registration and protocol Although registries do not include trials of peer review, the protocol design was submitted to sponsored research projects (Instituto de Salud Carlos III, PI081903).
              Article 0 comments Cited 100 times – based on 0 reviews     Review now
                Bookmark
                All references

                Author and article information

                Contributors
                Isabelle Boutron: +33 01 44 07 39 75 , isabelle.boutron@htd.aphp.fr
                Journal
                Journal ID (nlm-ta): BMC Med
                Journal ID (iso-abbrev): BMC Med
                Title: BMC Medicine
                Publisher: BioMed Central (London )
                ISSN (Electronic): 1741-7015
                Publication date (Electronic): 10 June 2016
                Publication date PMC-release: 10 June 2016
                Publication date Collection: 2016
                Volume: 14
                Electronic Location Identifier: 85
                Affiliations
                [ ]Department of Epidemiology, Mailman School of Public Health, Columbia University, New York, NY USA
                [ ]INSERM, UMR 1153, Centre of Research in Epidemiology and Statistics Sorbonne Paris Cité – (CRESS), METHODS team, Paris, France
                [ ]Paris Descartes University, Paris, France
                [ ]Emergency department, Hôpital Lariboisière, Assistance Publique des Hôpitaux de Paris, University Diderot, Paris, France
                [ ]Centre d’Épidémiologie Clinique, Hôpital Hôtel-Dieu Escalier A2, 1 er étage, Assistance Publique des Hôpitaux de Paris, 1 Place du Parvis Notre-Dame, Paris, 75004 France
                Article
                Publisher ID: 631
                DOI: 10.1186/s12916-016-0631-5
                PMC ID: 4902984
                PubMed ID: 27287500
                SO-VID: 18421f34-3e65-4a54-b5a6-7daeac53813e
                Copyright © © The Author(s). 2016
                License:

                Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

                History
                Date received : 30 April 2016
                Date accepted : 28 May 2016
                Categories
                Subject: Research Article
                Custom metadata
                issue-copyright-statement © The Author(s) 2016

                ScienceOpen disciplines: Medicine
                Keywords: peer review process,peer reviewers,systematic review,meta-analysis
                Data availability:
                ScienceOpen disciplines: Medicine
                Keywords: peer review process, peer reviewers, systematic review, meta-analysis

                Comments

                Comment on this article

                scite_

                Similar content162

                See all similar

                Cited by60

                See all cited by

                Most referenced authors1,459

                See all reference authors