+1 Recommend
5 collections
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      What is open peer review? A systematic review

      a , 1
      open peer review, Open Science, scholarly communication, research evaluation, publishing

      Read this article at

          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.


          Background: “Open peer review” (OPR), despite being a major pillar of Open Science, has neither a standardized definition nor an agreed schema of its features and implementations. The literature reflects this, with a myriad of overlapping and often contradictory definitions. While the term is used by some to refer to peer review where the identities of both author and reviewer are disclosed to each other, for others it signifies systems where reviewer reports are published alongside articles. For others it signifies both of these conditions, and for yet others it describes systems where not only “invited experts” are able to comment. For still others, it includes a variety of combinations of these and other novel methods.

          Methods: Recognising the absence of a consensus view on what open peer review is, this article undertakes a systematic review of definitions of “open peer review” or “open review”, to create a corpus of 122 definitions. These definitions are then systematically analysed to build a coherent typology of the many different innovations in peer review signified by the term, and hence provide the precise technical definition currently lacking.

          Results: This quantifiable data yields rich information on the range and extent of differing definitions over time and by broad subject area. Quantifying definitions in this way allows us to accurately portray exactly how  ambiguously the phrase “open peer review”  has been used thus far, for the literature offers a total of 22 distinct configurations of seven traits, effectively meaning that there are 22 different definitions of OPR in the literature.

          Conclusions: Based on this work, I propose a pragmatic definition of open peer review as an umbrella term for a number of overlapping ways that peer review models can be adapted in line with the ethos of Open Science, including making reviewer and author identities open, publishing review reports and enabling greater participation in the peer review process.

          Related collections

          Most cited references64

          • Record: found
          • Abstract: not found
          • Article: not found

          Publication prejudices: An experimental study of confirmatory bias in the peer review system

            • Record: found
            • Abstract: found
            • Article: found
            Is Open Access

            Do Pressures to Publish Increase Scientists' Bias? An Empirical Support from US States Data

            The growing competition and “publish or perish” culture in academia might conflict with the objectivity and integrity of research, because it forces scientists to produce “publishable” results at all costs. Papers are less likely to be published and to be cited if they report “negative” results (results that fail to support the tested hypothesis). Therefore, if publication pressures increase scientific bias, the frequency of “positive” results in the literature should be higher in the more competitive and “productive” academic environments. This study verified this hypothesis by measuring the frequency of positive results in a large random sample of papers with a corresponding author based in the US. Across all disciplines, papers were more likely to support a tested hypothesis if their corresponding authors were working in states that, according to NSF data, produced more academic papers per capita. The size of this effect increased when controlling for state's per capita R&D expenditure and for study characteristics that previous research showed to correlate with the frequency of positive results, including discipline and methodology. Although the confounding effect of institutions' prestige could not be excluded (researchers in the more productive universities could be the most clever and successful in their experiments), these results support the hypothesis that competitive academic environments increase not only scientists' productivity but also their bias. The same phenomenon might be observed in other countries where academic competition and pressures to publish are high.
              • Record: found
              • Abstract: not found
              • Article: not found

              Peer review: a flawed process at the heart of science and journals.


                Author and article information

                F1000Research (London, UK )
                27 April 2017
                : 6
                [1 ]Göttingen State and University Library, University of Göttingen, Göttingen, 37073, Germany
                [1 ]Urban & Public Affairs Librarian, Portland State University, Portland, OR, USA
                [1 ]Global Publishing Development department, Elsevier, RELX Group, Amsterdam, Netherlands
                [1 ]The BMJ, London, UK
                [1 ]Blue Brain Project, Swiss Federal Institute of Technology in Lausanne, Geneva, Switzerland
                Author notes

                Competing interests: No competing interests were disclosed.

                Competing interests: No competing interests were disclosed.

                Competing interests: No competing interests were disclosed.

                Competing interests: I am Executive Editor of The BMJ, which operates a version of open peer review, and I have previously been employed by PLOS and BioMed Central which operate different versions.

                Competing interests: I am a consultant for Frontiers Media SA, an Open Access publisher with its own system of Open Peer Review

                Copyright: © 2017 Ross-Hellauer T

                This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

                Funded by: European Commission H2020 project OpenAIRE2020
                Award ID: 643410
                This work is funded by the European Commission H2020 project OpenAIRE2020 (Grant agreement: 643410, Call: H2020-EINFRA-2014-1).
                The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript .
                Systematic Review
                Data Sharing
                Public Engagement
                Publishing & Peer Review
                Web and Social Media

                open peer review,open science,scholarly communication,research evaluation,publishing


                A must-read for those which are interested in open peer review (OPR), a pre-requisite for scholarly communication in the post-print 21st century. It's the first paper I read in which the author has tried to analyse and compare the manifold definitions and practices of open peer review at different (mega-)journals or platforms.

                2017-07-23 19:33 UTC

                Comment on this article