40
views
0
recommends
+1 Recommend
1 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      The effect of publishing peer review reports on referee behavior in five scholarly journals

      research-article

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          To increase transparency in science, some scholarly journals are publishing peer review reports. But it is unclear how this practice affects the peer review process. Here, we examine the effect of publishing peer review reports on referee behavior in five scholarly journals involved in a pilot study at Elsevier. By considering 9,220 submissions and 18,525 reviews from 2010 to 2017, we measured changes both before and during the pilot and found that publishing reports did not significantly compromise referees’ willingness to review, recommendations, or turn-around times. Younger and non-academic scholars were more willing to accept to review and provided more positive and objective recommendations. Male referees tended to write more constructive reports during the pilot. Only 8.1% of referees agreed to reveal their identity in the published report. These findings suggest that open peer review does not compromise the process, at least when referees are able to protect their anonymity.

          Abstract

          To increase transparency in science, some scholarly journals have begun publishing peer review reports. Here, the authors show how this policy shift affects reviewer behavior by analyzing data from five journals piloting open peer review.

          Related collections

          Most cited references29

          • Record: found
          • Abstract: not found
          • Book: not found

          Categorical Data Analysis

            Bookmark
            • Record: found
            • Abstract: found
            • Article: found
            Is Open Access

            Academic Research in the 21st Century: Maintaining Scientific Integrity in a Climate of Perverse Incentives and Hypercompetition

            Abstract Over the last 50 years, we argue that incentives for academic scientists have become increasingly perverse in terms of competition for research funding, development of quantitative metrics to measure performance, and a changing business model for higher education itself. Furthermore, decreased discretionary funding at the federal and state level is creating a hypercompetitive environment between government agencies (e.g., EPA, NIH, CDC), for scientists in these agencies, and for academics seeking funding from all sources—the combination of perverse incentives and decreased funding increases pressures that can lead to unethical behavior. If a critical mass of scientists become untrustworthy, a tipping point is possible in which the scientific enterprise itself becomes inherently corrupt and public trust is lost, risking a new dark age with devastating consequences to humanity. Academia and federal agencies should better support science as a public good, and incentivize altruistic and ethical outcomes, while de-emphasizing output.
              Bookmark
              • Record: found
              • Abstract: found
              • Article: not found

              Measuring the effectiveness of scientific gatekeeping.

              Peer review is the main institution responsible for the evaluation and gestation of scientific research. Although peer review is widely seen as vital to scientific evaluation, anecdotal evidence abounds of gatekeeping mistakes in leading journals, such as rejecting seminal contributions or accepting mediocre submissions. Systematic evidence regarding the effectiveness--or lack thereof--of scientific gatekeeping is scant, largely because access to rejected manuscripts from journals is rarely available. Using a dataset of 1,008 manuscripts submitted to three elite medical journals, we show differences in citation outcomes for articles that received different appraisals from editors and peer reviewers. Among rejected articles, desk-rejected manuscripts, deemed as unworthy of peer review by editors, received fewer citations than those sent for peer review. Among both rejected and accepted articles, manuscripts with lower scores from peer reviewers received relatively fewer citations when they were eventually published. However, hindsight reveals numerous questionable gatekeeping decisions. Of the 808 eventually published articles in our dataset, our three focal journals rejected many highly cited manuscripts, including the 14 most popular; roughly the top 2 percent. Of those 14 articles, 12 were desk-rejected. This finding raises concerns regarding whether peer review is ill--suited to recognize and gestate the most impactful ideas and research. Despite this finding, results show that in our case studies, on the whole, there was value added in peer review. Editors and peer reviewers generally--but not always-made good decisions regarding the identification and promotion of quality in scientific manuscripts.
                Bookmark

                Author and article information

                Contributors
                flaminio.squazzoni@unimi.it
                Journal
                Nat Commun
                Nat Commun
                Nature Communications
                Nature Publishing Group UK (London )
                2041-1723
                18 January 2019
                18 January 2019
                2019
                : 10
                : 322
                Affiliations
                [1 ]ISNI 0000 0001 2174 3522, GRID grid.8148.5, Department of Social Studies and Centre for Data Intensive Sciences and Applications, , Linnaeus University, ; 35195 Växjö, Sweden
                [2 ]ISNI 0000 0001 2173 938X, GRID grid.5338.d, Department of Computer Science, , University of Valencia, ; Av. de la Universitat, s/n, 46100 Burjassot, Spain
                [3 ]ISNI 0000 0001 2173 938X, GRID grid.5338.d, Department of Didactics of Mathematics, , University of Valencia, ; Av. Tarongers, 4, 46022 Valencia, Spain
                [4 ]ISNI 0000 0001 0672 9757, GRID grid.462207.5, STM Journals, Elsevier, ; Radarweg 29, 1043NX Amsterdam, The Netherlands
                [5 ]ISNI 0000 0004 1757 2822, GRID grid.4708.b, Department of Social and Political Sciences, , University of Milan, ; via Conservatorio 7, 20122 Milan, Italy
                Author information
                http://orcid.org/0000-0003-2837-0137
                http://orcid.org/0000-0002-1357-7170
                http://orcid.org/0000-0002-1325-2501
                http://orcid.org/0000-0003-4038-4531
                Article
                8250
                10.1038/s41467-018-08250-2
                6338763
                30659186
                953f1034-f2a0-4793-8809-1db2ddb976b0
                © The Author(s) 2019

                Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.

                History
                : 5 July 2018
                : 20 December 2018
                Categories
                Article
                Custom metadata
                © The Author(s) 2019

                Uncategorized
                Uncategorized

                Comments

                Comment on this article