+1 Recommend
2 collections
    • Record: found
    • Abstract: found
    • Article: found
    Is Open Access

    Peer review – issues, limitations, and future development

    Jan Velterop1,*

    ScienceOpen Research – Section: SOR-EDU


    Assessment, Evaluation & Research methods, Peer review, Scientific publishing, Open access

    Read this article at

        There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.


        Peer review is almost universally seen as the crux of scientific journal publishing. The role of peer reviewers is (1) to help avoid unnecessary errors in the published article, and (2) to judge publication-worthiness (in the journal that arranges for the review). This happens. Sometimes. But the notion of peer review is rather vague, and since most of it is anonymous, it is very difficult – arguably impossible – for researchers to know if the articles they read have been reliably peer reviewed and which criteria have been used to come to the decision to accept for publication. On top of that, peer review is very expensive. Not the peer review itself, as it is mostly done by researchers without being paid for it, but the process as arranged by publishers. This has several underlying causes, but it is clear that the actual cost of technically publishing an article is but a fraction of the average APC (Article Processing Charge) income or per-article subscription revenues publishers routinely realize. Some (e.g. Richard Smith, ex-Editor of the British Medical Journal) advocate abolishing peer review altogether. This is certainly not without merit, but even without abolishing it, there are ways to make peer review more reliable and transparent, and much cheaper to the scientific community.

        Related collections

        Most cited references 4

        • Record: found
        • Abstract: found
        • Article: found
        Is Open Access

        A Reliability-Generalization Study of Journal Peer Reviews: A Multilevel Meta-Analysis of Inter-Rater Reliability and Its Determinants

        Background This paper presents the first meta-analysis for the inter-rater reliability (IRR) of journal peer reviews. IRR is defined as the extent to which two or more independent reviews of the same scientific document agree. Methodology/Principal Findings Altogether, 70 reliability coefficients (Cohen's Kappa, intra-class correlation [ICC], and Pearson product-moment correlation [r]) from 48 studies were taken into account in the meta-analysis. The studies were based on a total of 19,443 manuscripts; on average, each study had a sample size of 311 manuscripts (minimum: 28, maximum: 1983). The results of the meta-analysis confirmed the findings of the narrative literature reviews published to date: The level of IRR (mean ICC/r2 = .34, mean Cohen's Kappa = .17) was low. To explain the study-to-study variation of the IRR coefficients, meta-regression analyses were calculated using seven covariates. Two covariates that emerged in the meta-regression analyses as statistically significant to gain an approximate homogeneity of the intra-class correlations indicated that, firstly, the more manuscripts that a study is based on, the smaller the reported IRR coefficients are. Secondly, if the information of the rating system for reviewers was reported in a study, then this was associated with a smaller IRR coefficient than if the information was not conveyed. Conclusions/Significance Studies that report a high level of IRR are to be considered less credible than those with a low level of IRR. According to our meta-analysis the IRR of peer assessments is quite limited and needs improvement (e.g., reader system).
          • Record: found
          • Abstract: not found
          • Article: not found

          The outflow of academic papers from China: why is it happening and can it be stemmed?

            • Record: found
            • Abstract: not found
            • Article: not found

            The pain of rejection


              Author and article information

              [1]Independent Scholar, Guildford, Surrey, UK
              Author notes
              [*]Corresponding author's e-mail address:
              ScienceOpen Research
              29 September 2015
              : 0 (ID: 1dcfbe69-c30c-4eaa-a003-948c9700da40)
              : 0
              : 1-5
              © 2015 J. Velterop.

              This work has been published open access under Creative Commons Attribution License CC BY 4.0, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Conditions, terms of use and publishing policy can be found at

              Figures: 0, Tables: 0, References: 16, Pages: 5
              Original article
              ScienceOpen disciplines:


              Comment on this article

              Register to benefit from advanced discovery features on more than 42,000,000 articles

              Already registered?