A multi-disciplinary perspective on emergent and future innovations in peer review

There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

Abstract

Peer review of research articles is a core part of our scholarly communication system. In spite of its importance, the status and purpose of peer review is often contested. What is its role in our modern digital research and communications infrastructure? Does it perform to the high standards with which it is generally regarded? Studies of peer review have shown that it is prone to bias and abuse in numerous dimensions, frequently unreliable, and can fail to detect even fraudulent research. With the advent of Web technologies, we are now witnessing a phase of innovation and experimentation in our approaches to peer review. These developments prompted us to examine emerging models of peer review from a range of disciplines and venues, and to ask how they might address some of the issues with our current systems of peer review. We examine the functionality of a range of social Web platforms, and compare these with the traits underlying a viable peer review system: quality control, quantified performance metrics as engagement incentives, and certification and reputation. Ideally, any new systems will demonstrate that they out-perform current models while avoiding as many of the biases of existing systems as possible. We conclude that there is considerable scope for new peer review initiatives to be developed, each with their own potential issues and advantages. We also propose a novel hybrid platform model that, at least partially, resolves many of the technical and social issues associated with peer review, and can potentially disrupt the entire scholarly communication system. Success for any such development relies on reaching a critical threshold of research community engagement with both the process and the platform, and therefore cannot be achieved without a significant change of incentives in research environments.

Related collections

Most cited references 230

Record: found
Abstract: found
Article: not found

The Matthew Effect in Science: The reward and communication systems of science are considered.

R K Merton (1968)

This account of the Matthew effect is another small exercise in the psychosociological analysis of the workings of science as a social institution. The initial problem is transformed by a shift in theoretical perspective. As originally identified, the Matthew effect was construed in terms of enhancement of the position of already eminent scientists who are given disproportionate credit in cases of collaboration or of independent multiple discoveries. Its significance was thus confined to its implications for the reward system of science. By shifting the angle of vision, we note other possible kinds of consequences, this time for the communication system of science. The Matthew effect may serve to heighten the visibility of contributions to science by scientists of acknowledged standing and to reduce the visibility of contributions by authors who are less well known. We examine the psychosocial conditions and mechanisms underlying this effect and find a correlation between the redundancy function of multiple discoveries and the focalizing function of eminent men of science-a function which is reinforced by the great value these men place upon finding basic problems and by their self-assurance. This self-assurance, which is partly inherent, partly the result of experiences and associations in creative scientific environments, and partly a result of later social validation of their position, encourages them to search out risky but important problems and to highlight the results of their inquiry. A macrosocial version of the Matthew principle is apparently involved in those processes of social selection that currently lead to the concentration of scientific resources and talent (50).

0 comments Cited 1202 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: found

Is Open Access

Where Is Current Research on Blockchain Technology?—A Systematic Review

Jesse Yli-Huumo, Deokyoon Ko, Sujin Choi … (2016)

Blockchain is a decentralized transaction and data management technology developed first for Bitcoin cryptocurrency. The interest in Blockchain technology has been increasing since the idea was coined in 2008. The reason for the interest in Blockchain is its central attributes that provide security, anonymity and data integrity without any third party organization in control of the transactions, and therefore it creates interesting research areas, especially from the perspective of technical challenges and limitations. In this research, we have conducted a systematic mapping study with the goal of collecting all relevant research on Blockchain technology. Our objective is to understand the current research topics, challenges and future directions regarding Blockchain technology from the technical perspective. We have extracted 41 primary papers from scientific databases. The results show that focus in over 80% of the papers is on Bitcoin system and less than 20% deals with other Blockchain applications including e.g. smart contracts and licensing. The majority of research is focusing on revealing and improving limitations of Blockchain from privacy and security perspectives, but many of the proposed solutions lack concrete evaluation on their effectiveness. Many other Blockchain scalability related challenges including throughput and latency have been left unstudied. On the basis of this study, recommendations on future research directions are provided for researchers.

0 comments Cited 317 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: found

Is Open Access

The Oligopoly of Academic Publishers in the Digital Era

Vincent Larivière, Stefanie Haustein, Philippe Mongeon (2015)

The consolidation of the scientific publishing industry has been the topic of much debate within and outside the scientific community, especially in relation to major publishers’ high profit margins. However, the share of scientific output published in the journals of these major publishers, as well as its evolution over time and across various disciplines, has not yet been analyzed. This paper provides such analysis, based on 45 million documents indexed in the Web of Science over the period 1973-2013. It shows that in both natural and medical sciences (NMS) and social sciences and humanities (SSH), Reed-Elsevier, Wiley-Blackwell, Springer, and Taylor & Francis increased their share of the published output, especially since the advent of the digital era (mid-1990s). Combined, the top five most prolific publishers account for more than 50% of all papers published in 2013. Disciplines of the social sciences have the highest level of concentration (70% of papers from the top five publishers), while the humanities have remained relatively independent (20% from top five publishers). NMS disciplines are in between, mainly because of the strength of their scientific societies, such as the ACS in chemistry or APS in physics. The paper also examines the migration of journals between small and big publishing houses and explores the effect of publisher change on citation impact. It concludes with a discussion on the economics of scholarly publishing.

0 comments Cited 235 times     Rated -3 of 5. – based on 1 reviews

Bookmark

All references

Author and article information

Contributors

Jonathan P. Tennant: Role: ConceptualizationRole: Writing – Original Draft PreparationRole: Writing – Review & Editing URI : https://orcid.org/0000-0001-7794-0218

Jonathan M. Dugan: Role: Writing – Original Draft PreparationRole: Writing – Review & Editing URI : https://orcid.org/0000-0001-8525-6221

Daniel Graziotin: Role: Writing – Original Draft PreparationRole: Writing – Review & Editing URI : https://orcid.org/0000-0002-9107-7681

Damien C. Jacques: Role: Writing – Original Draft PreparationRole: Writing – Review & Editing URI : https://orcid.org/0000-0002-9069-4143

François Waldner: Role: Writing – Original Draft PreparationRole: Writing – Review & Editing URI : https://orcid.org/0000-0002-5599-7456

Daniel Mietchen: Role: Writing – Original Draft PreparationRole: Writing – Review & Editing

Yehia Elkhatib: Role: Writing – Original Draft PreparationRole: Writing – Review & Editing URI : https://orcid.org/0000-0003-4639-436X

Lauren B. Collister: Role: Writing – Original Draft PreparationRole: Writing – Review & Editing URI : https://orcid.org/0000-0001-5767-8486

Christina K. Pikas: Role: Writing – Original Draft PreparationRole: Writing – Review & Editing URI : https://orcid.org/0000-0002-4455-8036

Tom Crick: Role: Writing – Original Draft PreparationRole: Writing – Review & Editing URI : https://orcid.org/0000-0001-5196-9389

Paola Masuzzo: Role: Writing – Original Draft PreparationRole: Writing – Review & Editing URI : https://orcid.org/0000-0003-3699-1195

Anthony Caravaggi: Role: Writing – Original Draft PreparationRole: Writing – Review & Editing URI : https://orcid.org/0000-0002-1763-8970

Devin R. Berg: Role: Writing – Original Draft PreparationRole: Writing – Review & Editing URI : https://orcid.org/0000-0002-1193-3848

Kyle E. Niemeyer: Role: Writing – Original Draft PreparationRole: Writing – Review & Editing URI : https://orcid.org/0000-0003-4425-7097

Tony Ross-Hellauer: Role: Writing – Original Draft PreparationRole: Writing – Review & Editing URI : https://orcid.org/0000-0003-4470-7027

Sara Mannheimer: Role: Writing – Original Draft PreparationRole: Writing – Review & Editing URI : https://orcid.org/0000-0002-1433-6782

Lillian Rigling: Role: Writing – Original Draft PreparationRole: Writing – Review & Editing URI : https://orcid.org/0000-0002-9178-0422

Daniel S. Katz: Role: Writing – Original Draft PreparationRole: Writing – Review & Editing URI : https://orcid.org/0000-0001-5934-7525

Bastian Greshake Tzovaras: Role: Writing – Original Draft PreparationRole: Writing – Review & Editing URI : https://orcid.org/0000-0002-9925-9623

Josmel Pacheco-Mendoza: Role: Writing – Original Draft PreparationRole: Writing – Review & Editing URI : https://orcid.org/0000-0002-2251-8092

Nazeefa Fatima: Role: Writing – Original Draft PreparationRole: Writing – Review & Editing URI : https://orcid.org/0000-0001-7791-4984

Marta Poblet: Role: Writing – Original Draft PreparationRole: Writing – Review & Editing URI : https://orcid.org/0000-0002-0026-989X

Marios Isaakidis: Role: Writing – Original Draft PreparationRole: Writing – Review & Editing URI : https://orcid.org/0000-0001-8498-1309

Dasapta Erwin Irawan: Role: Writing – Original Draft PreparationRole: Writing – Review & Editing URI : https://orcid.org/0000-0002-1526-0863

Sébastien Renaut: Role: Writing – Original Draft PreparationRole: Writing – Review & Editing URI : https://orcid.org/0000-0002-8111-5082

Christopher R. Madan: Role: Writing – Original Draft PreparationRole: Writing – Review & Editing URI : https://orcid.org/0000-0003-3228-6501

Lisa Matthias: Role: Writing – Original Draft PreparationRole: Writing – Review & Editing URI : https://orcid.org/0000-0002-2612-2132

Jesper Nørgaard Kjær: Role: Writing – Original Draft PreparationRole: Writing – Review & Editing URI : https://orcid.org/0000-0001-9183-9861

Daniel Paul O'Donnell: Role: Writing – Original Draft PreparationRole: Writing – Review & Editing URI : https://orcid.org/0000-0002-0127-4893

Cameron Neylon: Role: Writing – Original Draft PreparationRole: Writing – Review & Editing URI : https://orcid.org/0000-0002-0068-716X

Sarah Kearns: Role: Writing – Original Draft PreparationRole: Writing – Review & Editing URI : https://orcid.org/0000-0001-5640-909X

Manojkumar Selvaraju: Role: Writing – Original Draft PreparationRole: Writing – Review & Editing URI : https://orcid.org/0000-0001-6439-5652

Julien Colomb: Role: Writing – Original Draft PreparationRole: Writing – Review & Editing URI : https://orcid.org/0000-0002-3127-5520

Journal

Journal ID (pmc): F1000Research

Title: F1000Research

Publisher: F1000 Research Limited (London, UK )

ISSN (Electronic): 2046-1402

Publication date (Electronic): 20 July 2017

Publication date Collection: 2017

Volume: 6

Electronic Location Identifier: 1151

Affiliations

[1 ]ScienceOpen, London, Germany

[2 ]University of California, Berkeley, San Francisco, CA, USA

[3 ]Institute of Software Technology, University of Stuttgart, Stuttgart, Germany

[4 ]Earth and Life Institute, Université catholique de Louvain, Louvain-la-Neuve, Belgium

[5 ]Data Science Institute, University of Virginia, Charlottesville, VA, USA

[6 ]School of Computing and Communications, Lancaster University, Lancaster, UK

[7 ]University Library System, University of Pittsburgh, Pittsburgh, PA, USA

[8 ]Johns Hopkins University Applied Physics Laboratory, Laurel, MD, USA

[9 ]Cardiff Metropolitan University, Cardiff, UK

[10 ]Department of Biochemistry, Ghent University, Ghent, Belgium

[11 ]VIB-UGent Center for Medical Biotechnology, Ghent, Belgium

[12 ]School of Biological, Earth and Environmental Sciences, University College Cork, Cork, Ireland

[13 ]Engineering & Technology Department, University of Wisconsin-Stout, Menomonie, WI, USA

[14 ]School of Mechanical, Industrial, and Manufacturing Engineering, Oregon State University, Corvallis, OR, USA

[15 ]State and University Library, University of Göttingen, Göttingen, Germany

[16 ]Montana State University, Bozeman, MT, USA

[17 ]Western University Libraries, London, ON, USA

[18 ]School of Information Sciences, University of Illinois at Urbana-Champaign, Urbana, IL, USA

[19 ]Department of Electrical and Computer Engineering, University of Illinois Urbana-Champaign, Urbana, IL, USA

[20 ]Department of Computer Science, University of Illinois Urbana-Champaign, Urbana, IL, USA

[21 ]National Center for Supercomputing Applications, University of Illinois at Urbana-Champaign, Urbana, IL, USA

[22 ]Institute of Cell Biology and Neuroscience, Goethe University Frankfurt, Frankfurt, Germany

[23 ]Universidad San Ignacio de Loyola, Lima, Peru

[24 ]Department of Biology, Faculty of Science, Lund University, Lund, Sweden

[25 ]Graduate School of Business and Law, RMIT University, Melbourne, Australia

[26 ]Department of Computer Science, University College London, London, UK

[27 ]Department of Groundwater Engineering, Faculty of Earth Sciences and Technology, Institut Teknologi Bandung, Bandung, Indonesia

[28 ]Département de Sciences Biologiques, Institut de Recherche en Biologie Végétale, Université de Montréal, Montreal, QC, Canada

[29 ]Department of Psychology, Boston College, Chestnut Hill, MA, USA

[30 ]Independent Researcher, Berlin, Germany

[31 ]Department of Affective Disorders, Psychiatric Research Academy, Aarhus University Hospital, Risskov, Denmark

[32 ]Department of English and Centre for the Study of Scholarly Communications, University of Lethbridge, Lethbridge, AB, Canada

[33 ]Centre for Culture and Technology, Curtin University, Perth, Australia

[34 ]Department of Chemical Biology, University of Michigan, Ann Arbor, MI, USA

[35 ]Integrated Gulf Biosystems, Riyadh, Saudi Arabia

[36 ]Saudi Human Genome Program, King Abdulaziz City for Science and Technology (KACST), Riyadh, Saudi Arabia

[1 ]Queensland University of Technology (QUT), Brisbane, Qld, Australia

[1 ]Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, ON, Canada

Author notes

[a ] jon.tennant.2@ 123456gmail.com

Competing interests: JPT works for ScienceOpen; TRH works for OpenAIRE.

Competing interests: I declare the following competing interests. I was aware of this paper before submission to F1000 and had considered participating in writing it when a call for collaborators was circulated on social media. However, in the end I did not read it or participate in writing it. I was the Chair of COPE (COPE is mentioned in the paper) until May this year and am still a Trustee. In addition, I know several of the authors. Jonathan Dugan and Cameron Neylon were colleagues at PLOS (various PLOS journals are mentioned in the paper), where I was involved with all the PLOS journals at one time or another. I was Medicine and Biology Editorial Director at PLOS at the time I left in April 2015. I was invited to give a talk by Marta Poblet at RMIT. I know some of the other authors by reputation.

Competing interests: I’m a member of the International Congress on Peer Review and Scientific Publication advisory committee; content expert for the Publons Academy; a policy advisory board member for the Journal of Clinical Epidemiology; and on the editorial board of Research Integrity and Peer Review.

Article

DOI: 10.12688/f1000research.12037.1

PubMed ID: 29188015

SO-VID: 74dfc0c7-7c41-467f-8185-33a6a66ab8b0

License:

This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

History

Date accepted : 13 July 2017

Funding

Funded by: Horizon 2020

Award ID: 643410

Award ID: H2020-EINFRA-2014-1

TRH was supported by funding from the European Commission H2020 project OpenAIRE2020 (Grant agreement: 643410, Call: H2020-EINFRA-2014-1).

The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Comments

Alexander Grossmann wrote:

What I really like is the comprehensive way in which the authors analysed and compared pros and cons of established and open peer review. Part 3 provides brilliant perspective of future models for scholarly communication!

2017-11-23 19:57 UTC

2 people recommend this

Comment on this article

scite_

Cited by 14

See all cited by

- Version 1

A multi-disciplinary perspective on emergent and future innovations in peer review

Read this article at

Abstract

Related collections

Open Research, Open Science, Open Scholarship

Most cited references 230

The Matthew Effect in Science: The reward and communication systems of science are considered.

Where Is Current Research on Blockchain Technology?—A Systematic Review

The Oligopoly of Academic Publishers in the Digital Era

Author and article information

Contributors

Journal

Affiliations

Author notes

Article

History

Funding

Categories

Comments

Comment on this article

Similar content 183

Cited by 14