Online-consultation “ scientific publication system ” : documentation and main results

This short report provides a description of an online-consultation on the scientific publication system. German-speaking scientists from all disciplines were invited to articulate their perspectives on principles and current problems in scientific publishing in the dialogical procedure. 697 participants addressed their opinion in two areas of consultation (a) Consultation area “evaluate principles”: the goal in this section was to find out whether there is a general consensus throughout academia of what constitutes a good publication system. For this purpose, principles of a good scientific publication system could be commented on and evaluated with positive or negative votes. (b) Consultation area “name problems”: this section aimed at obtaining the perspective of the participants on current challenges and problems of the publication system. The contributions of the participants focus on eight topics: (1) printed vs. digital publication, (2) business models of large publishing houses, (3) open access, (4) publication-based performance indicators, (5) authorship, (6) peer review, (7) publication bias, and (8)


INTRODUCTION
The objective of the interdisciplinary working group (IAG) "Future of the scientific communication system" of the Berlin-Brandenburg Academy of Sciences is to develop recommendations on the future of scientific publishing.In this context, four dynamics that currently affect the scientific communication system, and which are usually looked at separately, should be taken into account: (1) the diverse influences of digitalization processes, (2) the increasing indicator-based observation of publication activities by means of bibliometric and usage-based metrics, (3) the economic orientation of scientific publishers, in particular in the fields of science, technology, and medicine (STM), as well as (4) the repercussions which result from the coverage of science by mass media.In the course of the development of recommendations, the perspectives of three groups of actors are taken into account: The working group has become familiar with the perspectives of scientific publishers and libraries in three rounds of interviews.The perspective of the most important groupthe scientistswas obtained via interviews with representatives from different disciplines and the online-consultation, and was included in the development of recommendations.In view of the heterogeneity of the requirements for scientific publishing, the different traditions of communication and the different media, the objective of the online-consultation is to mobilize the practical knowledge that already exists in different disciplines in as broad a manner as possible.The procedure is supposed to bundle a diverse range of opinions and perspectives from different disciplines on current problems and challenges as well as suggestions for improvement and to inform the process of developing recommendations.This summary provides a short description of the procedure and gives an overview over the main results of the onlineconsultation.A more detailed analysis and evaluation of all contributions are also available. 1The summary and the full report are limited to the presentation of the perspectives and arguments that were collected in the course of the onlineconsultation.Both do not provide a final assessment or extensive conclusions.This limitation served to clearly differentiate the contributions of the participants of the online-consultation from the further work of the IAG in the development of recommendations.

TOPIC OF THE ONLINE-CONSULTATION
In the online-consultation, German-speaking scientists were invited to articulate their perspectives on problems and suggestions for improvement with respect to the publication system of science.According to our understanding, the "publication system" comprises the following elements: (1) All internal processes within science of formal distribution of research results in scientific communities, thus a network of communication which is often referred to as the (formal) scientific communication system.
(2) Publication infrastructure: the technical components that are utilized in this context, media of publication (e.g.journals, monographs, anthologies, conference proceedings, and literature reviews) as well as components which serve the production and dissemination of publications (e.g.journal databases, online repositories, library classification systems, abstracting and review databases, search engines, citation databases, and online editorial management systems).(3) All organizations which provide and maintain the abovementioned publication infrastructure as well as ensure their functionality (libraries, scientific publishing houses, but also research and service institutions and learned societies).

METHODS: MAIN CHARACTERISTICS OF THE PROCEDURE
The addressees of the online-consultation were: (1) Germanspeaking scientists at universities, research organizations, and privately funded research institutes.In order to invite these addressees to participate in the online-consultation, we asked research institutions in Germany and German-speaking learned societies to forward an invitation email to their members.Overall, 204 organizations supported the online-consultation and disseminated the invitation.The participants were (2) asked to include the experiences they have made with respect to the publication system in the process of developing recommendations.To allow unpredicted perspectives, the procedure (3) was designed as open and weakly structured.
In contrast to a questionnaire, it was not asked about individual dimensions of the topic via items.Instead, a space of communication was created in which the aspects could be addressed by the participants according to their perceived relevance.The objective was not a statistical representativeness but the collection of a range of opinions that was as broad as possible without claiming to be complete.The (4) dialogical structure of the consultation allowed the participants to react to the contributions of the other participants.
In order to enable the participants to address their perspective, two areas of consultation were created.(a) Consultation area "evaluate principles": the goal was to find out whether there is a general consensus throughout academia of what constitutes a good publication system.For this purpose, the IAG developed principles of a good scientific publication system for discussion.These principles could be commented on and evaluated with positive or negative votes.(b) Consultation area "name problems": the perspective of the participants on current challenges and problems of the publication system was supposed to be obtained.Such problems could then refer to science as a wholeas, for example, to general conditions of copyright lawas well as to individual areas and fields of research.In this second consultation area, participants had the opportunity to write contributions that described problems.These descriptions could also be commented on and evaluated by other participants.

PARTICIPATION AND RESPONSE RATE
Overall, 697 people participated in the online-consultation. 2 The large majority of 651 chose the role of registered user and a small group of 46 people chose the role of guest.The majority of the registered participants (542 people, 83.3% of registered participants, or 77.8% of all participants) have voluntarily provided six items of personal information in the course of the registration.These data allow for an assessment of the composition of the participants.A strong degree of diversity could be achieved in the dimensions "age," "highest qualification," "position," and (to some extent) also "discipline."3The distribution in the dimensions "gender" and "type of research organization," however, is less diverse.
( In the succeeding interpretation of the results, aside from the strong representation of the Humanities and Social Sciences, it has to be taken into account that the results rather reveal perspectives of males and members of universities.In total, the participants have provided 124 descriptions of problems and challenges, 527 comments and 2,884 ratings/assessments, and thus valuable input for the development of recommendations.
RESULTS 1: CONSULTATION AREA "EVALUATE PRINCIPLES" In the consultation area, "evaluate principles" the IAG provided six basic sets of rules for a good scientific publication system for discussion.The online-consultation brought the following results: Principle 1: freedom of scientific exchange The scientific publication system should support the free exchange of research results and scientific knowledge.Access barriers should be as low as possible, so that every interested person can participate This principle was mainly approved and there were only few comments suggesting need for improvement: there is need of clarification with respect to the role (the reader, reviewer, or author) in which participation should be possible.Moreover, there was a controversial discussion about what is meant by "as low as possible access barriers" and how this can be realized.This principle was mainly approved.There were, however, a significant amount of negative votes and a number of comments indicating that there is need for revision in order to gain approval for the principle in science as a whole.The need for clarification first of all refers to the entity that regulates the exchange, which has so far been not always adequately termed "discipline" and "field."Second, it is questioned on which aspect of exchange self-regulation should refer to.In the case of quality, this is quite clear, with regard to relevance, however, it is not.Third, it should be clarified which processes are precisely meant by "self-regulation." Principle 3: choice of the medium The choice/decision to publish as well as the choice of the appropriate medium of publication should lie exclusively in the hands of those responsible for the research results.Prerequisites are a plurality of publication media and the absence of directives that would force scientists to publish in a particular medium N = 203 This principle is formulated in a clear manner.The majority of the comments and a significant proportion of negative votes indicate that the principle in this form is not shared by all.Controversial is the question whether the principle of choice should be limited by an obligation to publish open access.In this context, there is a broad range of opinions in which the freedom to choose and free accessibility are emphasized to different degrees.In the case of restrictions regarding the principle of free choice, it should be made sure that the acquisition of reputation is not affected.

Principle 4: sustainability
Regarding publications, access should be permanent and as open as possible.This requires a reliable archiving of publications.Second, with respect to media of publication, this means that the operation is permanently ensured in terms of resources and organization and that it is possible to adapt to changing requirements N = 214 The principle of sustainability received the most approval, but its name is criticized.Aside from that, the discussion focuses on the realization of this principle.
Principle 5: transparency of funding The scientific publication system is largely financed by public funds.The receptionists of the financial resources thus have to document their extent and usage.This concerns all receptionists and, aside from researchers, includes libraries, publishing houses, learned societies and research organizations The second largest amount of negative votes and the critical comments indicate that there is need for revision: first, it needs to be clarified what transparency actually refers to here.Second, the call for transparency has to be weighed against the amount of documentation that it entails.Third, the generality of the principle is criticized.It is also pointed out that the government has to provide justification to the taxpayer, but enterprises do not have to provide justification to the government.Therefore, transparency cannot refer to the "usage" of the funds by the receptionists.
Principle 6: efficiency of resources Resources for the scientific publication system should be used sparingly.It should only take advantage of as many resources as are necessary for success.Efficiency of resources refers to monetary resources necessary for operation as well as to the resource "time" that is voluntarily provided by the researchers in their roles as authors, editors, readers, and reviewers A significant amount of the participants reject this principle: it is mainly criticized for its vagueness, the imprecise definition of the contents and the orientation toward efficiency on which the principle is based.A large number of comments indicated that there are unnecessary work phases during the production process of publications, thus pointing out a potential direction for a revision of the principle.
RESULTS 2: CONSULTATION AREA "NAME PROBLEMS" The 124 problems that were discussed in the second consultation area cover a broad thematic spectrum.A large amount of the contributions can be attributed to one of the following eight fields: (1) A smaller field concerns the basic technologies of the media, the print and digital technologies.Among the participants, there are advocates for both technologies whereby their positions strongly correlate with the field in which they work.In the Natural and Life Sciences, there seems to be a preference toward electronic publishing.In the humanities, however, there are proponents of digital publishing as well as in print.The discussion shows that these preferences are closely connected to how the participants are used to accessing or receiving publications as well as to different perspectives regarding longterm archiving.The issues of what is regarded as good accessibility or which type of medium stands for good quality also play a role.
(2) The business models and practices of the large publishing houses in STM form a larger field and are criticized in several contributions.The critique refers to the costs of journal subscriptions, the return rates, and structural problems regarding the market for scientific publications.The participants report about problems of accessibility as an immediate result of higher prices.This is especially apparent at locations with less financial resources.Moreover, other interested people like specialists, the public and journalists are partly excluded from access to publications.The business models are also problematic in the Humanities and Social Sciences since the license fees in STM bind large amounts of the library budgets, and this has a negative effect on the supply of monographs and anthologies.The participants' suggestions for solving this problem all aim at reducing the power of large publishing houses and call for structural change.The complexity of the discussion is to some extent due to the fact that three fields of application were deliberated: The use of publication-based indicators in the performance-based allocation of funds, in the context of recruitment procedures and in the evaluation of project proposals.(5) Authorship is a smaller field of discussion.The central question is according to what rules authorship should be granted.The discussion revealed four types of perspectives: a "writing," "exclusive," "inclusive," and "documenting" understanding of "authorship."It is noteworthy that the discussion is mainly about a fair recognition of achievement while other aspects (like authors' responsibility) are not taken into account at all.This could be an effect of the use of publication-based performance measurement and the increased importance of (first) authorship.(6) The contributions in the field peer review mainly focus on two issues: First, problems in the current practice of review processes of journals.The participants mention qualitative problems of reviews, which are partly considered to be the result of the overall increase of publications and carelessness among the reviewers.Second, proposals on the reorganization of the procedure on the basis of digital technologies.The digital connectivity of those involved in the review process is viewed as an opportunity to experiment with new types such as open peer review and open discussion.The objective is to improve traditional procedures like single blind/double blind peer review).( 7) Contributions in the field publication bias refer to problems that mainly occur in empirical research where hypotheses are tested: research results that do not confirm a hypothesis are less likely to be published than results that show a correlation or prove an effect.To solve this problem, different measures are considered, such as the introduction of a review procedure that consists of two phases, or the establishment of a second layer in the publication system for the publication of null results that are frequently rejected by journals.(8) Research data are a smaller field of discussion.Aside from the advantages of accessibility to research resultssuch as improving comprehensibility and increasing trust in publicationsemphasis is put on future tasks of the development of a corresponding infrastructure: in addition to the creation of longterm financed research databases, the development of routines in archiving and the establishment of standards, the necessity of development processes within science is pointed out.The willingness to publish research data depends in many cases on corresponding mechanisms that acknowledge the provision of data.

OUTLOOK
Overall, the participants of the online-consultation have provided the IAG with valuable input for the process of developing recommendations.Therefore, the results of the experiment with this new tool in science policy are basically considered to be positive: The discussion of the principles for a good publication system in the first part of the consultation provides a large number of points that can be used for making the principles more precise or for revising them.The contributions in the second part, on the other hand, provide a broad overview of current problems and challenges that have resulted from the dynamic development of the publication system.This overview can be regarded as the most important result of the online-consultation and helps the IAG to recognize the fundamental problems and their connections.
In the course of the development of recommendations, the results of the online-consultation were and will continue to be taken into account in the IAG's deliberations.The development of recommendations is completed and the publication of the recommendations is scheduled for the May 26, 2015.

Principle 2 :
self-regulationThe exchange of research results should be determined by criteria of the respective discipline or field.The quality of a contribution should solely be defined through science and not influenced by other factorssuch as public perception or monetary factors N = 229 (3) The field open access bears strong reference to (2).The majority of contributions favors the free accessibility to publications and provides two types of arguments in support of it: (1) in terms of fairness, publicly funded research should also be publicly accessible.(2) In terms of utility, open access has the potential to increase the dynamic of the development of science and to improve the transfer of knowledge in fields of practice.In addition, advantages and disadvantages of the different forms of open access (Green and Gold) are discussed.It is noteworthy that especially the Gold Open Access model, which is financed by publication fees, is criticized in a number of comments.(4) Contributions that refer to the field publicationbased performance indicators are basically critical or negative.Aside from methodological critique, the way how publication-based performance indicators are used is criticized, as is the problematization of unintended consequences.The participants complain that the pressure to publish lead to strong incentives to publish research results separately in as many publications as possible, thus leading to an increase in the number of overall publications and to a decrease in quality.There is a broad spectrum of opinions on what role publication-based performance indicators should play in the future.These range from fundamental criticism to calls for reform.