33
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Changing how we evaluate research is difficult, but not impossible

      discussion
      1 , ,   2 ,
      eLife
      eLife Sciences Publications, Ltd
      research assessment, research culture, culture change, institutional change, careers in science, None

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          The San Francisco Declaration on Research Assessment (DORA) was published in 2013 and described how funding agencies, institutions, publishers, organizations that supply metrics, and individual researchers could better evaluate the outputs of scientific research. Since then DORA has evolved into an active initiative that gives practical advice to institutions on new ways to assess and evaluate research. This article outlines a framework for driving institutional change that was developed at a meeting convened by DORA and the Howard Hughes Medical Institute. The framework has four broad goals: understanding the obstacles to changes in the way research is assessed; experimenting with different approaches; creating a shared vision when revising existing policies and practices; and communicating that vision on campus and beyond.

          Related collections

          Most cited references32

          • Record: found
          • Abstract: found
          • Article: not found

          Decoupling of the minority PhD talent pool and assistant professor hiring in medical school basic science departments in the US

          Faculty diversity is a longstanding challenge in the US. However, we lack a quantitative and systemic understanding of how the career transitions into assistant professor positions of PhD scientists from underrepresented minority (URM) and well-represented (WR) racial/ethnic backgrounds compare. Between 1980 and 2013, the number of PhD graduates from URM backgrounds increased by a factor of 9.3, compared with a 2.6-fold increase in the number of PhD graduates from WR groups. However, the number of scientists from URM backgrounds hired as assistant professors in medical school basic science departments was not related to the number of potential candidates (R2=0.12, p>0.07), whereas there was a strong correlation between these two numbers for scientists from WR backgrounds (R2=0.48, p<0.0001). We built and validated a conceptual system dynamics model based on these data that explained 79% of the variance in the hiring of assistant professors and posited no hiring discrimination. Simulations show that, given current transition rates of scientists from URM backgrounds to faculty positions, faculty diversity would not increase significantly through the year 2080 even in the context of an exponential growth in the population of PhD graduates from URM backgrounds, or significant increases in the number of faculty positions. Instead, the simulations showed that diversity increased as more postdoctoral candidates from URM backgrounds transitioned onto the market and were hired. DOI: http://dx.doi.org/10.7554/eLife.21393.001
            Bookmark
            • Record: found
            • Abstract: found
            • Article: found
            Is Open Access

            Use of the Journal Impact Factor in academic review, promotion, and tenure evaluations

            We analyzed how often and in what ways the Journal Impact Factor (JIF) is currently used in review, promotion, and tenure (RPT) documents of a representative sample of universities from the United States and Canada. 40% of research-intensive institutions and 18% of master’s institutions mentioned the JIF, or closely related terms. Of the institutions that mentioned the JIF, 87% supported its use in at least one of their RPT documents, 13% expressed caution about its use, and none heavily criticized it or prohibited its use. Furthermore, 63% of institutions that mentioned the JIF associated the metric with quality, 40% with impact, importance, or significance, and 20% with prestige, reputation, or status. We conclude that use of the JIF is encouraged in RPT evaluations, especially at research-intensive universities, and that there is work to be done to avoid the potential misuse of metrics like the JIF.
              Bookmark
              • Record: found
              • Abstract: not found
              • Article: not found

              “Excellence R Us”: university research and the fetishisation of excellence

                Bookmark

                Author and article information

                Contributors
                Journal
                eLife
                Elife
                eLife
                eLife
                eLife Sciences Publications, Ltd
                2050-084X
                12 August 2020
                2020
                : 9
                : e58654
                Affiliations
                [1 ]DORA RockvilleUnited States
                [2 ]Imperial College LondonUnited Kingdom
                Author information
                https://orcid.org/0000-0002-2111-3237
                https://orcid.org/0000-0002-0552-8870
                Article
                58654
                10.7554/eLife.58654
                7423335
                32782065
                4dfb846d-127a-4245-8bde-c4f5e75c9759
                © 2020, Hatch and Curry

                This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.

                History
                : 07 May 2020
                : 06 August 2020
                Funding
                No external funding was received for this work.
                Categories
                Feature Article
                Research Culture
                Custom metadata
                The DORA initiative offers practical guidance on improving the assessment of research by universities, research institutions, and funders.
                4

                Life sciences
                research assessment,research culture,culture change,institutional change,careers in science,none

                Comments

                Comment on this article