Blog
About

  • Record: found
  • Abstract: found
  • Article: found
Is Open Access

Do altmetrics correlate with the quality of papers? A large-scale empirical study based on F1000Prime data

1 , * , 2

PLoS ONE

Public Library of Science

Read this article at

Bookmark
      There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

      Abstract

      In this study, we address the question whether (and to what extent, respectively) altmetrics are related to the scientific quality of papers (as measured by peer assessments). Only a few studies have previously investigated the relationship between altmetrics and assessments by peers. In the first step, we analyse the underlying dimensions of measurement for traditional metrics (citation counts) and altmetrics–by using principal component analysis (PCA) and factor analysis (FA). In the second step, we test the relationship between the dimensions and quality of papers (as measured by the post-publication peer-review system of F1000Prime assessments)–using regression analysis. The results of the PCA and FA show that altmetrics operate along different dimensions, whereas Mendeley counts are related to citation counts, and tweets form a separate dimension. The results of the regression analysis indicate that citation-based metrics and readership counts are significantly more related to quality, than tweets. This result on the one hand questions the use of Twitter counts for research evaluation purposes and on the other hand indicates potential use of Mendeley reader counts.

      Related collections

      Most cited references 24

      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      How well developed are altmetrics? A cross-disciplinary analysis of the presence of 'alternative metrics' in scientific publications

      In this paper an analysis of the presence and possibilities of altmetrics for bibliometric and performance analysis is carried out. Using the web based tool Impact Story, we collected metrics for 20,000 random publications from the Web of Science. We studied both the presence and distribution of altmetrics in the set of publications, across fields, document types and over publication years, as well as the extent to which altmetrics correlate with citation indicators. The main result of the study is that the altmetrics source that provides the most metrics is Mendeley, with metrics on readerships for 62.6% of all the publications studied, other sources only provide marginal information. In terms of relation with citations, a moderate spearman correlation (r=0.49) has been found between Mendeley readership counts and citation indicators. Other possibilities and limitations of these indicators are discussed and future research lines are outlined.
        Bookmark
        • Record: found
        • Abstract: found
        • Article: not found

        On exploratory factor analysis: a review of recent evidence, an assessment of current practice, and recommendations for future use.

        Exploratory factor analysis (hereafter, factor analysis) is a complex statistical method that is integral to many fields of research. Using factor analysis requires researchers to make several decisions, each of which affects the solutions generated. In this paper, we focus on five major decisions that are made in conducting factor analysis: (i) establishing how large the sample needs to be, (ii) choosing between factor analysis and principal components analysis, (iii) determining the number of factors to retain, (iv) selecting a method of data extraction, and (v) deciding upon the methods of factor rotation. The purpose of this paper is threefold: (i) to review the literature with respect to these five decisions, (ii) to assess current practices in nursing research, and (iii) to offer recommendations for future use. The literature reviews illustrate that factor analysis remains a dynamic field of study, with recent research having practical implications for those who use this statistical method. The assessment was conducted on 54 factor analysis (and principal components analysis) solutions presented in the results sections of 28 papers published in the 2012 volumes of the 10 highest ranked nursing journals, based on their 5-year impact factors. The main findings from the assessment were that researchers commonly used (a) participants-to-items ratios for determining sample sizes (used for 43% of solutions), (b) principal components analysis (61%) rather than factor analysis (39%), (c) the eigenvalues greater than one rule and screen tests to decide upon the numbers of factors/components to retain (61% and 46%, respectively), (d) principal components analysis and unweighted least squares as methods of data extraction (61% and 19%, respectively), and (e) the Varimax method of rotation (44%). In general, well-established, but out-dated, heuristics and practices informed decision making with respect to the performance of factor analysis in nursing studies. Based on the findings from factor analysis research, it seems likely that the use of such methods may have had a material, adverse effect on the solutions generated. We offer recommendations for future practice with respect to each of the five decisions discussed in this paper. Crown Copyright © 2013. Published by Elsevier Ltd. All rights reserved.
          Bookmark
          • Record: found
          • Abstract: not found
          • Article: not found

          Do altmetrics point to the broader impact of research? An overview of benefits and disadvantages of altmetrics

           Lutz Bornmann (2014)
            Bookmark

            Author and article information

            Affiliations
            [1 ] Division for Science and Innovation Studies, Administrative Headquarters of the Max Planck Society, Munich, Germany
            [2 ] Max Planck Institute for Solid State Research, Stuttgart, Germany
            Universidad de las Palmas de Gran Canaria, SPAIN
            Author notes

            Competing Interests: The authors have declared that no competing interests exist.

            Contributors
            ORCID: http://orcid.org/0000-0003-0810-7091, Role: Conceptualization, Role: Formal analysis, Role: Investigation, Role: Methodology, Role: Validation, Role: Writing – original draft, Role: Writing – review & editing
            Role: Conceptualization, Role: Data curation, Role: Writing – original draft, Role: Writing – review & editing
            Role: Editor
            Journal
            PLoS One
            PLoS ONE
            plos
            plosone
            PLoS ONE
            Public Library of Science (San Francisco, CA USA )
            1932-6203
            23 May 2018
            2018
            : 13
            : 5
            29791468
            5965816
            10.1371/journal.pone.0197133
            PONE-D-18-05457
            (Editor)
            © 2018 Bornmann, Haunschild

            This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

            Counts
            Figures: 0, Tables: 6, Pages: 12
            Product
            Funding
            The authors received no specific funding for this work.
            Categories
            Research Article
            Research and Analysis Methods
            Research Assessment
            Altmetrics
            Research and Analysis Methods
            Mathematical and Statistical Techniques
            Statistical Methods
            Multivariate Analysis
            Principal Component Analysis
            Physical Sciences
            Mathematics
            Statistics (Mathematics)
            Statistical Methods
            Multivariate Analysis
            Principal Component Analysis
            Research and Analysis Methods
            Research Assessment
            Citation Analysis
            Social Sciences
            Sociology
            Communications
            Social Communication
            Social Media
            Twitter
            Computer and Information Sciences
            Network Analysis
            Social Networks
            Social Media
            Twitter
            Social Sciences
            Sociology
            Social Networks
            Social Media
            Twitter
            Research and Analysis Methods
            Research Assessment
            Bibliometrics
            Research and Analysis Methods
            Mathematical and Statistical Techniques
            Statistical Methods
            Factor Analysis
            Physical Sciences
            Mathematics
            Statistics (Mathematics)
            Statistical Methods
            Factor Analysis
            Research and Analysis Methods
            Mathematical and Statistical Techniques
            Statistical Methods
            Regression Analysis
            Physical Sciences
            Mathematics
            Statistics (Mathematics)
            Statistical Methods
            Regression Analysis
            Research and Analysis Methods
            Research Assessment
            Research Quality Assessment
            Custom metadata
            The complete dataset excluding DOIs have been made available at https://doi.org/10.6084/m9.figshare.6120158.v1.

            Uncategorized

            Comments

            Comment on this article