11
views
0
recommends
+1 Recommend
1 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Questionable research practices in ecology and evolution

      research-article

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          We surveyed 807 researchers (494 ecologists and 313 evolutionary biologists) about their use of Questionable Research Practices (QRPs), including cherry picking statistically significant results, p hacking, and hypothesising after the results are known (HARKing). We also asked them to estimate the proportion of their colleagues that use each of these QRPs. Several of the QRPs were prevalent within the ecology and evolution research community. Across the two groups, we found 64% of surveyed researchers reported they had at least once failed to report results because they were not statistically significant (cherry picking); 42% had collected more data after inspecting whether results were statistically significant (a form of p hacking) and 51% had reported an unexpected finding as though it had been hypothesised from the start (HARKing). Such practices have been directly implicated in the low rates of reproducible results uncovered by recent large scale replication studies in psychology and other disciplines. The rates of QRPs found in this study are comparable with the rates seen in psychology, indicating that the reproducibility problems discovered in psychology are also likely to be present in ecology and evolution.

          Related collections

          Most cited references 24

          • Record: found
          • Abstract: found
          • Article: not found

          An Agenda for Purely Confirmatory Research.

          The veracity of substantive research claims hinges on the way experimental data are collected and analyzed. In this article, we discuss an uncomfortable fact that threatens the core of psychology's academic enterprise: almost without exception, psychologists do not commit themselves to a method of data analysis before they see the actual data. It then becomes tempting to fine tune the analysis to the data in order to obtain a desired result-a procedure that invalidates the interpretation of the common statistical tests. The extent of the fine tuning varies widely across experiments and experimenters but is almost impossible for reviewers and readers to gauge. To remedy the situation, we propose that researchers preregister their studies and indicate in advance the analyses they intend to conduct. Only these analyses deserve the label "confirmatory," and only for these analyses are the common statistical tests valid. Other analyses can be carried out but these should be labeled "exploratory." We illustrate our proposal with a confirmatory replication attempt of a study on extrasensory perception.
            Bookmark
            • Record: found
            • Abstract: found
            • Article: found
            Is Open Access

            “Positive” Results Increase Down the Hierarchy of the Sciences

            The hypothesis of a Hierarchy of the Sciences with physical sciences at the top, social sciences at the bottom, and biological sciences in-between is nearly 200 years old. This order is intuitive and reflected in many features of academic life, but whether it reflects the “hardness” of scientific research—i.e., the extent to which research questions and results are determined by data and theories as opposed to non-cognitive factors—is controversial. This study analysed 2434 papers published in all disciplines and that declared to have tested a hypothesis. It was determined how many papers reported a “positive” (full or partial) or “negative” support for the tested hypothesis. If the hierarchy hypothesis is correct, then researchers in “softer” sciences should have fewer constraints to their conscious and unconscious biases, and therefore report more positive outcomes. Results confirmed the predictions at all levels considered: discipline, domain and methodology broadly defined. Controlling for observed differences between pure and applied disciplines, and between papers testing one or several hypotheses, the odds of reporting a positive result were around 5 times higher among papers in the disciplines of Psychology and Psychiatry and Economics and Business compared to Space Science, 2.3 times higher in the domain of social sciences compared to the physical sciences, and 3.4 times higher in studies applying behavioural and social methodologies on people compared to physical and chemical studies on non-biological material. In all comparisons, biological studies had intermediate values. These results suggest that the nature of hypotheses tested and the logical and methodological rigour employed to test them vary systematically across disciplines and fields, depending on the complexity of the subject matter and possibly other factors (e.g., a field's level of historical and/or intellectual development). On the other hand, these results support the scientific status of the social sciences against claims that they are completely subjective, by showing that, when they adopt a scientific approach to discovery, they differ from the natural sciences only by a matter of degree.
              Bookmark
              • Record: found
              • Abstract: found
              • Article: not found

              Data archiving in ecology and evolution: best practices.

              Many ecology and evolution journals have recently adopted policies requiring that data from their papers be publicly archived. I present suggestions on how data generators, data re-users, and journals can maximize the fairness and scientific value of data archiving. Data should be archived with enough clarity and supporting information that they can be accurately interpreted by others. Re-users should respect their intellectual debt to the originators of data through citation both of the paper and of the data package. In addition, journals should consider requiring that all data for published papers be archived, just as DNA sequences must be deposited in GenBank. Data are another valuable part of the legacy of a scientific career and archiving them can lead to new scientific insights. Archiving also increases opportunities for credit to be given to the scientists who originally collected the data. Copyright © 2010 Elsevier Ltd. All rights reserved.
                Bookmark

                Author and article information

                Contributors
                Role: ConceptualizationRole: Data curationRole: Formal analysisRole: InvestigationRole: VisualizationRole: Writing – original draftRole: Writing – review & editing
                Role: ConceptualizationRole: InvestigationRole: MethodologyRole: Writing – review & editing
                Role: ConceptualizationRole: InvestigationRole: MethodologyRole: Writing – review & editing
                Role: ConceptualizationRole: Data curationRole: InvestigationRole: MethodologyRole: Writing – review & editing
                Role: ConceptualizationRole: Formal analysisRole: Funding acquisitionRole: InvestigationRole: MethodologyRole: Project administrationRole: ResourcesRole: SupervisionRole: Writing – original draftRole: Writing – review & editing
                Role: Editor
                Journal
                PLoS One
                PLoS ONE
                plos
                plosone
                PLoS ONE
                Public Library of Science (San Francisco, CA USA )
                1932-6203
                16 July 2018
                2018
                : 13
                : 7
                Affiliations
                [1 ] School of BioSciences, University of Melbourne, Parkville, VIC, Australia
                [2 ] Biology Department, Whitman College, Walla Walla, WA, United States of America
                [3 ] School of Biological, Earth and Environmental Sciences, University of New South Wales, Sydney, NSW, Australia
                [4 ] School of Historical and Philosophical Studies, University of Melbourne, Parkville, VIC, Australia
                Tilburg University, NETHERLANDS
                Author notes

                Competing Interests: The authors have declared that no competing interests exist.

                Article
                PONE-D-18-08573
                10.1371/journal.pone.0200303
                6047784
                30011289
                9f8db988-52e6-41fe-80a0-9a8b98e59f0e
                © 2018 Fraser et al

                This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

                Page count
                Figures: 2, Tables: 4, Pages: 16
                Product
                Funding
                Funded by: Australian Research Council Future Fellowship
                Award ID: FT150100297
                Award Recipient :
                Fiona Fidler is supported by an Australian Research Council Future Fellowship (FT150100297). T. Parker was supported by a sabbatical provided by Whitman College and was hosted by S. Griffith at Macquarie University.
                Categories
                Research Article
                Biology and Life Sciences
                Evolutionary Biology
                Biology and Life Sciences
                Ecology
                Evolutionary Ecology
                Ecology and Environmental Sciences
                Ecology
                Evolutionary Ecology
                Biology and Life Sciences
                Evolutionary Biology
                Evolutionary Ecology
                Biology and Life Sciences
                Psychology
                Social Sciences
                Psychology
                Physical Sciences
                Mathematics
                Statistics (Mathematics)
                Statistical Data
                Biology and Life Sciences
                Ecology
                Community Ecology
                Ecology and Environmental Sciences
                Ecology
                Community Ecology
                Science Policy
                Research Integrity
                Publication Ethics
                Biology and Life Sciences
                Behavior
                Animal Behavior
                Behavioral Ecology
                Biology and Life Sciences
                Zoology
                Animal Behavior
                Behavioral Ecology
                Biology and Life Sciences
                Ecology
                Behavioral Ecology
                Ecology and Environmental Sciences
                Ecology
                Behavioral Ecology
                Ecology and Environmental Sciences
                Custom metadata
                All quantitative data and code will be opely available on the Open Science Framework https://osf.io/qxt3u/, DOI: 10.17605/OSF.IO/QXT3U. Due to ethical constraints, only re-ordered qualitative information can be openly shared (in publicly available data there is no way to link these qualitative responses to any other answers given by participants). This data is sufficient to reproduce all of the results presented in this article. The full raw dataset is stored on the same OSF page but kept private. Authors will provide access for the purpose of validation and fraud detection.

                Uncategorized

                Comments

                Comment on this article