Blog
About

  • Record: found
  • Abstract: found
  • Article: found
Is Open Access

Using Bayes to get the most out of non-significant results

Read this article at

Bookmark
      There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

      Abstract

      No scientific conclusion follows automatically from a statistically non-significant result, yet people routinely use non-significant results to guide conclusions about the status of theories (or the effectiveness of practices). To know whether a non-significant result counts against a theory, or if it just indicates data insensitivity, researchers must use one of: power, intervals (such as confidence or credibility intervals), or else an indicator of the relative evidence for one theory over another, such as a Bayes factor. I argue Bayes factors allow theory to be linked to data in a way that overcomes the weaknesses of the other approaches. Specifically, Bayes factors use the data themselves to determine their sensitivity in distinguishing theories (unlike power), and they make use of those aspects of a theory’s predictions that are often easiest to specify (unlike power and intervals, which require specifying the minimal interesting value in order to address theory). Bayes factors provide a coherent approach to determining whether non-significant results support a null hypothesis over a theory, or whether the data are just insensitive. They allow accepting and rejecting the null hypothesis to be put on an equal footing. Concrete examples are provided to indicate the range of application of a simple online Bayes calculator, which reveal both the strengths and weaknesses of Bayes factors.

      Related collections

      Most cited references 78

      • Record: found
      • Abstract: found
      • Article: not found

      Statistical power analyses using G*Power 3.1: tests for correlation and regression analyses.

      G*Power is a free power analysis program for a variety of statistical tests. We present extensions and improvements of the version introduced by Faul, Erdfelder, Lang, and Buchner (2007) in the domain of correlation and regression analyses. In the new version, we have added procedures to analyze the power of tests based on (1) single-sample tetrachoric correlations, (2) comparisons of dependent correlations, (3) bivariate linear regression, (4) multiple linear regression based on the random predictor model, (5) logistic regression, and (6) Poisson regression. We describe these new features and provide a brief introduction to their scope and handling.
        Bookmark
        • Record: found
        • Abstract: found
        • Article: not found

        Power failure: why small sample size undermines the reliability of neuroscience.

        A study with low statistical power has a reduced chance of detecting a true effect, but it is less well appreciated that low power also reduces the likelihood that a statistically significant result reflects a true effect. Here, we show that the average statistical power of studies in the neurosciences is very low. The consequences of this include overestimates of effect size and low reproducibility of results. There are also ethical dimensions to this problem, as unreliable research is inefficient and wasteful. Improving reproducibility in neuroscience is a key priority and requires attention to well-established but often ignored methodological principles.
          Bookmark
          • Record: found
          • Abstract: found
          • Article: not found

          Bayesian t tests for accepting and rejecting the null hypothesis.

          Progress in science often comes from discovering invariances in relationships among variables; these invariances often correspond to null hypotheses. As is commonly known, it is not possible to state evidence for the null hypothesis in conventional significance testing. Here we highlight a Bayes factor alternative to the conventional t test that will allow researchers to express preference for either the null hypothesis or the alternative. The Bayes factor has a natural and straightforward interpretation, is based on reasonable assumptions, and has better properties than other methods of inference that have been advocated in the psychological literature. To facilitate use of the Bayes factor, we provide an easy-to-use, Web-based program that performs the necessary calculations.
            Bookmark

            Author and article information

            Affiliations
            School of Psychology and Sackler Centre for Consciousness Science, University of Sussex Brighton, UK
            Author notes

            Edited by: Prathiba Natesan, University of North Texas, USA

            Reviewed by: Clintin P. Davis-Stober, University of Missouri, USA; Rink Hoekstra, University of Groningen, Netherlands

            *Correspondence: Zoltan Dienes, School of Psychology and Sackler Centre for Consciousness Science, University of Sussex, Brighton BN1 9QH, UK e-mail: dienes@ 123456sussex.ac.uk

            This article was submitted to Quantitative Psychology and Measurement, a section of the journal Frontiers in Psychology.

            Contributors
            Journal
            Front Psychol
            Front Psychol
            Front. Psychol.
            Frontiers in Psychology
            Frontiers Media S.A.
            1664-1078
            29 July 2014
            2014
            : 5
            4114196 10.3389/fpsyg.2014.00781
            Copyright © 2014 Dienes.

            This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

            Counts
            Figures: 3, Tables: 3, Equations: 0, References: 115, Pages: 17, Words: 0
            Categories
            Psychology
            Methods Article

            Comments

            Comment on this article