949
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Small is beautiful: In defense of the small- N design

      research-article
      ,
      Psychonomic Bulletin & Review
      Springer US
      Methodology, Replication, Inference, Mathematical psychology

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          The dominant paradigm for inference in psychology is a null-hypothesis significance testing one. Recently, the foundations of this paradigm have been shaken by several notable replication failures. One recommendation to remedy the replication crisis is to collect larger samples of participants. We argue that this recommendation misses a critical point, which is that increasing sample size will not remedy psychology’s lack of strong measurement, lack of strong theories and models, and lack of effective experimental control over error variance. In contrast, there is a long history of research in psychology employing small- N designs that treats the individual participant as the replication unit, which addresses each of these failings, and which produces results that are robust and readily replicated. We illustrate the properties of small- N and large- N designs using a simulated paradigm investigating the stage structure of response times. Our simulations highlight the high power and inferential validity of the small- N design, in contrast to the lower power and inferential indeterminacy of the large- N design. We argue that, if psychology is to be a mature quantitative science, then its primary theoretical aim should be to investigate systematic, functional relationships as they are manifested at the individual participant level and that, wherever possible, it should use methods that are optimized to identify relationships of this kind.

          Related collections

          Most cited references95

          • Record: found
          • Abstract: found
          • Article: not found

          The time course of perceptual choice: the leaky, competing accumulator model.

          The time course of perceptual choice is discussed in a model of gradual, leaky, stochastic, and competitive information accumulation in nonlinear decision units. Special cases of the model match a classical diffusion process, but leakage and competition work together to address several challenges to existing diffusion, random walk, and accumulator models. The model accounts for data from choice tasks using both time-controlled (e.g., response signal) and standard reaction time paradigms and its adequacy compares favorably with other approaches. A new paradigm that controls the time of arrival of information supporting different choice alternatives provides further support. The model captures choice behavior regardless of the number of alternatives, accounting for the log-linear relation between reaction time and number of alternatives (Hick's law) and explains a complex pattern of visual and contextual priming in visual word identification.
            Bookmark
            • Record: found
            • Abstract: found
            • Article: not found

            Using confidence intervals in within-subject designs.

            We argue that to best comprehend many data sets, plotting judiciously selected sample statistics with associated confidence intervals can usefully supplement, or even replace, standard hypothesis-testing procedures. We note that most social science statistics textbooks limit discussion of confidence intervals to their use in between-subject designs. Our central purpose in this article is to describe how to compute an analogous confidence interval that can be used in within-subject designs. This confidence interval rests on the reasoning that because between-subject variance typically plays no role in statistical analyses of within-subject designs, it can legitimately be ignored; hence, an appropriate confidence interval can be based on the standard within-subject error term-that is, on the variability due to the subject × condition interaction. Computation of such a confidence interval is simple and is embodied in Equation 2 on p. 482 of this article. This confidence interval has two useful properties. First, it is based on the same error term as is the corresponding analysis of variance, and hence leads to comparable conclusions. Second, it is related by a known factor (√2) to a confidence interval of the difference between sample means; accordingly, it can be used to infer the faith one can put in some pattern of sample means as a reflection of the underlying pattern of population means. These two properties correspond to analogous properties of the more widely used between-subject confidence interval.
              Bookmark
              • Record: found
              • Abstract: found
              • Article: not found

              Inference by eye: confidence intervals and how to read pictures of data.

              Wider use in psychology of confidence intervals (CIs), especially as error bars in figures, is a desirable development. However, psychologists seldom use CIs and may not understand them well. The authors discuss the interpretation of figures with error bars and analyze the relationship between CIs and statistical significance testing. They propose 7 rules of eye to guide the inferential use of figures with error bars. These include general principles: Seek bars that relate directly to effects of interest, be sensitive to experimental design, and interpret the intervals. They also include guidelines for inferential interpretation of the overlap of CIs on independent group means. Wider use of interval estimation in psychology has the potential to improve research communication substantially. ((c) 2005 APA, all rights reserved).
                Bookmark

                Author and article information

                Contributors
                philipls@unimelb.edu.au
                daniel.little@unimelb.edu.au
                Journal
                Psychon Bull Rev
                Psychon Bull Rev
                Psychonomic Bulletin & Review
                Springer US (New York )
                1069-9384
                1531-5320
                19 March 2018
                19 March 2018
                2018
                : 25
                : 6
                : 2083-2101
                Affiliations
                ISNI 0000 0001 2179 088X, GRID grid.1008.9, The University of Melbourne, ; Melbourne, Australia
                Article
                1451
                10.3758/s13423-018-1451-8
                6267527
                29557067
                18b9e9cd-6eb0-4914-9f85-7065c5d1fef6
                © The Author(s) 2018

                Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

                History
                Categories
                Theoretical Review
                Custom metadata
                © Psychonomic Society, Inc. 2018

                Clinical Psychology & Psychiatry
                methodology,replication,inference,mathematical psychology
                Clinical Psychology & Psychiatry
                methodology, replication, inference, mathematical psychology

                Comments

                Comment on this article