7
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Dyslexia treatment studies: A systematic review and suggestions on testing treatment efficacy with small effects and small samples

      research-article

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Poor response to treatment is a defining characteristic of reading disorder. In the present systematic review and meta-analysis, we found that the overall average effect size for treatment efficacy was modest, with a mean standardized difference of 0.38. Small true effects, combined with the difficulty to recruit large samples, seriously challenge researchers planning to test treatment efficacy in dyslexia and potentially in other learning disorders. Nonetheless, most published studies claim effectiveness, generally based on liberal use of multiple testing. This inflates the risk that most statistically significant results are associated with overestimated effect sizes. To enhance power, we propose the strategic use of repeated measurements with mixed-effects modelling. This novel approach would enable us to estimate both individual parameters and population-level effects more reliably. We suggest assessing a reading outcome not once, but three times, at pre-treatment and three times at post-treatment. Such design would require only modest additional efforts compared to current practices. Based on this, we performed ad hoc a priori design analyses via simulation studies. Results showed that using the novel design may allow one to reach adequate power even with low sample sizes of 30–40 participants (i.e., 15–20 participants per group) for a typical effect size of d = 0.38. Nonetheless, more conservative assumptions are warranted for various reasons, including a high risk of publication bias in the extant literature. Our considerations can be extended to intervention studies of other types of neurodevelopmental disorders.

          Supplementary Information

          The online version contains supplementary material available at 10.3758/s13428-021-01549-x.

          Related collections

          Most cited references35

          • Record: found
          • Abstract: not found
          • Article: not found

          Fitting Linear Mixed-Effects Models Usinglme4

            Bookmark
            • Record: found
            • Abstract: not found
            • Article: not found

            brms: An R Package for Bayesian Multilevel Models Using Stan

              Bookmark
              • Record: found
              • Abstract: found
              • Article: not found

              Power failure: why small sample size undermines the reliability of neuroscience.

              A study with low statistical power has a reduced chance of detecting a true effect, but it is less well appreciated that low power also reduces the likelihood that a statistically significant result reflects a true effect. Here, we show that the average statistical power of studies in the neurosciences is very low. The consequences of this include overestimates of effect size and low reproducibility of results. There are also ethical dimensions to this problem, as unreliable research is inefficient and wasteful. Improving reproducibility in neuroscience is a key priority and requires attention to well-established but often ignored methodological principles.
                Bookmark

                Author and article information

                Contributors
                enrico.toffalini@unipd.it
                Journal
                Behav Res Methods
                Behav Res Methods
                Behavior Research Methods
                Springer US (New York )
                1554-351X
                1554-3528
                10 March 2021
                10 March 2021
                2021
                : 53
                : 5
                : 1954-1972
                Affiliations
                [1 ]GRID grid.5608.b, ISNI 0000 0004 1757 3470, Department of General Psychology, , University of Padua, ; Via Venezia, 8, 35131 Padua, PD Italy
                [2 ]GRID grid.5606.5, ISNI 0000 0001 2151 3065, DISFOR, , University of Genoa, ; Genova, Italy
                [3 ]GRID grid.5608.b, ISNI 0000 0004 1757 3470, Department of Developmental Psychology and Socialisation, , University of Padua, ; Padova, Italy
                [4 ]GRID grid.5335.0, ISNI 0000000121885934, Department of Psychology, Centre for Neuroscience in Education, , University of Cambridge, ; Cambridge, UK
                Article
                1549
                10.3758/s13428-021-01549-x
                8516770
                33694077
                d3fe96eb-f661-446a-a141-f7b70149dfe3
                © The Author(s) 2021

                Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

                History
                : 20 January 2021
                Funding
                Funded by: Università degli Studi di Padova
                Categories
                Article
                Custom metadata
                © The Psychonomic Society, Inc. 2021

                Clinical Psychology & Psychiatry
                dyslexia,treatment efficacy,design analysis,meta-analysis,mixed-effects modelling

                Comments

                Comment on this article