+1 Recommend
0 collections
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      The validity and reliability of a digital Ruff Figural Fluency Test (RFFT)


      Read this article at

          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.



          The Ruff Figural Fluency Test (RFFT) is a valid but time-consuming and labour-intensive cognitive paper-and-pencil test. A digital RFFT was developed that can be conducted independently using an iPad and Apple Pencil and RFFT scores are computed automatically. We investigated the validity and reliability of this digital RFFT.


          We randomly allocated participants to the digital or paper-and-pencil RFFT. After the first test, the other test was performed immediately (cross-over). Participants were invited for a second digital RFFT 1 week later. For the digital RFFT, an (automatic) algorithm and two independent raters (criterion standard) assessed the number of unique designs (UD) and perseverative errors (PE). These raters also assessed the paper-and-pencil RFFT. We used Intraclass correlation coefficients (ICC), sensitivity, specificity, %-agreement, Kappa, and Bland–Altman plots.


          We included 94 participants (mean (SD) age 39.9 (14.8), 73.4% follow-up). Mean (SD) UD and median (IQR) PE of the digital RFFT were 84.2 (26.0) and 4 (2–7.3), respectively. Agreement between manual and automatic scoring of the digital RFFT was high for UD (ICC = 0.99, 95% CI 0.98, 0.99, sensitivity = 0.98; specificity = 0.96) and PE (ICC = 0.99, 95% CI 0.98, 0.99; sensitivity = 0.90, specificity = 1.00), indicating excellent criterion validity. Small but significant differences in UD were found between the automatic and manual scoring (mean difference: − 1.12, 95% CI − 1.92, − 0.33). Digital and paper-and-pencil RFFT had moderate agreement for UD (ICC = 0.73, 95% CI 0.34, 0.87) and poor agreement for PE (ICC = 0.47, 95% CI 0.30, 0.62). Participants had fewer UD on the digital than paper-and-pencil RFFT (mean difference: − 7.09, 95% CI − 11.80, − 2.38). The number of UD on the digital RFFT was associated with higher education (Spearman’s r = 0.43, p < 0.001), and younger age (Pearson’s r = − 0.36, p < 0.001), showing its ability to discriminate between different age categories and levels of education. Test–retest reliability was moderate (ICC = 0.74, 95% CI 0.61, 0.83).


          The automatic scoring of the digital RFFT has good criterion and convergent validity. There was low agreement between the digital RFFT and paper-and-pencil RFFT and moderate test–retest reliability, which can be explained by learning effects. The digital RFFT is a valid and reliable instrument to measure executive cognitive function among the general population and is a feasible alternative to the paper-and-pencil RFFT in large-scale studies. However, its scores cannot be used interchangeably with the paper-and-pencil RFFT scores.

          Supplementary Information

          The online version contains supplementary material available at 10.1186/s40359-021-00566-x.

          Related collections

          Most cited references17

          • Record: found
          • Abstract: found
          • Article: not found

          A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research.

          Intraclass correlation coefficient (ICC) is a widely used reliability index in test-retest, intrarater, and interrater reliability analyses. This article introduces the basic concept of ICC in the content of reliability analysis.
            • Record: found
            • Abstract: found
            • Article: found

            Executive Functions

            Executive functions (EFs) make possible mentally playing with ideas; taking the time to think before acting; meeting novel, unanticipated challenges; resisting temptations; and staying focused. Core EFs are inhibition [response inhibition (self-control—resisting temptations and resisting acting impulsively) and interference control (selective attention and cognitive inhibition)], working memory, and cognitive flexibility (including creatively thinking “outside the box,” seeing anything from different perspectives, and quickly and flexibly adapting to changed circumstances). The developmental progression and representative measures of each are discussed. Controversies are addressed (e.g., the relation between EFs and fluid intelligence, self-regulation, executive attention, and effortful control, and the relation between working memory and inhibition and attention). The importance of social, emotional, and physical health for cognitive health is discussed because stress, lack of sleep, loneliness, or lack of exercise each impair EFs. That EFs are trainable and can be improved with practice is addressed, including diverse methods tried thus far.
              • Record: found
              • Abstract: found
              • Article: not found

              Toward defining the preclinical stages of Alzheimer's disease: recommendations from the National Institute on Aging-Alzheimer's Association workgroups on diagnostic guidelines for Alzheimer's disease.

              The pathophysiological process of Alzheimer's disease (AD) is thought to begin many years before the diagnosis of AD dementia. This long "preclinical" phase of AD would provide a critical opportunity for therapeutic intervention; however, we need to further elucidate the link between the pathological cascade of AD and the emergence of clinical symptoms. The National Institute on Aging and the Alzheimer's Association convened an international workgroup to review the biomarker, epidemiological, and neuropsychological evidence, and to develop recommendations to determine the factors which best predict the risk of progression from "normal" cognition to mild cognitive impairment and AD dementia. We propose a conceptual framework and operational research criteria, based on the prevailing scientific evidence to date, to test and refine these models with longitudinal clinical research studies. These recommendations are solely intended for research purposes and do not have any clinical implications at this time. It is hoped that these recommendations will provide a common rubric to advance the study of preclinical AD, and ultimately, aid the field in moving toward earlier intervention at a stage of AD when some disease-modifying therapies may be most efficacious. Copyright © 2011. Published by Elsevier Inc.

                Author and article information

                BMC Psychol
                BMC Psychol
                BMC Psychology
                BioMed Central (London )
                28 April 2021
                28 April 2021
                : 9
                : 65
                [1 ]GRID grid.4830.f, ISNI 0000 0004 0407 1981, University Medical Centre Groningen, Department of Epidemiology, , University of Groningen, ; Hanzeplein 1, FA40, P.O. Box 30 001, 9700 RB Groningen, The Netherlands
                [2 ]GRID grid.4830.f, ISNI 0000 0004 0407 1981, University Medical Center Groningen, Department of Internal Medicine, , University of Groningen, ; Groningen, The Netherlands
                © The Author(s) 2021

                Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

                : 2 December 2020
                : 13 April 2021
                Custom metadata
                © The Author(s) 2021

                cognition,cognitive dysfunction,executive function,neuropsychological test,ruff figural fluency test,software,reproducibility of results,validation study


                Comment on this article