+1 Recommend
    • Review: found
    Is Open Access

    Review of 'Cognitive Reserve in the Healthy Elderly: Cognitive and Psychological Factors'

    Cognitive Reserve in the Healthy Elderly: Cognitive and Psychological FactorsCrossref
    Interesting study, but obvious methodological and conceptual problems
    Average rating:
        Rated 3 of 5.
    Level of importance:
        Rated 3 of 5.
    Level of validity:
        Rated 2 of 5.
    Level of completeness:
        Rated 2 of 5.
    Level of comprehensibility:
        Rated 4 of 5.
    Competing interests:

    Reviewed article

    • Record: found
    • Abstract: found
    • Article: found
    Is Open Access

    Cognitive Reserve in the Healthy Elderly: Cognitive and Psychological Factors

    Cognitive reserve (CR) helps explain the mismatch between expected cognitive decline and observed maintenance of cognitive functioning in older age. Factors such as education, literacy, lifestyle, and social networking are usually considered to be proxies of CR and its variability between individuals. A more direct approach to examine CR is through the assessment of capacity to gain from practice in a standardized challenging cognitive task that demands activation of cognitive resources. In this study, we applied a testing-the-limits paradigm to a group of 136 healthy elderly subjects (60-75 years) and additionally examined the possible contribution of complex mental activities and quality of sleep to cognitive performance gain. We found a significant, but variable gain and identified verbal memory, cognitive flexibility and problem solving as significant factors. This outcome is in line with our earlier study on CR in healthy mental aging (Zihl et al., 2014). Interestingly and contrary to expectations, our analysis revealed that complex mental activities and sleep quality do not significantly influence CR. Contrasting “high” and ”low” cognitive performers revealed significant differences in verbal memory and cognitive flexibility; again, complex mental activities and sleep quality did not contribute to this measure of CR. In conclusion, the results of this study support and extend previous findings on CR in older age; further they underline the need for improvements in existing protocols for assessing CR in a dynamic manner.

      Review information

      This work has been published open access under Creative Commons Attribution License CC BY 4.0, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Conditions, terms of use and publishing policy can be found at www.scienceopen.com.

      Review text

      The authors want to replicate the findings from a previous study (Zihl et al., PLoS One. 2014;9: e84590) and want to demonstrate that the Digit Symbol Substitution Test (DSST) is a valid and reliable measure of cognitive reserve (CR) when used in a testing-the-limits paradigm.

      Their main hypotheses were that (1) participants with higher processing resources and higher executive capacities will show higher CR, and (2) CR is positively correlated with cognitive lifestyles, mood, well-being, and sleep quality.

      To test the hypotheses, 136 healthy elderly participants (age range: 60–75 years) with high level of education (≥13 years of schooling) had to perform 10 consecutive DSST trials. In addition, the authors assessed several cognitive and non-cognitive variables that might contribute to, or interact with CR.

      In my view this is an interesting study. However, there are several methodological problems that limit its scientific merit.

      A major problem of the present study is that the performance of the elderly participants was not compared to the performance of a young group. Therefore, it is not possible to replicate previous findings and to clarify whether age is an important moderating factor. It is also not possible to clarify whether the performance gain in the DSST is special or extraordinary. It is well known that repeated practice leads to an improvement in performance, regardless whether elderly participants or brain-damaged patients were investigated.

      Another problem in the present manuscript is the fact that the statistical analyses were unambitious. As Satz et al. (2010) indicated more complex statistical methods (e.g. Structural Equation Modeling, SEM) are needed to achieve “greater clarity to the conceptualization and study of the reserve construct”. By using SEM or regression analyses the moderating effects of the different variables (e.g. age, cognitive lifestyles, mood, well-being, and sleep quality) could be analyzed in a more sophisticated way.

      A further problem is that there is no a priori model of the moderating or mediating variables presented and tested. For example, sleep quality or reading performance are not included in the two models postulated by Satz et al. (2010).

      Due to these reasons the scientific merit of the present study is limited and it is difficult for me to see how the findings “... support and extend the model proposed by Satz et al. [3] ...”.

      Here are further points that need some clarification:

      • The authors should indicate that participants in this study did not participate in the first study (Zihl et al., PLoS One. 2014;9: e84590).
      • Performance gain was calculated as the difference between correctly assigned symbols in the first and in the best trial of the 10 consecutive trials. I wonder whether this calculation is really indicating the pure performance gain. What if the best trials is just an outlier? Couldn´t a mean score of the last two or three trials not be a more stable and reliable indicator of pure performance gain?
      • Please indicate when the more complex CR index is used and which score is presented in Table 1?
      • In addition to the effect sizes, the respective confidence intervals should be reported.
      • The Bonferroni correction is not clear to me. How many contrasts were calculated and what was the real selected alpha level? Please keep in mind that the Bonferroni correction is very conservative and leads to an increase in beta error.
      • Please indicate the number of participants in the extreme group comparisons.
      • Table 1: Scoring of the Stroop Test is not clear to me. If I understand correctly participants needed 100.45 seconds to read the word plate and 41.22 seconds the interference plate. Shouldn´t this be the other way round? The response time is fairly accurate. How was it measured?
      • Obvious limitations (e.g. missing control group) of the study were not mentioned in the Discussion section of the manuscript.


      Many thanks for the comments. We have considered all issues that were relevant and

      helpful, and have revised the manuscript accordingly. Because of the small number of

      participants it was impossible to perform the complex statistical analysis proposed, but have

      pursued a more ambitious statistical approach by using a regression model. The outcome is

      very similar and supports the conclusions we have drawn based on our first statistical


      Other issues.

      1. Almost half of the sample (n= 63) participated in the first study. However, the results are

      unlikely influenced by the repeated presentation of the test for the following reasons.

      First, we used a modified version of the DSST in this study by replacing all symbols to

      ensure that all subjects are confronted with the material for the first time.  Second, to test

      whether the results are influenced by the familiarity of the test, we compared the

      performance of participants confronted for the first time with the (modified) DSST and

      subjects who had participated in the first study (original DSST) in all measures used in

      calculating CR. We found no significant differences regarding the baseline performance

      in the modified DSST (p=.51), best performance (p=.36), absolute gain (p=.56), or the

      measure of CR (p=.42).    

      2. Our measure of CR is not just a difference measure of the first and best trial but

      represents the area under the curve of the consecutive trials. It takes into account the

      increase per trial (compared to the baseline) as well as baseline performance. Thus, an

      individual with high baseline performance will have a higher measure of CR than an

      individual with the same improvement but a lower baseline level. In addition, an ‘outlier’

      (although this represents the best possible performance) will contribute less to the CR

      score than a constant performance on a high level.

      3. Because we used gain as operationalization of CR, we refer now insistently to this

      measure in the results section.

      4. We decided not to use confidence intervals, because they are not necessary for the

      interpretation of our results..

      5. We have now stated more clearly in the text how the Bonferroni correction was applied.

      6. The numbers of participants in the extreme group comparisons are now provided.

      7. We have changed the scoring method and counted the items completed within 45s

      instead of taking the time to complete the single test sheets. Thus, as mentioned in the

      method section, all measures of the Stroop represent items completed within 45 seconds.

      2015-11-28 08:11 UTC
      One person recommends this

      Comment on this review