Commentary: Correction procedures in brain-age prediction

Abstract

Brain-age prediction uses machine learning to estimate an individuals apparent brain aging based on structural and functional brain characteristics derived from neuroimaging, commonly magnetic resonance imaging (MRI) (Cole, Franke, 2017, Cole, Poudel, Tsagkrasoulis, Caan, Steves, Spector, Montana, 2017, Cole, Ritchie, Bastin, Hernández, Maniega, Royle, Corley, Pattie, Harris, Zhang, et al., 2018, Franke, Gaser, 2019, Liem, Varoquaux, Kynast, Beyer, Masouleh, Huntenburg, Lampe, Rahim, Abraham, Craddock, et al., 2017, Richard, Kolskår, Sanders, Kaufmann, Petersen, Doan, Sanchez, Alnaes, Ulrichsen, Dørum, et al., 2018, Smith, Elliott, Alfaro-Almagro, McCarthy, Nichols, Douaud, Miller). Subtracting chronological age from estimated brain age provides a measure of the difference between an individuals predicted and chronological age; the brain age delta. For instance, if a 60 year old individual exhibits a brain age delta of -5 years, their typical aging pattern resembles the brain structure of a 55 year old, i.e., their estimated brain age is younger than what is expected for their chronological age (Franke and Gaser, 2019). Individual variation in delta estimations have been associated with a range of biological and cognitive variables (Cole, Cole, Marioni, Harris, Deary, 2019, Kaufmann, van der Meer, Doan, Schwarz, Lund, Agartz, Alnæs, Barch, Baur-Streubel, Bertolino, et al., 2019, Koutsouleris, Davatzikos, Borgwardt, Gaser, Bottlender, Frodl, Falkai, Riecher-Rössler, Möller, Reiser, et al., 2013, de Lange, Kaufmann, van der Meer, Maglanoc, Alnæs, Moberget, Douaud, Andreassen, Westlye, 2019 de Lange et al., 2020 de Lange, Kaufmann, van der Meer, Maglanoc, Alnæs, Moberget, Douaud, Andreassen, Westlye, 2019, Schnack, Van Haren, Nieuwenhuis, Hulshoff Pol, Cahn, Kahn, 2016, Smith, Vidaurre, Alfaro-Almagro, Nichols, Miller, 2019), but brain-age estimation also involves a frequently observed bias: brain age is overestimated in younger subjects and underestimated in older subjects, while brain age for participants with an age closer to the mean age (of the training dataset) are predicted more accurately (Cole, Le, Kuplicki, McKinney, Yeh, Thompson, Paulus, Investigators, et al., 2018, Liang, Zhang, Niu, 2019, Niu, Zhang, Kounios, Liang, 2019, Smith, Vidaurre, Alfaro-Almagro, Nichols, Miller, 2019). Common practice is to apply a statistical bias correction to the age prediction or the brain age delta estimate. We here provide a brief commentary on the correction methods discussed in the paper ‘Bias-adjustment in neuroimaging-based brain age frameworks: a robust scheme’ by Beheshti et al. (2019), and the use of these methods in brain-age related research. 1 Overview of correction methods Beheshti et al. state that they have developed a new method for adjusting age bias in brain age prediction. Their method does however provide similar corrections to methods previously applied by others (e.g. de Lange, Kaufmann, van der Meer, Maglanoc, Alnæs, Moberget, Douaud, Andreassen, Westlye, 2019, Liang, Zhang, Niu, 2019, Smith, Vidaurre, Alfaro-Almagro, Nichols, Miller, 2019). In the procedure applied by Beheshti et al., the relationship between brain age delta and chronological age is fitted using (1) Offset = α × Ω + β , where Ω represents chronological age, and Offset = Predicted Age – Ω, i.e., the brain age delta. The coefficients α and β represent the slope and intercept, which are then used to correct the predictions in a test set using (2) Corrected Predicted Age = Predicted Age − ( α × Ω + β ) . One example of an equivalent method is the procedure applied in a previous paper by de Lange et al. (2019b), which provides a mathematically identical correction by first fitting (3) Predicted Age = α × Ω + β , and then using the derived values of α and β to correct predicted age with (4) Corrected Predicted Age = Predicted Age + [ Ω − ( α × Ω + β ) ] . Beheshti et al. further compare their correction procedure to a method used by Cole et al. (2018), which can be described with (5) Corrected Predicted Age = P r e d i c t e d A g e − β α . This procedure defines α and β using predicted age as the outcome variable (as opposed to the offset) and corrects the slope without using chronological age. This method inevitably increases the variance of the data as it divides the predicted age for each subject on the slope value (α) obtained from the regression fit. The procedures applied by Beheshti et al. as well as others (de Lange, Kaufmann, van der Meer, Maglanoc, Alnæs, Moberget, Douaud, Andreassen, Westlye, 2019, Liang, Zhang, Niu, 2019, Smith, Vidaurre, Alfaro-Almagro, Nichols, Miller, 2019) include chronological age in the correction (Eq. (3)), which reduces the variance and results in a lower mean absolute error (MAE) when MAE is calculated after applying the correction. 2 Comparison of correction methods 2.1 Model performance To investigate the implications of the different correction methods, we used data from de Lange et al. (2019b) including estimated brain age in a sample of 12,021 women from the UK Biobank. The brain-age prediction was run using XGBoost with 10-fold cross validation as described in de Lange et al. (2019b), and included 1118 imaging-derived brain measures. The MAE values and correlations between a) predicted age and chronological age and b) brain age delta and chronological age are shown for each correction method in Table 1. The results showed that the methods equally eliminated the dependence of brain age delta on chronological age. As emphasized in the paper by Beheshti et al., the use of chronological age in the correction (M1 and M2) reduced the MAE, while the correction that did not include chronological age (M3) increased the variance and thus the MAE. Table 1 Mean absolute error (MAE) and correlations between a) predicted age and chronological age and b) brain age delta and chronological age. Method (M) 0 represents the values before any corrections. The results after applying the corrections are shown by M1 (Beheshti et al.), M2 (de Lange et al.), and M3 (Cole et al.). 95% confidence intervals are indicated in square brackets. Table 1 M MAE Predicted age vs. age brain age delta vs. age 0 4.74 r = 0.61 , p < 0.001 , [ 0.60 , 0.62 ] r = − 0.85 , p < 0.001 , [ − 0.86 , − 0.85 ] 1 2.44 r = 0.92 , p < 0.001 , [ 0.92 , 0.93 ] r = 0.00 , p = 1.00 , [ − 0.02 , 0.02 ] 2 2.44 r = 0.92 , p < 0.001 , [ 0.92 , 0.93 ] r = 0.00 , p = 1.00 , [ − 0.02 , 0.02 ] 3 7.62 r = 0.61 , p < 0.001 , [ 0.60 , 0.62 ] r = 0.00 , p = 1.00 , [ − 0.02 , 0.02 ] 2.2 Variables of interest While MAE is commonly used to compare model precision, the main aim of brain-age prediction is to provide a biomarker that can be analysed in relation to other variables of interest, for example cognitive or clinical data. Using the different correction methods, we re-analysed data from de Lange et al. (2019b) including the association between brain age delta and the variable number of childbirths. The results are shown in Tables 2 and 3. Table 2 Correlations between brain age delta and number (n) of childbirths (CB) without any correction (M0), and after applying correction method 1/2 and 3, shown with and without age included as a covariate. Table 2 M brain age delta vs. n CB brain age delta vs. n CB incl. age 0 r = − 0.176 , p < 0.001 , [ − 0.19 , − 0.16 ] r = − 0.074 , p < 0.001 , [ − 0.09 , − 0.06 ] 1/2 r = − 0.073 , p < 0.001 , [ − 0.09 , − 0.05 ] r = − 0.074 , p < 0.001 , [ − 0.09 , − 0.06 ] 3 r = − 0.073 , p < 0.001 , [ − 0.09 , − 0.05 ] r = − 0.074 , p < 0.001 , [ − 0.09 , − 0.06 ] Table 3 Mean difference in brain age delta [years] and effect sizes (d) for nulliparous (N = 2453) versus parous (N = 9568) women without any correction (M0), and after applying correction method 1/2 and 3. Table 3 M Mean diff t p Effect size ( d ) Error 0 2.28 17.54  < 0.001 0.40 0.02 1/2 1.80 8.45  < 0.001 0.19 0.02 3 0.57 8.45  < 0.001 0.19 0.02 As a cross-check, we ran the same analyses with a second variable of interest, systolic blood pressure (SBP), in the same sample. The results are provided in Table 4. Table 4 Correlations between brain age delta and SBP without any correction (M0), and after applying correction method 1/2 and 3, shown with and without age included as a covariate. Table 4 M brain age delta vs. SBP. brain age delta vs. SBP incl. age 0 r = − 0.284 , p < 0.001 , [ − 0.3 , − 0.27 ] r = 0.035 , p < 0.001 , [ 0.02 , 0.05 ] 1/2 r = 0.032 , p < 0.001 , [ 0.01 , 0.05 ] r = 0.035 , p < 0.001 , [ 0.02 , 0.05 ] 3 r = 0.032 , p < 0.001 , [ 0.01 , 0.05 ] r = 0.035 , p < 0.001 , [ 0.02 , 0.05 ] In accordance with the findings by Beheshti et al., correlations and effect sizes for group differences did not change with the correction methods. Behesti et al. also compare mean of brain age delta in clinical samples after applying the different correction methods. As Method 3 involves a shift in the brain age delta scale by dividing the predictions by the slope value ((Predicted age – intercept) / slope), the corrected brain age delta values will in general differ depending on the method used. 3 Conclusions Two main conclusions can be drawn based on the examples in this commentary: I) The method proposed by Behesti et al. provides age-bias correction that is equivalent to methods used in previous studies. These methods include chronological age in the correction, which reduces the variance in brain age delta values and leads to lower MAE after correction. In contrast, the correction method that does not include chronological age leads to a higher MAE due to increased variance, and a shift in the brain age delta scale. While both methods equally correct the dependence of brain age delta on chronological age, group differences in mean brain age delta [years] depend on the method used. This is important to be aware of when comparing results across studies. II) While methods that include chronological age in the correction reduce the MAE, they do not appear to increase sensitivity to subsequent correlations or group effects. In such cases, using age as a covariate (Le et al., 2018) can achieve the goal of correcting for age bias equally as effectively as explicit correction of the brain age prediction or the brain age delta estimate. Including age as a covariate also accounts for potential age-dependence in variables of interest. Several correction methods are available in brain-age research, many of which provide equivalent corrections to the age bias. With this article, we hoped to clarify some areas of potential confusion around bias correction for brain age by providing a consistent notation that should be useful for the community.