Internal consistency reliability is a poor predictor of responsiveness

Abstract

Background

Whether responsiveness represents a measurement property of health-related quality of life (HRQL) instruments that is distinct from reliability and validity is an issue of debate. We addressed the claims of a recent study, which suggested that investigators could rely on internal consistency to reflect instrument responsiveness.

Methods

516 patients with chronic obstructive pulmonary disease or knee injury participating in four longitudinal studies completed generic and disease-specific HRQL questionnaires before and after an intervention that impacted on HRQL. We used Pearson correlation coefficients and linear regression to assess the relationship between internal consistency reliability (expressed as Cronbach's alpha), instrument type (generic and disease-specific) and responsiveness (expressed as the standardised response mean, SRM).

Results

Mean Cronbach's alpha was 0.83 (SD 0.08) and mean SRM was 0.59 (SD 0.33). The correlation between Cronbach's alpha and SRMs was 0.10 (95% CI -0.12 to 0.32) across all studies. Cronbach's alpha alone did not explain variability in SRMs (p = 0.59, r ² = 0.01) whereas the type of instrument was a strong predictor of the SRM (p = 0.012, r ² = 0.37). In multivariable models applied to individual studies Cronbach's alpha consistently failed to predict SRMs (regression coefficients between -0.45 and 1.58, p-values between 0.15 and 0.98) whereas the type of instrument did predict SRMs (regression coefficients between -0.25 to -0.59, p-values between <0.01 and 0.05).

Conclusion

Investigators must look to data other than internal consistency reliability to select a responsive instrument for use as an outcome in clinical trials.