Are Fit Indices Biased in Favor of Bi-Factor Models in Cognitive Ability Research?: A Comparison of Fit in Correlated Factors, Higher-Order, and Bi-Factor Models via Monte Carlo Simulations

We propose the application of a bifactor model for exploring the dimensional structure of an item response matrix, and for handling multidimensionality. We argue that a bifactor analysis can complement traditional dimensionality investigations by: (a) providing an evaluation of the distortion that may occur when unidimensional models are fit to multidimensional data, (b) allowing researchers to examine the utility of forming subscales, and, (c) providing an alternative to non-hierarchical multidimensional models for scaling individual differences. To demonstrate our arguments, we use responses (N = 1,000 Medicaid recipients) to 16 items in the Consumer Assessment of Healthcare Providers and Systems (CAHPS2.0) survey. Exploratory and confirmatory factor analytic and item response theory models (unidimensional, multidimensional, and bifactor) were estimated. CAHPS items are consistent with both unidimensional and multidimensional solutions. However, the bifactor model revealed that the overwhelming majority of common variance was due to a general factor. After controlling for the general factor, subscales provided little measurement precision. The bifactor model provides a valuable tool for exploring dimensionality related questions. In the Discussion, we describe contexts where a bifactor analysis is most productively used, and we contrast bifactor with multidimensional IRT models (MIRT). We also describe implications of bifactor models for IRT applications, and raise some limitations.