Parsimonious Model-Based Clustering with Covariates

Abstract

In model-based clustering methods using finite mixture models, the clustering is implemented on the outcome variables only and reference is not made to the associated covariates until the structure of the clustering is investigated in light of the information present in the covariates. It is desirable to have these covariates incorporated into the clustering process and not only into the interpretation of the clustering structure and model parameters, in order to exploit clustering capabilities and provide richer insight into the type of observation which characterises each cluster. The mixture of experts model provides such a framework: it extends the mixture model to accommodate the presence of covariates by modelling the parameters of the mixture model as functions of the concomitant variables. However, parsimonious parameterisations of the component covariance matrices have to date been lacking in the mixture of experts context. We consider model-based clustering methods that account for external information available in the presence of covariates by proposing the MoEClust suite of models, which allow covariates enter the gating and/or expert networks. This paper addresses the aim of including covariates in parsimonious Gaussian finite mixture models or, equivalently, the aim of incorporating parsimonious covariance structures into the mixture of experts framework. The MoEClust models demonstrate significant improvement on both fronts in applications to univariate and multivariate data sets.