A remarkably simple and accurate method for computing the Bayes Factor from a Markov chain Monte Carlo Simulation of the Posterior Distribution in high dimension

Abstract

Weinberg (2012) described a constructive algorithm for computing the marginal likelihood, Z, from a Markov chain simulation of the posterior distribution. Its key point is: the choice of an integration subdomain that eliminates subvolumes with poor sampling owing to low tail-values of posterior probability. Conversely, this same idea may be used to choose the subdomain that optimizes the accuracy of Z. Here, we explore using the simulated distribution to define a small region of high posterior probability, followed by a numerical integration of the sample in the selected region using the volume tessellation algorithm described in Weinberg (2012). Even more promising is the resampling of this small region followed by a naive Monte Carlo integration. The new enhanced algorithm is computationally trivial and leads to a dramatic improvement in accuracy. For example, this application of the new algorithm to a four-component mixture with random locations in 16 dimensions yields accurate evaluation of Z with 5% errors. This enables Bayes-factor model selection for real-world problems that have been infeasible with previous methods.