Nonstandard Bayes and Complete Class Theorems

Abstract

For finite parameter spaces under finite loss, every Bayesian procedure derived from a prior with full support is admissible, and every admissible procedure is Bayes. This relationship begins to break down as we move to continuous parameter spaces. Under some regularity conditions, admissible procedures can be shown to be the limits of Bayesian procedures. Under additional regularity, they are generalized Bayesian, i.e., they minimize the Bayes risk with respect to an improper prior. In both these cases, one must venture beyond the strict confines of Bayesian analysis. Using methods from mathematical logic and nonstandard analysis, we introduce the class of nonstandard Bayesian decision procedures---namely, those whose Bayes risk with respect to some prior is within an infinitesimal of the optimal Bayes risk. Without any regularity conditions, we show that a decision procedure is extended admissible if and only if its nonstandard extension is nonstandard Bayes. We apply the nonstandard theory to derive a purely standard theorem: on a compact parameter space, every extended admissible estimator is Bayes if the risk function is continuous.