Parameter estimation with data-driven nonparametric likelihood functions

Abstract

This paper presents a nonparametric modeling technique to avoid expensive evaluation of the likelihood function in estimating parameters of dynamical systems. The proposed method trains the likelihood function using the spectral expansion formulation known as the kernel embedding of conditional distribution. In particular, the spectral expansion uses a set of basis functions of a weighted Hilbert space defined on the data manifold with sampling density as the weight function. This choice of basis functions is justified by a rigorous error estimate. Numerically, these basis functions are estimated using the diffusion maps algorithm with variable bandwidth kernels on the training data set. The estimated likelihood function and posterior densities of the parameters based on this data-driven basis expansion will be compared to those that use analytical basis functions such as the Hermite polynomials and the Fourier cosine functions, which enforce simple geometry assumptions on the data manifold. The numerical results demonstrate the robustness of the data-driven basis functions on parameter estimation of stochastic and deterministic differential equations. In particular, the data-driven basis expansion produces superior estimates compared to those from the Hermite and cosine basis expansions when the intrinsic dimension of the data manifold is strictly less than the dimension of the ambient space.