Statistical Learning in Tree-Based Tensor Format

Tensor methods are widely used tools for the approximation of high dimensional functions. Such problems are encountered in uncertainty quantification and statistical learning, where the high dimensionality imposes to use specific techniques, such as rank-structured approximations [1]. In this work, we introduce a statistical learning algorithm for the approximation in tree-based tensor format, which are tensor networks whose graphs are dimension partition trees. This tensor format includes the Tucker format, the Tensor-Train format, as well as the more general Hierarchical tensor formats [4]. It can be interpreted as a deep neural network with a particular architecture [2]. The proposed algorithm uses random evaluations of a function to provide a tree-based tensor approximation, with adaptation of the tree-based rank by using a heuristic criterion based on the higher-order singular values to select the ranks to increase, and of the approximation spaces of the leaves of the tree. We then present a learning algorithm for the approximation under the form u(x) ≈ v(z_1,...,z_m) where v is a tensor in tree-based format and the z_i = g_i(x), 1 ≤ i ≤ m, are new variables. A strategy based on the projection pursuit regression [3] is proposed to compute the mappings g_i and increase the effective dimension m. The methods are illustrated on different examples to show their efficiency and adaptability as well as the power of representation of the tree-based tensor format, possibly combined with changes of variables.