33
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      A Residual Bootstrap for High-Dimensional Regression with Near Low-Rank Designs

      Preprint

      Read this article at

          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          We study the residual bootstrap (RB) method in the context of high-dimensional linear regression. Specifically, we analyze the distributional approximation of linear contrasts \(c^{\top} (\hat{\beta}_{\rho}-\beta)\), where \(\hat{\beta}_{\rho}\) is a ridge-regression estimator. When regression coefficients are estimated via least squares, classical results show that RB consistently approximates the laws of contrasts, provided that \(p\ll n\), where the design matrix is of size \(n\times p\). Up to now, relatively little work has considered how additional structure in the linear model may extend the validity of RB to the setting where \(p/n\asymp 1\). In this setting, we propose a version of RB that resamples residuals obtained from ridge regression. Our main structural assumption on the design matrix is that it is nearly low rank --- in the sense that its singular values decay according to a power-law profile. Under a few extra technical assumptions, we derive a simple criterion for ensuring that RB consistently approximates the law of a given contrast. We then specialize this result to study confidence intervals for mean response values \(X_i^{\top} \beta\), where \(X_i^{\top}\) is the \(i\)th row of the design. More precisely, we show that conditionally on a Gaussian design with near low-rank structure, RB simultaneously approximates all of the laws \(X_i^{\top}(\hat{\beta}_{\rho}-\beta)\), \(i=1,\dots,n\). This result is also notable as it imposes no sparsity assumptions on \(\beta\). Furthermore, since our consistency results are formulated in terms of the Mallows (Kantorovich) metric, the existence of a limiting distribution is not required.

          Related collections

          Author and article information

          Journal
          2016-07-04
          Article
          1607.00743
          cb5e649f-9ac3-46bd-9a12-5ede9e418afa

          http://arxiv.org/licenses/nonexclusive-distrib/1.0/

          History
          Custom metadata
          The main text of this paper was published at NIPS 2014. Proofs are included here in the appendices
          math.ST stat.ME stat.ML stat.TH

          Machine learning,Methodology,Statistics theory
          Machine learning, Methodology, Statistics theory

          Comments

          Comment on this article