Symmetry breaking for representations of rank one orthogonal groups II

Abstract

For a pair \((G,G')=(O(n+1,1), O(n,1))\) of reductive groups, we investigate intertwining operators (symmetry breaking operators) between principal series representations \(I_\delta(V,\lambda)\) of \(G\), and \(J_\varepsilon(W,\nu)\) of the subgroup \(G'\). The representations are parametrized by finite-dimensional representations \(V\), \(W\) of \(O(n)\) respectively of \(O(n-1)\), characters \(\delta\), \(\varepsilon\) of \(O(1)\), and \(\lambda, \ \nu \in {\mathbb C}\). Let \([V:W]\) be the multiplicity of \(W\) occurring in the restriction \(V|_{O(n-1)}\), which is either 0 or 1. If \([V:W] \ne 0\) then we construct a meromorphic family of symmetry breaking operators and prove that \(\operatorname{Hom}_{G'}(I_{\delta}(V, \lambda)|_{G'}, J_{\varepsilon}(W, \nu))\) is nonzero for all the parameters \(\lambda\), \(\nu\) and \(\delta\), \(\varepsilon\), whereas if \([V:W] = 0\) there may exist sporadic differential symmetry breaking operators. Moreover we obtain a complete classification of symmetry breaking operators in the special case \( (V,W)=(\bigwedge^i({\mathbb{C}}^{n}), \bigwedge^j({\mathbb{C}}^{n-1}))\). We use this information to determine the space of symmetry breaking operators for any pair of irreducible representations of \(G\) and the subgroup \(G'\) with trivial infinitesimal character \(\rho \). Furthermore we prove the multiplicity conjecture by B. Gross and D. Prasad for tempered principal series representations of \((SO(n+1,1), SO(n,1))\) and also for 3 tempered representations \(\Pi, \pi, \varpi\) of \(SO(2m+2,1)\), \(SO(2m+1,1)\) and \(SO(2m,1)\) with trivial infinitesimal character \(\rho\). We also apply our main results to find periods of irreducible representations of the Lorentz group having nonzero \(({\mathfrak{g}}, K)\)-cohomologies. This article is an extension of [Memoirs Amer. Math. Soc. 2015] that treated spherical principal series representations.