1 Department of Mathematics, Science and Technology, Aarhus University2 University of Tokyo3 Department of Mathematics, Science and Technology, Aarhus University
We find the complete branching law for the restriction of complementary series representations of $O(1,n+1)$ to the symmetric subgroup $O(1,m+1)\times O(n-m)$, $0\leq m<n$. The decomposition consists of a continuous part and a discrete part which is trivial for some parameters. The continuous part is given by a direct integral of principal series representations whereas the discrete part consists of finitely many complementary series representations. The explicit Plancherel formula is computed on the Fourier transformed side of the non-compact realization of the complementary series by using the spectral decomposition of a certain hypergeometric type ordinary differential operator. The main tool connecting this differential operator with the representations are second order Bessel operators which describe the Lie algebra action in this realization.