Classical and Sobolev orthogonality of the nonclassical Jacobi polynomials with parameters \alpha =\beta =-1

In this paper, we consider the second-order differential expression $$\begin{aligned} \ell y](x)=(1-x^{2})(-(y^{\prime }(x))^{\prime }+k(1-x^{2})^{-1} y(x))\quad (x\in (-1,1)). \end{aligned}$$ This is the Jacobi differential expression with nonclassical parameters $\alpha =\beta =-1$ in contrast to the classical case when $\alpha ,\beta >-1$ . For fixed $k\ge 0$ and appropriate values of the spectral parameter $\lambda ,$ the equation $\ell y]=\lambda y$ has, as in the classical case, a sequence of (Jacobi) polynomial solutions $\{P_{n}^{(-1,-1)} \}_{n=0}^{\infty }.$ These Jacobi polynomial solutions of degree $\ge 2$ form a complete orthogonal set in the Hilbert space $L^{2}((-1,1);(1-x^{2})^{-1})$ . Unlike the classical situation, every polynomial of degree one is a solution of this eigenvalue equation. Kwon and Littlejohn showed that, by careful selection of this first-degree solution, the set of polynomial solutions of degree $\ge 0$ are orthogonal with respect to a Sobolev inner product. Our main result in this paper is to construct a self-adjoint operator $T$ , generated by $\ell \cdot ],$ in this Sobolev space that has these Jacobi polynomials as a complete orthogonal set of eigenfunctions. The classical Glazman–Krein–Naimark theory is essential in helping to construct $T$ in this Sobolev space as is the left-definite theory developed by Littlejohn and Wellman.