Strongly consistent estimators of k-th order regression curves and rates of convergence

Summary Let X and Y be two jointly distributed real valued random variables, and let the conditional distribution of X given Y be either in a Lebesgue exponential family or in a discrete exponential family. Let r_k be the k-th order regression curve of Y on X. Let X_n=(X₁,..., X_n) be a random sample of size n on X. For a subset S of the real line R, statistics based on X_n are exhibited and sufficient conditions are given under which is close to O(n^–1/2) with probability one. To obtain this result, with uf (u known and f unknown) denoting the unconditional (on y) density of X, the problem of estimating r_k(·) is reduced to the one of estimating f^(k)(·)/f(·) if the density is wrt the Lebesgue measure on R and f^(k) is the k-th order derivative of f; and to the one of estimating f(·+k)/f(·) if the density is wrt the counting measure on a countable subset of R.