Strong consistency of automatic kernel regression estimates

Regression function estimation from independent and identically distributed bounded data is considered. TheL ₂ error with integration with respect to the design measure is used as an error criterion. It is shown that the kernel regression estimate with an arbitrary random bandwidth is weakly and strongly consistent forall distributions whenever the random bandwidth is chosen from some deterministic interval whose upper and lower bounds satisfy the usual conditions used to prove consistency of the kernel estimate for deterministic bandwidths. Choosing discrete bandwidths by cross-validation allows to weaken the conditions on the bandwidths. Research supported by DAAD, NSERC and Alexander von Humboldt Foundation. The research of the second author was completed during his stay at the Technical University of Szczecin, Poland.     

Strong universal consistency of smooth kernel regression estimates

The paper deals with kernel estimates of Nadaraya-Watson type for a regression function with square integrable response variable. For usual bandwidth sequences and smooth nonnegative kernels, e.g., Gaussian and quartic kernels, strongL ₂-consistency is shown without any further condition on the underlying distribution. The proof uses a Tauberian theorem for Cesàro summability.     

Weak and strong uniform consistency of kernel regression estimates

Summary We study the estimation of a regression function by the kernel method. Under mild conditions on the window, the bandwidth and the underlying distribution of the bivariate observations {(X _i , Y _i)}, we obtain the weak and strong uniform convergence rates on a bounded interval. These results parallel those of Silverman (1978) on density estimation and extend those of Schuster and Yakowitz (1979) and Collomb (1979) on regression estimation.This research was carried out in part while the authors were guests at the University of Heidelberg, Germany, under the sponsorship of the Sonderforschungsbereich 123 in the summer of 1980     

Strong consistency of least squares estimates in multiple regression II

The strong consistency of least squares estimates in multiple regression models is established under minimal assumptions on the design and weak dependence and moment restrictions on the errors.     

Strong consistency of nearest neighbor regression function estimators

For a well-known class of nonparametric regression function estimators of nearest neighbor type the uniform measure of deviation from the estimators to the true regression function is studied. Under weak regularity conditions it is shown that the estimators are uniformly consistent with probability one and the corresponding rate of convergence is near-optimal.     

Probabilities of maximal deviations for nonparametric regression function estimates

Let (X, Y) have regression function m(x) = E(Y | X = x), and let X have a marginal density f₁(x). We consider two nonparameteric estimates of m(x): the Watson estimate when f₁ is known and the Yang estimate when f₁ is known or unknown. For both estimates the asymptotic distribution of the maximal deviation from m(x) is proved, thus extending results of Bickel and Rosenblatt for the estimation of density functions.     

On posterior consistency in nonparametric regression problems

We provide sufficient conditions to establish posterior consistency in nonparametric regression problems with Gaussian errors when suitable prior distributions are used for the unknown regression function and the noise variance. When the prior under consideration satisfies certain properties, the crucial condition for posterior consistency is to construct tests that separate from the outside of the suitable neighborhoods of the parameter. Under appropriate conditions on the regression function, we show there exist tests, of which the type I error and the type II error probabilities are exponentially small for distinguishing the true parameter from the complements of the suitable neighborhoods of the parameter. These sufficient conditions enable us to establish almost sure consistency based on the appropriate metrics with multi-dimensional covariate values fixed in advance or sampled from a probability distribution. We consider several examples of nonparametric regression problems.     

Strong consistency of sample path derivative estimates

We give general conditions for strong consistency of sample-path-based derivative estimators. Such estimators are useful in gradient-based optimization of complex stochastic systems. The results substantially generalize and extend previous results.     

Modified nonparametric kernel estimates of a regression function and their consistencies with rates

Summary Two sets of modified kernel estimates of a regression function are proposed: one when a bound on the regression function is known and the other when nothing of this sort is at hand. Explicit bounds on the mean square errors of the estimators are obtained. Pointwise as well as uniform consistency in mean square and consistency in probability of the estimators are proved. Speed of convergence in each case is investigated. Major work of this research was completed during the first author's two visits (November–December, 1983 and August–September 1984) to the second author at the Universite du Quebec a Montreal. Part of the work of the second author was supported by the Air Force Office of Scientific Research under contract F49620-85-C-0008 while he was at the University of Pittsburgh during Spring in 1985.     

Strong convergence in nonparametric estimation of regression functions

The nonparametric regression problem has the objective of estimating conditional expectation. Consider the model $$Y = R(X) + Z$$ , where the random variableZ has mean zero and is independent ofX. The regression functionR(x) is the conditional expectation ofY givenX = x. For an estimator of the form $$R_n (x) = \sum\limits_{i = 1}^n {Y_i K{{\left[ {{{\left( {x - X_i } \right)} \mathord{\left/ {\vphantom {{\left( {x - X_i } \right)} {c_n }}} \right. \kern-\nulldelimiterspace} {c_n }}} \right]} \mathord{\left/ {\vphantom {{\left[ {{{\left( {x - X_i } \right)} \mathord{\left/ {\vphantom {{\left( {x - X_i } \right)} {c_n }}} \right. \kern-\nulldelimiterspace} {c_n }}} \right]} {\sum\limits_{i = 1}^n {K\left[ {{{\left( {x - X_i } \right)} \mathord{\left/ {\vphantom {{\left( {x - X_i } \right)} {c_n }}} \right. \kern-\nulldelimiterspace} {c_n }}} \right]} }}} \right. \kern-\nulldelimiterspace} {\sum\limits_{i = 1}^n {K\left[ {{{\left( {x - X_i } \right)} \mathord{\left/ {\vphantom {{\left( {x - X_i } \right)} {c_n }}} \right. \kern-\nulldelimiterspace} {c_n }}} \right]} }}} $$ , we obtain the rate of strong uniform convergence $$\mathop {\sup }\limits_{x\varepsilon C} \left| {R_n (x) - R(x)} \right|\mathop {w \cdot p \cdot 1}\limits_ = o({{n^{{1 \mathord{\left/ {\vphantom {1 {(2 + d)}}} \right. \kern-\nulldelimiterspace} {(2 + d)}}} } \mathord{\left/ {\vphantom {{n^{{1 \mathord{\left/ {\vphantom {1 {(2 + d)}}} \right. \kern-\nulldelimiterspace} {(2 + d)}}} } {\beta _n \log n}}} \right. \kern-\nulldelimiterspace} {\beta _n \log n}}),\beta _n \to \infty $$ . HereX is ad-dimensional variable andC is a suitable subset ofR ^d .     

Asymptotic theory of nonparametric regression estimates with censored data

For regression analysis, some useful information may have been lost when the responses are right censored. To estimate nonparametric functions, several estimates based on censored data have been proposed and their consistency and convergence rates have been studied in literature, but the optimal rates of global convergence have not been obtained yet. Because of the possible information loss, one may think that it is impossible for an estimate based on censored data to achieve the optimal rates of global convergence for nonparametric regression, which were established by Stone based on complete data. This paper constructs a regression spline estimate of a general nonparametric regression function based on right_censored response data, and proves, under some regularity conditions, that this estimate achieves the optimal rates of global convergence for nonparametric regression. Since the parameters for the nonparametric regression estimate have to be chosen based on a data driven criterion, we also obtain the asymptotic optimality of AIC, AICC, GCV, Cp and FPE criteria in the process of selecting the parameters.     

Strong convergence in nonparametric regression with truncated dependent data

In this paper we derive rates of uniform strong convergence for the kernel estimator of the regression function in a left-truncation model. It is assumed that the lifetime observations with multivariate covariates form a stationary α-mixing sequence. The estimation of the covariate’s density is considered as well. Under the assumption that the lifetime observations are bounded, we show that, by an appropriate choice of the bandwidth, both estimators of the covariate’s density and regression function attain the optimal strong convergence rate known from independent complete samples.     

Strong consistency of maximum quasi-likelihood estimates in generalized linear models

In a generalized linear model with q×1 responses, bounded and fixed p×q regressors zi and general link function, under the most general assumption on the minimum eigenvalue of ∑in=1 ZiZi', the moment condition on responses as weak as possible and other mild regular conditions, we prove that with probability one, the quasi-likelihood equation has a solution βn for all large sample size n, which converges to the true regression parameter β0. This result is an essential improvement over the relevant results in literature.     

污染数据半参数回归模型中估计的相合性

利用最小二乘估计方法和权函数法给出了半参数模型Y=βX g(T) ε在某种污染方式下,,βg和污染系数的估计,并在适当条件下证明了它们具有相合性.     

Adaptive nonparametric estimation of a multivariate regression function

We consider the kernel estimation of a multivariate regression function at a point. Theoretical choices of the bandwidth are possible for attaining minimum mean squared error or for local scaling, in the sense of asymptotic distribution. However, these choices are not available in practice. We follow the approach of Krieger and Pickands (Ann. Statist.9 (1981) 1066–1078) and Abramson (J. Multivariate Anal.12 (1982), 562–567) in constructing adaptive estimates after demonstrating the weak convergence of some error process. As consequences, efficient data-driven consistent estimation is feasible, and data-driven local scaling is also feasible. In the latter instance, nearest-neighbor-type estimates and variance-stabilizing estimates are obtained as special cases.