Asymptotic properties of posterior distributions in nonparametric regression with non-Gaussian errors

We investigate the asymptotic behavior of posterior distributions in nonparametric regression problems when the distribution of noise structure of the regression model is assumed to be non-Gaussian but symmetric such as the Laplace distribution. Given prior distributions for the unknown regression function and the scale parameter of noise distribution, we show that the posterior distribution concentrates around the true values of parameters. Following the approach by Choi and Schervish (Journal of Multivariate Analysis, 98, 1969–1987, 2007) and extending their results, we prove consistency of the posterior distribution of the parameters for the nonparametric regression when errors are symmetric non-Gaussian with suitable assumptions.     

Nonparametric Bayesian drift estimation for multidimensional stochastic differential equations*

We consider nonparametric Bayesian estimation of the drift coefficient of a multidimensional stochastic differential equation from discrete-time observations on the solution of this equation. Under suitable regularity conditions, we establish posterior consistency in this context.     

Posterior consistency for partially observed Markov models

We establish the posterior consistency for parametric, partially observed, fully dominated Markov models. The prior is assumed to assign positive probability to all neighborhoods of the true parameter, for a distance induced by the expected Kullback–Leibler divergence between the parametric family members’ Markov transition densities. This assumption is easily checked in general. In addition, we show that the posterior consistency is implied by the consistency of the maximum likelihood estimator. The result is extended to possibly improper priors and non-stationary observations. Finally, we check our assumptions on a linear Gaussian model and a well-known stochastic volatility model.     

Uniform convergence of nonparametric regressions in competing risk models with right censoring

We consider, in the presence of covariates, non-independent competing risks that are subject to right censoring. We define a nonparametric estimator of the incident regression function through the generalized product-limit estimator of the conditional censorship distribution function. Under suitable conditions, we establish the almost sure uniform convergence of those estimators with an appropriate rate.     

截尾数据非参数回归函数加权核估计

在截尾数据下研究非参数回归函数加权核估计的相合性,对强相合性给出一些较弱的充分条件,这些结论较大程度地改进了现有的结论．     

Strong consistency of nonparametric Bayes density estimation on compact metric spaces with applications to specific manifolds

This article considers a broad class of kernel mixture density models on compact metric spaces and manifolds. Following a Bayesian approach with a nonparametric prior on the location mixing distribution, sufficient conditions are obtained on the kernel, prior and the underlying space for strong posterior consistency at any continuous density. The prior is also allowed to depend on the sample size n and sufficient conditions are obtained for weak and strong consistency. These conditions are verified on compact Euclidean spaces using multivariate Gaussian kernels, on the hypersphere using a von Mises-Fisher kernel and on the planar shape space using complex Watson kernels.     

Generalized Partially Linear Measurement Error Models

This article considers generalized partially linear models when the linear covariate is measured with additive error. We propose estimators of parameter and nonparametric function by using local linear regression, the SIMEX technique, and generalized estimating equation. The asymptotic normality of the estimators of the parameter, and bias and variance of the estimators of the nonparametric component are derived under appropriate assumptions. In addition, the generalization to clustered measurements is discussed. The approaches are used to the analysis of data from the Framingham Heart Study. A simulation experiment is conducted for an illustration.     

Priors for Bayesian adaptive spline smoothing

Adaptive smoothing has been proposed for curve-fitting problems where the underlying function is spatially inhomogeneous. Two Bayesian adaptive smoothing models, Bayesian adaptive smoothing splines on a lattice and Bayesian adaptive P-splines, are studied in this paper. Estimation is fully Bayesian and carried out by efficient Gibbs sampling. Choice of prior is critical in any Bayesian non-parametric regression method. We use objective priors on the first level parameters where feasible, specifically independent Jeffreys priors (right Haar priors) on the implied base linear model and error variance, and we derive sufficient conditions on higher level components to ensure that the posterior is proper. Through simulation, we demonstrate that the common practice of approximating improper priors by proper but diffuse priors may lead to invalid inference, and we show how appropriate choices of proper but only weakly informative priors yields satisfactory inference.     

左截断相依数据下非参数回归的局部M 估计

本文对左截断模型, 利用局部多项式的方法构造了非参数回归函数的局部M 估计. 在观察样本为平稳α-混合序列下, 建立了该估计量的强弱相合性以及渐近正态性. 模拟研究显示回归函数的局部M 估计比Nadaraya-Watson 型估计和局部多项式估计更稳健.     

Model Selection for Regularized Least-Squares Algorithm in Learning Theory

We investigate the problem of model selection for learning algorithms depending on a continuous parameter. We propose a model selection procedure based on a worst-case analysis and on a data-independent choice of the parameter. For the regularized least-squares algorithm we bound the generalization error of the solution by a quantity depending on a few known constants and we show that the corresponding model selection procedure reduces to solving a bias-variance problem. Under suitable smoothness conditions on the regression function, we estimate the optimal parameter as a function of the number of data and we prove that this choice ensures consistency of the algorithm.     

Using the bootstrap to estimate mean squared error and select smoothing parameter in nonparametric problems

We describe a bootstrap method for estimating mean squared error and smoothing parameter in nonparametric problems. The method involves using a resample of smaller size than the original sample. There are many applications, which are illustrated using the special cases of nonparametric density estimation, nonparametric regression, and tail parameter estimation.     

纵向数据半参数回归模型的估计方法

本文考虑纵向数据半参数回归模型,通过考虑纵向数据的协方差结构,基于Profile最小二乘法和局部线性拟合的方法建立了模型中参数分量、回归函数和误差方差的估计量,来提高估计的有效性,在适当条件下给出了这些估计量的相合性.并通过模拟研究将该方法与最小二乘局部线性拟合估计方法进行了比较,表明了Profile最小二乘局部线性拟合方法在有限样本情况下具有良好的性质.     

Bayesian Conditional Density Filtering

We propose a conditional density filtering (C-DF) algorithm for efficient online Bayesian inference. C-DF adapts MCMC sampling to the online setting, sampling from approximations to conditional posterior distributions obtained by propagating surrogate conditional sufficient statistics (a function of data and parameter estimates) as new data arrive. These quantities eliminate the need to store or process the entire dataset simultaneously and offer a number of desirable features. Often, these include a reduction in memory requirements and runtime and improved mixing, along with state-of-the-art parameter inference and prediction. These improvements are demonstrated through several illustrative examples including an application to high dimensional compressed regression. In the cases where dimension of the model parameter does not grow with time, we also establish sufficient conditions under which C-DF samples converge to the target posterior distribution asymptotically as sampling proceeds and more data arrive. Supplementary materials of C-DF are available online.     

NA样本回归函数估计的强相合性

在ＮＡ相依样本下研究非参数回归函数加权核估计的相合性,获得了一些较弱的充分条件,与此同时对ＮＡ序列给出一个简洁实用的Ｂｅｒｎｓｔｅｉｎ型不等式。     

Bayesian nonparametric regression with varying residual density

We consider the problem of robust Bayesian inference on the mean regression function allowing the residual density to change flexibly with predictors. The proposed class of models is based on a Gaussian process (GP) prior for the mean regression function and mixtures of Gaussians for the collection of residual densities indexed by predictors. Initially considering the homoscedastic case, we propose priors for the residual density based on probit stick-breaking mixtures. We provide sufficient conditions to ensure strong posterior consistency in estimating the regression function, generalizing existing theory focused on parametric residual distributions. The homoscedastic priors are generalized to allow residual densities to change nonparametrically with predictors through incorporating GP in the stick-breaking components. This leads to a robust Bayesian regression procedure that automatically down-weights outliers and influential observations in a locally adaptive manner. The methods are illustrated using simulated and real data applications.     

一类线性回归模型中核估计的渐近性质

考虑固定设计下具有一阶非参数自回归误差的线性模型,构造了参数和非参数函数的N-W核估计,在适当的条件下,证明了参数估计的强相合性,同时给出了非参数函数估计的渐近正态性.     

Generalized partially linear mixed-effects models incorporating mismeasured covariates

In this article we consider a semiparametric generalized mixed-effects model, and propose combining local linear regression, and penalized quasilikelihood and local quasilikelihood techniques to estimate both population and individual parameters and nonparametric curves. The proposed estimators take into account the local correlation structure of the longitudinal data. We establish normality for the estimators of the parameter and asymptotic expansion for the estimators of the nonparametric part. For practical implementation, we propose an appropriate algorithm. We also consider the measurement error problem in covariates in our model, and suggest a strategy for adjusting the effects of measurement errors. We apply the proposed models and methods to study the relation between virologic and immunologic responses in AIDS clinical trials, in which virologic response is classified into binary variables. A dataset from an AIDS clinical study is analyzed.     

Uniform limit laws of the logarithm for nonparametric estimators of the regression function in presence of censored data

In this paper, we establish uniform-in-bandwidth limit laws of the logarithm for nonparametric Inverse Probability of Censoring Weighted (I.P.C.W.) estimators of the multivariate regression function under random censorship. A similar result is deduced for estimators of the conditional distribution function. The uniform-in-bandwidth consistency for estimators of the conditional density and the conditional hazard rate functions are also derived from our main result. Moreover, the logarithm laws we establish are shown to yield almost sure simultaneous asymptotic confidence bands for the functions we consider. Examples of confidence bands obtained from simulated data are displayed.      

Local adaptive smoothing in kernel regression estimation

We consider nonparametric estimation of a smooth function of one variable. Global selection procedures cannot sufficiently account for local sparseness of the covariate nor can they adapt to local curvature of the regression function. We propose a new method for selecting local smoothing parameters which takes into account sparseness and adapts to local curvature. A Bayesian type argument provides an initial smoothing parameter which adapts to the local sparseness of the covariate and provides the basis for local bandwidth selection procedures which further adjust the bandwidth according to the local curvature of the regression function. Simulation evidence indicates that the proposed method can result in reduction of both pointwise mean squared error and integrated mean squared error.     

连续时间下非参数回归模型的误差密度估计

本文研究连续时间下非参数回归的误差密度估计问题，给出误差密度的一个核估计量，利用回归函数的核估计在紧区间上一致均收敛的结论证明了该统计量渐近无偏差，均方相合法，并说明了该核估计中窗宽选取的办法。