Nonconcave penalized inverse regression in single-index models with high dimensional predictors

In this paper we aim to estimate the direction in general single-index models and to select important variables simultaneously when a diverging number of predictors are involved in regressions. Towards this end, we propose the nonconcave penalized inverse regression method. Specifically, the resulting estimation with the SCAD penalty enjoys an oracle property in semi-parametric models even when the dimension, p_n, of predictors goes to infinity. Under regularity conditions we also achieve the asymptotic normality when the dimension of predictor vector goes to infinity at the rate of p_n=o(n^1/3) where n is sample size, which enables us to construct confidence interval/region for the estimated index. The asymptotic results are augmented by simulations, and illustrated by analysis of an air pollution dataset.     

Estimation in generalized linear models for functional data via penalized likelihood

We analyze in a regression setting the link between a scalar response and a functional predictor by means of a Functional Generalized Linear Model. We first give a theoretical framework and then discuss identifiability of the model. The functional coefficient of the model is estimated via penalized likelihood with spline approximation. The L² rate of convergence of this estimator is given under smoothness assumption on the functional coefficient. Heuristic arguments show how these rates may be improved for some particular frameworks.     

Partial sum process to check regression models with multiple correlated response: With an application for testing a change-point in profile data

We consider regression models with multiple correlated responses for each design point. Under the null hypothesis, a linear regression is assumed. For the least-squares residuals of this linear regression, we establish the limit of the partial sums. This limit is a projection on a certain subspace of the reproducing Kernel Hilbert space of a multivariate Brownian motion. Based on this limit, we propose a significance test of Kolmogorov-Smirnov type to test the null hypothesis and show that this result can be used to study a change-point problem in the case of linear profile data (panel data). We compare our proposed method, which does not rely on any distributional assumptions, with the likelihood ratio test in a simulation study.     

Strong consistency properties of nonparametric estimators for randomly censored data,I: The product-limit estimator

In reliability and survival-time studies one frequently encounters the followingrandom censorship model:X ₁,Y ₁,X ₂,Y ₂, is an independent sequence of nonnegative rv's, theX _n' s having common distributionF and theY _n' s having common distributionG, Z _n =min{X _n ,Y _n },T _n =I[X _n <-Y _n ]; ifX _n represents the (potential) time to death of then-th individual in the sample andY _n is his (potential) censoring time thenZ _n represents the actual observation time andT _n represents the type of observation (T _n =O is a censoring,T _n =1 is a death). One way to estimateF from the observationsZ ₁.T ₁,Z ₂,T ₂, (and without recourse to theX _n' s) is by means of theproduct limit estimator (Kaplan andMeier [6]). It is shown that a.s., uniformly on [0,T] ifH(T ^–)<1 wherel–H=(l–F) (l–G), uniformly onR if whereT _F =sup {x:F(x)<1}; rates of convergence are also established. These results are used in Part II of this study to establish strong consistency of some density and failure rate estimators based on .The third author's research was partly supported by National Research Council of Canada     

Strong consistency properties of nonparametric estimators for randomly censored data,II: Estimation of density and failure rate

This article is Part II of a two-part study. Properties of the product-limit estimator established in the previous part [2] are now used to prove the strong consistency of some nonparametric density and failure rate estimators which can be used with randomly censored data.The third author's research was partly supported by the National Research Council of Canada.     

Linear combination of concomitants of order statistics with application to testing and estimation

Summary Let (X ₁,Y ₁), (X ₂,Y ₂),…, (X _n,Y _n) be i.i.d. as (X, Y). TheY-variate paired with therth orderedX-variateX _rn is denoted byY _rn and terms the concomitant of therth order statistic. Statistics of the form are considered. The asymptotic normality ofT _n is established. The asymptotic results are used to test univariate and bivariate normality, to test independence and linearity ofX andY, and to estimate regression coefficient based on complete and censored samples.     

Estimation of two ordered bivariate mean residual life functions

Situations occur frequently in which the mean residual life (mrl) functions of two populations must be ordered. For example, if a mechanical device is improved, the mrl function for the improved device should not be less than that of the original device. Also, mrl functions for medical patients should often be ordered depending on the status of concomitant variables. This paper proposes nonparametric estimators of the bivariate mrl function under a mrl ordering. The estimators are shown to be asymptotically unbiased, strongly uniformly consistent and weakly convergent to a bivariate Gaussian process. The estimators are shown to be the projections, in a sense to be made precise, of the empirical mrl function onto an appropriate convex set of mrl functions. In the one-sample problem, the new estimators dominate the empirical mrl function in terms of risk with respect to a wide class of loss functions.     

Bartlett correctable two-sample adjusted empirical likelihood

We propose a two-sample adjusted empirical likelihood (AEL) to construct confidence regions for the difference of two d-dimensional population means. This method eliminates the non-definition of the usual two-sample empirical likelihood (EL) and is shown to be Bartlett correctable. We further show that when the adjustment level is half the Bartlett correction factor for the usual two-sample EL, the two-sample AEL has the same high-order precision as the EL with Bartlett correction. To enhance the performance of the two-sample AEL with adjustment level being half the Bartlett correction factor, we propose a less biased estimate of the Bartlett correction factor. The efficiency of the proposed method is illustrated by simulations and a real data example.     

Asymptotic behavior of central order statistics from stationary processes

In this paper, we show that central order statistics from strictly stationary and ergodic sequences are strongly consistent estimators of population quantiles provided that the quantiles are unique. We generalize this result to strictly stationary but not necessarily ergodic sequences. We also describe three types of possible asymptotic behavior of central order statistics in the case when the corresponding population quantile is not unique. We give applications of the presented results to linear processes with both absolutely continuous and discrete innovations.     

On estimation of a density and its derivatives

Summary Letf _n ^(p) be a recursive kernel estimate off ^(p) thepth order derivative of the probability density functionf, based on a random sample of sizen. In this paper, we provide bounds for the moments of and show that the rate of almost sure convergence of to zero isO(n ^−α), α<(r−p)/(2r+1), iff ^(r),r>p≧0, is a continuousL ₂(−∞, ∞) function. Similar rate-factor is also obtained for the almost sure convergence of to zero under different conditions onf. This work was supported in part by the Research Foundation of SUNY.     

On the distribution of penalized maximum likelihood estimators: The LASSO, SCAD, and thresholding

We study the distributions of the LASSO, SCAD, and thresholding estimators, in finite samples and in the large-sample limit. The asymptotic distributions are derived for both the case where the estimators are tuned to perform consistent model selection and for the case where the estimators are tuned to perform conservative model selection. Our findings complement those of Knight and Fu [K. Knight, W. Fu, Asymptotics for lasso-type estimators, Annals of Statistics 28 (2000) 1356–1378] and Fan and Li [J. Fan, R. Li, Variable selection via non-concave penalized likelihood and its oracle properties, Journal of the American Statistical Association 96 (2001) 1348–1360]. We show that the distributions are typically highly non-normal regardless of how the estimator is tuned, and that this property persists in large samples. The uniform convergence rate of these estimators is also obtained, and is shown to be slower than n^−1/2 in case the estimator is tuned to perform consistent model selection. An impossibility result regarding estimation of the estimators’ distribution function is also provided.     

Weak and universal consistency of moving weighted averages

The properties of weighted averages as linear estimators of a regression function and its derivatives are investigated for the fixed design case. Results on weak consistency and on universal consistency are derived, using a modification of the definition of Stone [10]. As examples we consider kernel estimates and weighted local regression estimators and show that the general results apply.     

High frequency asymptotics for wavelet-based tests for Gaussianity and isotropy on the torus

We prove a multivariate CLT for skewness and kurtosis of the wavelets coefficients of a stationary field on the torus. The results are in the framework of the fixed-domain asymptotics, i.e. we refer to observations of a single field which is sampled at higher and higher frequencies. We consider also studentized statistics for the case of an unknown correlation structure. The results are motivated by the analysis of high-frequency financial data or cosmological data sets, with a particular interest towards testing for Gaussianity and isotropy.     

Second-order accuracy of depth-based bootstrap confidence regions

We consider the problem of setting bootstrap confidence regions for multivariate parameters based on data depth functions. We prove, under mild regularity conditions, that depth-based bootstrap confidence regions are second-order accurate in the sense that their coverage error is of order n⁻¹, given a random sample of size n. The results hold in general for depth functions of types A and D, which cover as special cases the Tukey depth, the majority depth, and the simplicial depth. A simulation study is also provided to investigate empirically the bootstrap confidence regions constructed using these three depth functions.     

On the chernoff-savage theorem for dependent sequences

Summary Given a sequence of ϕ-mixing random variables not necessarily stationary, a Chernoff-Savage theorem for two-sample linear rank statistics is proved using the Pyke-Shorack [5] approach based on weak convergence properties of empirical processes in an extended metric. This result is a generalization of Fears and Mehra [4] in that the stationarity is not required and that the condition imposed on the mixing numbers is substantially relaxed. A similar result is shown to hold for strong mixing sequences under slightly stronger conditions on the mixing numbers. Research partially supported by the National Research Council of Canada under Grant No. A-3954.     

Nonparametric inference under competing risks and selection-biased sampling

The aim of this paper is to carry out statistical inference in a competing risks setup when only selection-biased observation of the data of interest is available. We introduce estimators of the cumulative incidence functions and study their joint large sample behavior.     

Asymptotic properties of Bayes estimators for Gaussian Itô-processes with noisy observations

The estimation of a real parameter θ in a linear stochastic differential equation of the simple type is investigated, based on noisy, time continuous observations of X_t. Sufficient conditions on the continuous functions β and σ are given such that the (conditionally normal) Bayes estimators of θ satisfy certain error bounds and are strongly consistent.