Semiparametric estimation in regression with missing covariates using single-index models

We investigate semiparametric estimation of regression coefficients through generalized estimating equations with single-index models when some covariates are missing at random. Existing popular semiparametric estimators may run into difficulties when some selection probabilities are small or the dimension of the covariates is not low. We propose a new simple parameter estimator using a kernel-assisted estimator for the augmentation by a single-index model without using the inverse of selection probabilities. We show that under certain conditions the proposed estimator is as efficient as the existing methods based on standard kernel smoothing, which are often practically infeasible in the case of multiple covariates. A simulation study and a real data example are presented to illustrate the proposed method. The numerical results show that the proposed estimator avoids some numerical issues caused by estimated small selection probabilities that are needed in other estimators.

     

Asymptotic theory of the adaptive Sparse Group Lasso

We study the asymptotic properties of a new version of the Sparse Group Lasso estimator (SGL), called adaptive SGL. This new version includes two distinct regularization parameters, one for the Lasso penalty and one for the Group Lasso penalty, and we consider the adaptive version of this regularization, where both penalties are weighted by preliminary random coefficients. The asymptotic properties are established in a general framework, where the data are dependent and the loss function is convex. We prove that this estimator satisfies the oracle property: the sparsity-based estimator recovers the true underlying sparse model and is asymptotically normally distributed. We also study its asymptotic properties in a double-asymptotic framework, where the number of parameters diverges with the sample size. We show by simulations and on real data that the adaptive SGL outperforms other oracle-like methods in terms of estimation precision and variable selection.

     

Bootstrapping the Kaplan–Meier estimator on the whole line

This article is concerned with proving the consistency of Efron’s bootstrap for the Kaplan–Meier estimator on the whole support of a survival function. While previous works address the asymptotic Gaussianity of the Kaplan–Meier estimator without restricting time, we enable the construction of bootstrap-based time-simultaneous confidence bands for the whole survival function. Other practical applications include bootstrap-based confidence bands for the mean residual lifetime function or the Lorenz curve as well as confidence intervals for the Gini index. Theoretical results are complemented with a simulation study and a real data example which result in statistical recommendations.

     

Simultaneous confidence bands for nonparametric regression with missing covariate data

We consider a weighted local linear estimator based on the inverse selection probability for nonparametric regression with missing covariates at random. The asymptotic distribution of the maximal deviation between the estimator and the true regression function is derived and an asymptotically accurate simultaneous confidence band is constructed. The estimator for the regression function is shown to be oracally efficient in the sense that it is uniformly indistinguishable from that when the selection probabilities are known. Finite sample performance is examined via simulation studies which support our asymptotic theory. The proposed method is demonstrated via an analysis of a data set from the Canada 2010/2011 Youth Student Survey.

     

Estimating and Visualizing Conditional Densities

Abstract

We consider the kernel estimator of conditional density and derive its asymptotic bias, variance, and mean-square error. Optimal bandwidths (with respect to integrated mean-square error) are found and it is shown that the convergence rate of the density estimator is order n ^–2/3. We also note that the conditional mean function obtained from the estimator is equivalent to a kernel smoother. Given the undesirable bias properties of kernel smoothers, we seek a modified conditional density estimator that has mean equivalent to some other nonparametric regression smoother with better bias properties. It is also shown that our modified estimator has smaller mean square error than the standard estimator in some commonly occurring situations. Finally, three graphical methods for visualizing conditional density estimators are discussed and applied to a data set consisting of maximum daily temperatures in Melbourne, Australia.     

A Computationally Efficient Oracle Estimator for Additive Nonparametric Regression with Bootstrap Confidence Intervals

Abstract

This article makes three contributions. First, we introduce a computationally efficient estimator for the component functions in additive nonparametric regression exploiting a different motivation from the marginal integration estimator of Linton and Nielsen. Our method provides a reduction in computation of order n which is highly significant in practice. Second, we define an efficient estimator of the additive components, by inserting the preliminary estimator into a backfitting˙ algorithm but taking one step only, and establish that it is equivalent, in various senses, to the oracle estimator based on knowing the other components. Our two-step estimator is minimax superior to that considered in Opsomer and Ruppert, due to its better bias. Third, we define a bootstrap algorithm for computing pointwise confidence intervals and show that it achieves the correct coverage.     

Error density estimation in high-dimensional sparse linear model

This paper is concerned with the error density estimation in high-dimensional sparse linear model, where the number of variables may be larger than the sample size. An improved two-stage refitted cross-validation procedure by random splitting technique is used to obtain the residuals of the model, and then traditional kernel density method is applied to estimate the error density. Under suitable sparse conditions, the large sample properties of the estimator including the consistency and asymptotic normality, as well as the law of the iterated logarithm are obtained. Especially, we gave the relationship between the sparsity and the convergence rate of the kernel density estimator. The simulation results show that our error density estimator has a good performance. A real data example is presented to illustrate our methods.

     

Nonparametric estimation of the cross ratio function

The cross ratio function (CRF) is a commonly used tool to describe local dependence between two correlated variables. Being a ratio of conditional hazards, the CRF can be rewritten in terms of (first and second derivatives of) the survival copula of these variables. Bernstein estimators for (the derivatives of) this survival copula are used to define a nonparametric estimator of the cross ratio, and asymptotic normality thereof is established. We consider simulations to study the finite sample performance of our estimator for copulas with different types of local dependency. A real dataset is used to investigate the dependence between food expenditure and net income. The estimated CRF reveals that families with a low net income relative to the mean net income will spend less money to buy food compared to families with larger net incomes. This dependence, however, disappears when the net income is large compared to the mean income.

     

Improvements of the Maximum Pseudo-Likelihood Estimators in Various Spatial Statistical Models

Abstract

Maximum pseudo-likelihood estimation has hitherto been viewed as a practical but flawed alternative to maximum likelihood estimation, necessary because the maximum likelihood estimator is too hard to compute, but flawed because of its inefficiency when the spatial interactions are strong. We demonstrate that a single Newton-Raphson step starting from the maximum pseudo-likelihood estimator produces an estimator which is close to the maximum likelihood estimator in terms of its actual value, attained likelihood, and efficiency, even in the presence of strong interactions. This hybrid technique greatly increases the practical applicability of pseudo-likelihood-based estimation. Additionally, in the case of the spatial point processes, we propose a proper maximum pseudo-likelihood estimator which is different from the conventional one. The proper maximum pseudo-likelihood estimator clearly shows better performance than the conventional one does when the spatial interactions are strong.     

RCV-based error density estimation in the ultrahigh dimensional additive model

In this paper, we mainly study how to estimate the error density in the ultrahigh dimensional sparse additive model, where the number of variables is larger than the sample size. First, a smoothing method based on B-splines is applied to the estimation of regression functions. Second, an improved two-stage refitted cross-validation (RCV) procedure by random splitting technique is used to obtain the residuals of the model, and then the residual-based kernel method is applied to estimate the error density function. Under suitable sparse conditions, the large sample properties of the estimator including the weak and strong consistency, as well as normality and the law of the iterated logarithm are obtained. Especially, the relationship between the sparsity and the convergence rate of the kernel density estimator is given. The methodology is illustrated by simulations and a real data example, which suggests that the proposed method performs well.

     

An accurate a posteriori error estimator for the Steklov eigenvalue problem and its applications

In this paper, a type of accurate a posteriori error estimator is proposed for the Steklov eigenvalue problem based on the complementary approach, which provides an asymptotic exact estimate for the approximate eigenpair. Besides, we design a type of cascadic adaptive finite element method for the Steklov eigenvalue problem based on the proposed a posteriori error estimator. In this new cascadic adaptive scheme, instead of solving the Steklov eigenvalue problem in each adaptive space directly, we only need to do some smoothing steps for linearized boundary value problems on a series of adaptive spaces and solve some Steklov eigenvalue problems on a low dimensional space. Furthermore, the proposed a posteriori error estimator provides the way to refine mesh and control the number of smoothing steps for the cascadic adaptive method. Some numerical examples are presented to validate the efficiency of the algorithm in this paper.

     

Estimation of the position and time of emission of a source

We consider the problem of the estimation of the position on the plane and of the moment of beginning of emission of a source by observations from K detectors on the plane. We propose the conditions of regularity and identifiability, which allow us to use two general theorems by Ibragimov and Khasminskii and to describe the asymptotic behavior of the maximum likelihood and Bayes estimators. Then we propose the construction of a linear estimator of unknown parameters and study its properties in slightly more general situation. Special attention is payed to condition of identifiability.

     

Poisson source localization on the plane: change-point case

We present a detection problem where several spatially distributed sensors observe Poisson signals emitted from a single radioactive source of unknown position. The measurements at each sensor are modeled by independent inhomogeneous Poisson processes. A method based on Bayesian change-point estimation is proposed to identify the location of the source’s coordinates. The asymptotic behavior of the Bayesian estimator is studied. In particular, the consistency and the asymptotic efficiency of the estimator are analyzed. The limit distribution and the convergence of the moments are also described. The similar statistical model could be used in GPS localization problems.

     

Particle-based online estimation of tangent filters with application to parameter estimation in nonlinear state-space models

This paper presents a novel algorithm for efficient online estimation of the filter derivatives in general hidden Markov models. The algorithm, which has a linear computational complexity and very limited memory requirements, is furnished with a number of convergence results, including a central limit theorem with an asymptotic variance that can be shown to be uniformly bounded in time. Using the proposed filter derivative estimator, we design a recursive maximum likelihood algorithm updating the parameters according the gradient of the one-step predictor log-likelihood. The efficiency of this online parameter estimation scheme is illustrated in a simulation study.

     

Parametric estimation for linear stochastic differential equations driven by mixed fractional Brownian motion

ABSTRACT

We investigate the asymptotic properties of the maximum likelihood estimator and Bayes estimator of the drift parameter for stochastic processes satisfying linear stochastic differential equations driven by a mixed fractional Brownian motion. We obtain a Bernstein–von Mises-type theorem also for such a class of processes.     

New kernel estimators of the hazard ratio and their asymptotic properties

We propose a kernel estimator of a hazard ratio that is based on a modification of Ćwik and Mielniczuk (Commun Stat-Theory Methods 18(8):3057–3069, 1989)’s method. A naive nonparametric estimator is Watson and Leadbetter (Sankhyā: Indian J Stat Ser A 26(1):101–116, 1964)’s one, which is naturally given by the kernel density estimator and the empirical distribution estimator. We compare the asymptotic mean squared error (AMSE) of the hazard estimators, and then, it is shown that the asymptotic variance of the new estimator is usually smaller than that of the naive one. We also discuss bias reduction of the proposed estimator and derived some modified estimators. While the modified estimators do not lose nonnegativity, their AMSE is small both theoretically and numerically.

     

Marginal quantile regression for varying coefficient models with longitudinal data

In this paper, we investigate the quantile varying coefficient model for longitudinal data, where the unknown nonparametric functions are approximated by polynomial splines and the estimators are obtained by minimizing the quadratic inference function. The theoretical properties of the resulting estimators are established, and they achieve the optimal convergence rate for the nonparametric functions. Since the objective function is non-smooth, an estimation procedure is proposed that uses induced smoothing and we prove that the smoothed estimator is asymptotically equivalent to the original estimator. Moreover, we propose a variable selection procedure based on the regularization method, which can simultaneously estimate and select important nonparametric components and has the asymptotic oracle property. Extensive simulations and a real data analysis show the usefulness of the proposed method.

     

Nonparametric regression with warped wavelets and strong mixing processes

We consider the situation of a univariate nonparametric regression where either the Gaussian error or the predictor follows a stationary strong mixing stochastic process and the other term follows an independent and identically distributed sequence. Also, we estimate the regression function by expanding it in a wavelet basis and applying a hard threshold to the coefficients. Since the observations of the predictor are unequally distant from each other, we work with wavelets warped by the density of the predictor variable. This choice enables us to retain some theoretical and computational properties of wavelets. We propose a unique estimator and show that some of its properties are the same for both model specifications. Specifically, in both cases the coefficients are unbiased and their variances decay at the same rate. Also the risk of the estimator, measured by the mean integrated square error is almost minimax and its maxiset remains unaltered. Simulations and an application illustrate the similarities and differences of the proposed estimator in both situations.

     

Parametric estimation for cusp-type signal driven by fractional Brownian motion

Abstract

We investigate the asymptotic properties of the maximum likelihood estimator of the drift parameter in a cusp-type signal driven by a fractional Brownian motion.     

Data driven time scale in Gaussian quasi-likelihood inference

We study parametric estimation of ergodic diffusions observed at high frequency. Different from the previous studies, we suppose that sampling stepsize is unknown, thereby making the conventional Gaussian quasi-likelihood not directly applicable. In this situation, we construct estimators of both model parameters and sampling stepsize in a fully explicit way, and prove that they are jointly asymptotically normally distributed. High order uniform integrability of the obtained estimator is also derived. Further, we propose the Schwarz (BIC) type statistics for model selection and show its model-selection consistency. We conducted some numerical experiments and found that the observed finite-sample performance well supports our theoretical findings.