Estimation of the regression operator from functional fixed-design with correlated errors

We consider the estimation of the regression operator r in the functional model: Y=r(x)+ε, where the explanatory variable x is of functional fixed-design type, the response Y is a real random variable and the error process ε is a second order stationary process. We construct the kernel type estimate of r from functional data curves and correlated errors. Then we study their performances in terms of the mean square convergence and the convergence in probability. In particular, we consider the cases of short and long range error processes. When the errors are negatively correlated or come from a short memory process, the asymptotic normality of this estimate is derived. Finally, some simulation studies are conducted for a fractional autoregressive integrated moving average and for an Ornstein-Uhlenbeck error processes.     

A profile-type smoothed score function for a varying coefficient partially linear model

The varying coefficient partially linear model is considered in this paper. When the plug-in estimators of coefficient functions are used, the resulting smoothing score function becomes biased due to the slow convergence rate of nonparametric estimations. To reduce the bias of the resulting smoothing score function, a profile-type smoothed score function is proposed to draw inferences on the parameters of interest without using the quasi-likelihood framework, the least favorable curve, a higher order kernel or under-smoothing. The resulting profile-type statistic is still asymptotically Chi-squared under some regularity conditions. The results are then used to construct confidence regions for the parameters of interest. A simulation study is carried out to assess the performance of the proposed method and to compare it with the profile least-squares method. A real dataset is analyzed for illustration.     

Curve forecasting by functional autoregression

This paper deals with the prediction of curve-valued autoregression processes. It develops a novel technique, predictive factor decomposition, for the estimation of the autoregression operator. The technique is based on finding a reduced-rank approximation to the autoregression operator that minimizes the expected squared norm of the prediction error.Implementing this idea, we relate the operator approximation problem to the singular value decomposition of a combination of cross-covariance and covariance operators. We develop an estimation method based on regularization of the empirical counterpart of this singular value decomposition, prove its consistency and evaluate convergence rates.The method is illustrated by an example of the term structure of the Eurodollar futures rates. In the sample corresponding to the period of normal growth, the predictive factor technique outperforms the principal components method and performs on a par with custom-designed prediction methods.     

Cramér–Karhunen–Loève representation and harmonic principal component analysis of functional time series

We develop a doubly spectral representation of a stationary functional time series, and study the properties of its empirical version. The representation decomposes the time series into an integral of uncorrelated frequency components (Cramér representation), each of which is in turn expanded in a Karhunen–Loève series. The construction is based on the spectral density operator, the functional analogue of the spectral density matrix, whose eigenvalues and eigenfunctions at different frequencies provide the building blocks of the representation. By truncating the representation at a finite level, we obtain a harmonic principal component analysis of the time series, an optimal finite dimensional reduction of the time series that captures both the temporal dynamics of the process, as well as the within-curve dynamics. Empirical versions of the decompositions are introduced, and a rigorous analysis of their large-sample behaviour is provided, that does not require any prior structural assumptions such as linearity or Gaussianity of the functional time series, but rather hinges on Brillinger-type mixing conditions involving cumulants.     

Functional semiparametric partially linear model with autoregressive errors

In this paper, we introduce a functional semiparametric model, where a real-valued random variable is explained by the sum of a unknown linear combination of the components of a multivariate random variable and an unknown transformation of a functional random variable. The errors can be autocorrelated. We focus here on the parametric estimation of the coefficients in the linear combination. First, we use a nonparametric kernel method to remove the effect of the functional explanatory variable. Then, we use generalized least squares approach to obtain an estimator of these coefficients. Under some technical assumptions, we prove consistency and asymptotic normality of our estimator. Finally, we present Monte Carlo simulations that illustrate these characteristics.     

Sensitivity analysis in functional principal component analysis

Summary  In the present paper empirical influence functions (EIFs) are derived for eigenvalues and eigenfunctions in functional principal component analysis in both cases where the smoothing parameter is fixed and unfixed. Based on the derived influence functions a sensitivity analysis procedure is proposed for detecting jointly as well as singly influential observations. A numerical example is given to show the usefulness of the proposed procedure. In dealing with the influence on the eigenfunctions two different kinds of influence statistics are introduced. One is based on the EIF for the coefficient vectors of the basis function expansion, and the other is based on the sampled vectors of the functional EIF. Under a certain condition it can be proved both kinds of statistics provide essentially equivalent results.     

Thresholding projection estimators in functional linear models

We consider the problem of estimating the regression function in functional linear regression models by proposing a new type of projection estimators which combine dimension reduction and thresholding. The introduction of a threshold rule allows us to get consistency under broad assumptions as well as minimax rates of convergence under additional regularity hypotheses. We also consider the particular case of Sobolev spaces generated by the trigonometric basis which permits us to get easily mean squared error of prediction as well as estimators of the derivatives of the regression function. We prove that these estimators are minimax and rates of convergence are given for some particular cases.     

Functional nonparametric estimation of conditional extreme quantiles

We address the estimation of quantiles from heavy-tailed distributions when functional covariate information is available and in the case where the order of the quantile converges to one as the sample size increases. Such “extreme” quantiles can be located in the range of the data or near and even beyond the boundary of the sample, depending on the convergence rate of their order to one. Nonparametric estimators of these functional extreme quantiles are introduced, their asymptotic distributions are established and their finite sample behavior is investigated.     

Empirical likelihood based confidence intervals for copulas

Copula as an effective way of modeling dependence has become more or less a standard tool in risk management, and a wide range of applications of copula models appear in the literature of economics, econometrics, insurance, finance, etc. How to estimate and test a copula plays an important role in practice, and both parametric and nonparametric methods have been studied in the literature. In this paper, we focus on interval estimation and propose an empirical likelihood based confidence interval for a copula. A simulation study and a real data analysis are conducted to compare the finite sample behavior of the proposed empirical likelihood method with the bootstrap method based on either the empirical copula estimator or the kernel smoothing copula estimator.     

Methods for tracking support boundaries with corners

In a range of practical problems the boundary of the support of a bivariate distribution is of interest, for example where it describes a limit to efficiency or performance, or where it determines the physical extremities of a spatially distributed population in forestry, marine science, medicine, meteorology or geology. We suggest a tracking-based method for estimating a support boundary when it is composed of a finite number of smooth curves, meeting together at corners. The smooth parts of the boundary are assumed to have continuously turning tangents and bounded curvature, and the corners are not allowed to be infinitely sharp; that is, the angle between the two tangents should not equal π. In other respects, however, the boundary may be quite general. In particular it need not be uniquely defined in Cartesian coordinates, its corners my be either concave or convex, and its smooth parts may be neither concave nor convex. Tracking methods are well suited to such generalities, and they also have the advantage of requiring relatively small amounts of computation. It is shown that they achieve optimal convergence rates, in the sense of uniform approximation.     

Asymptotic property and convergence estimation for the eigenelements of the Laplace operator

In this article, we provide a rigorous derivation of asymptotic expansions for eigenfunctions and we establish convergence estimation for both eigenvalues and eigenfunctions of the Laplacian. We address the integral equation method to investigate the interplay between the geometry, boundary conditions and spectral properties of the eigenelements of the Laplace operator under deformation of the domain. The asymptotic formula and convergence estimation are tested by numerical examples.     

Strong convergence in nonparametric regression with truncated dependent data

In this paper we derive rates of uniform strong convergence for the kernel estimator of the regression function in a left-truncation model. It is assumed that the lifetime observations with multivariate covariates form a stationary α-mixing sequence. The estimation of the covariate’s density is considered as well. Under the assumption that the lifetime observations are bounded, we show that, by an appropriate choice of the bandwidth, both estimators of the covariate’s density and regression function attain the optimal strong convergence rate known from independent complete samples.     

A Berry-Esseen type bound in kernel density estimation for strong mixing censored samples

In this paper, we discuss the estimation of a density function based on censored data by the kernel smoothing method when the survival and the censoring times form a stationary α-mixing sequence. A Berry-Esseen type bound is derived for the kernel density estimator at a fixed point x. For practical purposes, a randomly weighted estimator of the density function is also constructed and investigated.     

On rates of convergence in functional linear regression

This paper investigates the rate of convergence of estimating the regression weight function in a functional linear regression model. It is assumed that the predictor as well as the weight function are smooth and periodic in the sense that the derivatives are equal at the boundary points. Assuming that the functional data are observed at discrete points with measurement error, the complex Fourier basis is adopted in estimating the true data and the regression weight function based on the penalized least-squares criterion. The rate of convergence is then derived for both estimators. A simulation study is also provided to illustrate the numerical performance of our approach, and to make a comparison with the principal component regression approach.     

CONVERGENCE OF LAPLACIAN SPECTRA FROM RANDOM SAMPLES

Eigenvectors and eigenvalues of discrete Laplacians are often used for manifold learning and nonlinear dimensionality reduction. Graph Laplacian is one widely used discrete laplacian on point cloud. It was previously proved by Belkin and Niyogithat the eigenvectors and eigenvalues of the graph Laplacian converge to the eigenfunctions and eigenvalues of the Laplace-Beltrami operator of the manifold in the limit of infinitely many data points sampled independently from the uniform distribution over the manifold. Recently, we introduced Point Integral method (PIM) to solve elliptic equations and corresponding eigenvalue problem on point clouds. In this paper, we prove that the eigenvectors and eigenvalues obtained by PIM converge in the limit of infinitely many random samples. Moreover, estimation of the convergence rate is also given.     

A correction method for finding lower bounds of eigenvalues of the second‐order elliptic and Stokes operators

In this paper, for the second‐order elliptic and Stokes eigenvalue problems with variable coefficients, we propose a correction method to nonconforming eigenvalue approximations and prove that the corrected eigenvalues converge to the exact ones asymptotically from below. In particular, the asymptotic lower bound property of corrected eigenvalues is always valid whether the eigenfunctions are smooth or singular. Finally, we prove that the convergence order of corrected eigenvalues is still the same as that of uncorrected eigenvalues.     

A Geometric Approach to Maximum Likelihood Estimation of the Functional Principal Components From Sparse Longitudinal Data

In this article, we consider the problem of estimating the eigenvalues and eigenfunctions of the covariance kernel (i.e., the functional principal components) from sparse and irregularly observed longitudinal data. We exploit the smoothness of the eigenfunctions to reduce dimensionality by restricting them to a lower dimensional space of smooth functions. We then approach this problem through a restricted maximum likelihood method. The estimation scheme is based on a Newton–Raphson procedure on the Stiefel manifold using the fact that the basis coefficient matrix for representing the eigenfunctions has orthonormal columns. We also address the selection of the number of basis functions, as well as that of the dimension of the covariance kernel by a second-order approximation to the leave-one-curve-out cross-validation score that is computationally very efficient. The effectiveness of our procedure is demonstrated by simulation studies and an application to a CD4+ counts dataset. In the simulation studies, our method performs well on both estimation and model selection. It also outperforms two existing approaches: one based on a local polynomial smoothing, and another using an EM algorithm. Supplementary materials including technical details, the R package fpca, and data analyzed by this article are available online.     

Variable selection for semiparametric varying coefficient partially linear errors-in-variables models

This paper focuses on the variable selections for semiparametric varying coefficient partially linear models when the covariates in the parametric and nonparametric components are all measured with errors. A bias-corrected variable selection procedure is proposed by combining basis function approximations with shrinkage estimations. With appropriate selection of the tuning parameters, the consistency of the variable selection procedure and the oracle property of the regularized estimators are established. A simulation study and a real data application are undertaken to evaluate the finite sample performance of the proposed method.     

Near-optimal stochastic approximation for online principal component estimation

Principal component analysis (PCA) has been a prominent tool for high-dimensional data analysis. Online algorithms that estimate the principal component by processing streaming data are of tremendous practical and theoretical interests. Despite its rich applications, theoretical convergence analysis remains largely open. In this paper, we cast online PCA into a stochastic nonconvex optimization problem, and we analyze the online PCA algorithm as a stochastic approximation iteration. The stochastic approximation iteration processes data points incrementally and maintains a running estimate of the principal component. We prove for the first time a nearly optimal finite-sample error bound for the online PCA algorithm. Under the subgaussian assumption, we show that the finite-sample error bound closely matches the minimax information lower bound.