Computational considerations in functional principal component analysis

Computing estimates in functional principal component analysis (FPCA) from discrete data is usually based on the approximation of sample curves in terms of a basis (splines, wavelets, trigonometric functions, etc.) and a geometrical structure in the data space (L ² spaces, Sobolev spaces, etc.). Until now, the computational efforts have been focused in developing ad hoc algorithms to approximate those estimates by previously selecting an efficient approximating technique and a convenient geometrical structure. The main goal of this paper consists of establishing a procedure to formulate the algorithm for computing estimates of FPCA under general settings. The resulting algorithm is based on the classic multivariate PCA of a certain random vector and can thus be implemented in the majority of statistical packages. In fact, it is derived from the analysis of the effects of modifying the norm in the space of coordinates. Finally, an application on real data will be developed to illustrate the so derived theoretic results. This research has been supported by Project MTM2004-5992 from Dirección General de Investigación, Ministerio de Ciencia y Tecnología.     

Tapered Covariance: Bayesian Estimation and Asymptotics

The method of maximum tapered likelihood has been proposed as a way to quickly estimate covariance parameters for stationary Gaussian random fields. We show that under a useful asymptotic regime, maximum tapered likelihood estimators are consistent and asymptotically normal for covariance models in common use. We then formalize the notion of tapered quasi-Bayesian estimators and show that they too are consistent and asymptotically normal. We also present asymptotic confidence intervals for both types of estimators and show via simulation that they accurately reflect sampling variability, even at modest sample sizes. Proofs, an example, and detailed derivations are provided in the supplementary materials, available online.     

A Geometric Approach to Maximum Likelihood Estimation of the Functional Principal Components From Sparse Longitudinal Data

In this article, we consider the problem of estimating the eigenvalues and eigenfunctions of the covariance kernel (i.e., the functional principal components) from sparse and irregularly observed longitudinal data. We exploit the smoothness of the eigenfunctions to reduce dimensionality by restricting them to a lower dimensional space of smooth functions. We then approach this problem through a restricted maximum likelihood method. The estimation scheme is based on a Newton–Raphson procedure on the Stiefel manifold using the fact that the basis coefficient matrix for representing the eigenfunctions has orthonormal columns. We also address the selection of the number of basis functions, as well as that of the dimension of the covariance kernel by a second-order approximation to the leave-one-curve-out cross-validation score that is computationally very efficient. The effectiveness of our procedure is demonstrated by simulation studies and an application to a CD4+ counts dataset. In the simulation studies, our method performs well on both estimation and model selection. It also outperforms two existing approaches: one based on a local polynomial smoothing, and another using an EM algorithm. Supplementary materials including technical details, the R package fpca, and data analyzed by this article are available online.     

Iteratively Reweighted Generalized Least Squares for Estimation and Testing With Correlated Data: An Inference Function Framework

The focus of this article is on fitting regression models and testing of general linear hypotheses for correlated data using quasi-likelihood based techniques. The class of generalized method of moments or GMMs provides an elegant approach for estimating a vector of regression parameters from a set of score functions. Extending the principle of the GMMs, in the generalized estimating equation framework, leads to a quadratic inference function or QIF approach for the analysis of correlated data. We derive an iteratively reweighted generalized least squares or IRGLS algorithm for finding the QIF estimator and establish its convergence properties. A software library implementing the techniques is demonstrated through several datasets.     

Fast Kalman Filtering and Forward–Backward Smoothing via a Low-Rank Perturbative Approach

Kalman filtering-smoothing is a fundamental tool in statistical time-series analysis. However, standard implementations of the Kalman filter-smoother require O(d³) time and O(d²) space per time step, where d is the dimension of the state variable, and are therefore impractical in high-dimensional problems. In this article we note that if a relatively small number of observations are available per time step, the Kalman equations may be approximated in terms of a low-rank perturbation of the prior state covariance matrix in the absence of any observations. In many cases this approximation may be computed and updated very efficiently (often in just O(k²d) or O(k²d + kdlog?d) time and space per time step, where k is the rank of the perturbation and in general k ? d), using fast methods from numerical linear algebra. We justify our approach and give bounds on the rank of the perturbation as a function of the desired accuracy. For the case of smoothing, we also quantify the error of our algorithm because of the low-rank approximation and show that it can be made arbitrarily low at the expense of a moderate computational cost. We describe applications involving smoothing of spatiotemporal neuroscience data. This article has online supplementary material.     

Discrete chaotic calculus and covariance identities

We show that for the binomial process (or Bernoulli random walk) the orthogonal functionals constructed in Kroeker, J.P. (1980) "Wiener analysis of functionals of a Markov chain: application to neural transformations of random signals", Biol. Cybernetics 36 , 243-248, [14] for Markov chains can be expressed using the Krawtchouk polynomials, and by iterated stochastic integrals. This allows to construct a chaotic calculus based on gradient and divergence operators and structure equations, and to establish a Clark representation formula. As an application we obtain simple infinite dimensional proofs of covariance identities on the discrete cube.     

Design of RLS Wiener estimators from randomly delayed observations in linear discrete-time stochastic systems

This paper presents the design of a new recursive least-squares (RLS) Wiener filter and fixed-point smoother based on randomly delayed observed values by one sampling time in linear discrete-time wide-sense stationary stochastic systems. The mixed observed value y(k) consists of the past observed value by one sampling time with the probability p(k) and of the current observed value at time k with the probability 1 − p(k). It is assumed that the delayed measurements are characterized by Bernoulli random variables. The observation is given as the sum of the signal z(k) and the white observation noise v(k). The RLS Wiener estimators explicitly require the following information: (a) the system matrix for the state vector; (b) the observation matrix; (c) the variance of the state vector; (d) the delayed probability p(k); (e) the variance of white observation noise v(k).     

IMPROVED ESTIMATES OF THE COVARIANCE MATRIX IN GENERAL LINEAR MIXED MODELS

In this article, the problem of estimating the covariance matrix in general linear mixed models is considered. Two new classes of estimators obtained by shrinking the eigenvalues towards the origin and the arithmetic mean, respectively, are proposed. It is shown that these new estimators dominate the unbiased estimator under the squared error loss function. Finally, some simulation results to compare the performance of the proposed estimators with that of the unbiased estimator are reported. The simulation results indicate that these new shrinkage estimators provide a substantial improvement in risk under most situations.     

一种低复杂度的稳健自适应波束形成

稳健自适应波束形成可以解决经典自适应波束形成在非理想条件下(如存在方位失配、阵元位置误差、非平面波传播等)性能下降的问题,因而受到广泛关注,但目前多数稳健自适应波束形成方法计算复杂度高,在高信噪比时性能下降。为此,研究提出了一种分别重构干扰协方差矩阵和噪声协方差矩阵的低复杂度的稳健自适应波束形成方法,以减小计算量,提高稳健性。仿真实验表明,该算法能获得近似最优的输出信干噪比和更好的稳健性。     

New dependence coefficients. Examples and applications to statistics

To measure the dependence between a real-valued random variable X and a -algebra , we consider four distances between the conditional distribution function of X given and the distribution function of X. The coefficients obtained are weaker than the corresponding mixing coefficients and may be computed in many situations. In particular, we show that they are well adapted to functions of mixing sequences, iterated random functions and dynamical systems. Starting from a new covariance inequality, we study the mean integrated square error for estimating the unknown marginal density of a stationary sequence. We obtain optimal rates for kernel estimators as well as projection estimators on a well localized basis, under a minimal condition on the coefficients. Using recent results, we show that our coefficients may be also used to obtain various exponential inequalities, a concentration inequality for Lipschitz functions, and a Berry-Esseen type inequality.Mathematics Subject Classification (2000): 62G07, 60J10, 60E15, 37C30