Wavelet methods for continuous-time prediction using Hilbert-valued autoregressive processes

We consider the prediction problem of a continuous-time stochastic process on an entire time-interval in terms of its recent past. The approach we adopt is based on the notion of autoregressive Hilbert processes that represent a generalization of the classical autoregressive processes to random variables with values in a Hilbert space. A careful analysis reveals, in particular, that this approach is related to the theory of function estimation in linear ill-posed inverse problems. In the deterministic literature, such problems are usually solved by suitable regularization techniques. We describe some recent approaches from the deterministic literature that can be adapted to obtain fast and feasible predictions. For large sample sizes, however, these approaches are not computationally efficient.With this in mind, we propose three linear wavelet methods to efficiently address the aforementioned prediction problem. We present regularization techniques for the sample paths of the stochastic process and obtain consistency results of the resulting prediction estimators. We illustrate the performance of the proposed methods in finite sample situations by means of a real-life data example which concerns with the prediction of the entire annual cycle of climatological El Niño-Southern Oscillation time series 1 year ahead. We also compare the resulting predictions with those obtained by other methods available in the literature, in particular with a smoothing spline interpolation method and with a SARIMA model.     

Cramér–Karhunen–Loève representation and harmonic principal component analysis of functional time series

We develop a doubly spectral representation of a stationary functional time series, and study the properties of its empirical version. The representation decomposes the time series into an integral of uncorrelated frequency components (Cramér representation), each of which is in turn expanded in a Karhunen–Loève series. The construction is based on the spectral density operator, the functional analogue of the spectral density matrix, whose eigenvalues and eigenfunctions at different frequencies provide the building blocks of the representation. By truncating the representation at a finite level, we obtain a harmonic principal component analysis of the time series, an optimal finite dimensional reduction of the time series that captures both the temporal dynamics of the process, as well as the within-curve dynamics. Empirical versions of the decompositions are introduced, and a rigorous analysis of their large-sample behaviour is provided, that does not require any prior structural assumptions such as linearity or Gaussianity of the functional time series, but rather hinges on Brillinger-type mixing conditions involving cumulants.     

On same-realization prediction in an infinite-order autoregressive process

Let observations come from an infinite-order autoregressive (AR) process. For predicting the future of the observed time series (referred to as the same-realization prediction), we use the least-squares predictor obtained by fitting a finite-order AR model. We also allow the order to become infinite as the number of observations does in order to obtain a better approximation. Moment bounds for the inverse sample covariance matrix with an increasing dimension are established under various conditions. We then apply these results to obtain an asymptotic expression for the mean-squared prediction error of the least-squares predictor in same-realization and increasing-order settings. The second-order term of this expression is the sum of two terms which measure both the goodness of fit and model complexity. It forms the foundation for a companion paper by Ing and Wei (Order selection for same-realization predictions in autoregressive processes, Technical report C-00-09, Institute of Statistical Science, Academia Sinica, Taipei, Taiwan, ROC, 2000) which provides the first theoretical verification that AIC is asymptotically efficient for same-realization predictions. Finally, some comparisons between the least-squares predictor and the ridge regression predictor are also given.     

Asymptotic properties of computationally efficient alternative estimators for a class of multivariate normal models

Parameters of Gaussian multivariate models are often estimated using the maximum likelihood approach. In spite of its merits, this methodology is not practical when the sample size is very large, as, for example, in the case of massive georeferenced data sets. In this paper, we study the asymptotic properties of the estimators that minimize three alternatives to the likelihood function, designed to increase the computational efficiency. This is achieved by applying the information sandwich technique to expansions of the pseudo-likelihood functions as quadratic forms of independent normal random variables. Theoretical calculations are given for a first-order autoregressive time series and then extended to a two-dimensional autoregressive process on a lattice. We compare the efficiency of the three estimators to that of the maximum likelihood estimator as well as among themselves, using numerical calculations of the theoretical results and simulations.     

Maximum likelihood estimation for all-pass time series models

An autoregressive-moving average model in which all roots of the autoregressive polynomial are reciprocals of roots of the moving average polynomial and vice versa is called an all-pass time series model. All-pass models generate uncorrelated (white noise) time series, but these series are not independent in the non-Gaussian case. An approximate likelihood for a causal all-pass model is given and used to establish asymptotic normality for maximum likelihood estimators under general conditions. Behavior of the estimators for finite samples is studied via simulation. A two-step procedure using all-pass models to identify and estimate noninvertible autoregressive-moving average models is developed and used in the deconvolution of a simulated water gun seismogram.     

Detecting abrupt changes in a piecewise locally stationary time series

Non-stationary time series arise in many settings, such as seismology, speech-processing, and finance. In many of these settings we are interested in points where a model of local stationarity is violated. We consider the problem of how to detect these change-points, which we identify by finding sharp changes in the time-varying power spectrum. Several different methods are considered, and we find that the symmetrized Kullback-Leibler information discrimination performs best in simulation studies. We derive asymptotic normality of our test statistic, and consistency of estimated change-point locations. We then demonstrate the technique on the problem of detecting arrival phases in earthquakes.     

State space models on special manifolds

This paper concerns modeling time series observations in state space forms considered on the Stiefel and Grassmann manifolds. We develop a state space model relating the time series observations to a sequence of unobserved state or parameter matrices assuming the matrix Langevin noise processes on the Stiefel manifolds. We show a Bayes method for estimating the state matrices by the posterior modes. We consider a further extended state space model where two sequences of unobserved state matrices are involved. A simple state space model on the Grassmann manifolds with matrix Langevin noise processes is also investigated.     

Nonparametric time series prediction: A semi-functional partial linear modeling

There is a recent interest in developing new statistical methods to predict time series by taking into account a continuous set of past values as predictors. In this functional time series prediction approach, we propose a functional version of the partial linear model that allows both to consider additional covariates and to use a continuous path in the past to predict future values of the process. The aim of this paper is to present this model, to construct some estimates and to look at their properties both from a theoretical point of view by means of asymptotic results and from a practical perspective by treating some real data sets. Although the literature on the use of parametric or nonparametric functional modeling is growing, as far as we know, this is the first paper on semiparametric functional modeling for the prediction of time series.     

A note on using periodogram-based distances for comparing spectral densities

Motivated by a recent paper of Caiado et al. (2009), we investigate testing problems for spectral densities of time series with unequal sample sizes. We thereby focus on analyzing their mathematical properties and illustrate our results in a small simulation study.     

Robust estimation of periodic autoregressive processes in the presence of additive outliers

This paper suggests a robust estimation procedure for the parameters of the periodic AR (PAR) models when the data contains additive outliers. The proposed robust methodology is an extension of the robust scale and covariance functions given in, respectively, Rousseeuw and Croux (1993) [28], and Ma and Genton (2000) [23] to accommodate periodicity. These periodic robust functions are used in the Yule-Walker equations to obtain robust parameter estimates. The asymptotic central limit theorems of the estimators are established, and an extensive Monte Carlo experiment is conducted to evaluate the performance of the robust methodology for periodic time series with finite sample sizes. The quarterly Fraser River data was used as an example of application of the proposed robust methodology. All the results presented here give strong motivation to use the methodology in practical situations in which periodically correlated time series contain additive outliers.     

Log-linear Poisson autoregression

We consider a log-linear model for time series of counts. This type of model provides a framework where both negative and positive association can be taken into account. In addition time dependent covariates are accommodated in a straightforward way. We study its probabilistic properties and maximum likelihood estimation. It is shown that a perturbed version of the process is geometrically ergodic, and, under some conditions, it approaches the non-perturbed version. In addition, it is proved that the maximum likelihood estimator of the vector of unknown parameters is asymptotically normal with a covariance matrix that can be consistently estimated. The results are based on minimal assumptions and can be extended to the case of log-linear regression with continuous exogenous variables. The theory is applied to aggregated financial transaction time series. In particular, we discover positive association between the number of transactions and the volatility process of a certain stock.     

Characterization of the partial autocorrelation function of nonstationary time series

The second order properties of a process are usually characterized by the autocovariance function. In the stationary case, the parameterization by the partial autocorrelation function is relatively recent. We extend this parameterization to the nonstationary case. The advantage of this function is that it is subject to very simple constraints in comparison with the auto- covariance function which must be nonnegative definite. As in the stationary case, this parameterization is well adapted to autoregressive models or to the identification of deterministic processes.     

Shrinkage estimation in the frequency domain of multivariate time series

In this paper on developing shrinkage for spectral analysis of multivariate time series of high dimensionality, we propose a new nonparametric estimator of the spectral matrix with two appealing properties. First, compared to the traditional smoothed periodogram our shrinkage estimator has a smaller L₂ risk. Second, the proposed shrinkage estimator is numerically more stable due to a smaller condition number. We use the concept of “Kolmogorov” asymptotics where simultaneously the sample size and the dimensionality tend to infinity, to show that the smoothed periodogram is not consistent and to derive the asymptotic properties of our regularized estimator. This estimator is shown to have asymptotically minimal risk among all linear combinations of the identity and the averaged periodogram matrix. Compared to existing work on shrinkage in the time domain, our results show that in the frequency domain it is necessary to take the size of the smoothing span as “effective sample size” into account. Furthermore, we perform extensive Monte Carlo studies showing the overwhelming gain in terms of lower L₂ risk of our shrinkage estimator, even in situations of oversmoothing the periodogram by using a large smoothing span.     

Moving estimates test with time varying bandwidth

In this paper, we consider the problem of testing for parameter changes in time series models based on a moving estimates (ME) test. It is widely accepted that detecting some changes, for instance, those caused by temporary parameter shifts by the existing cusum test is difficult. A MV test with a fixed bandwidth has been developed to circumvent the defect, but the test still does not perform well under certain conditions. Motivated by this, we propose a MV test with a time varying bandwidth to outperform the original test. In order to illustrate our findings, we have provided simulation results.     

Autoregressive process modeling via the Lasso procedure

The Lasso is a popular model selection and estimation procedure for linear models that enjoys nice theoretical properties. In this paper, we study the Lasso estimator for fitting autoregressive time series models. We adopt a double asymptotic framework where the maximal lag may increase with the sample size. We derive theoretical results establishing various types of consistency. In particular, we derive conditions under which the Lasso estimator for the autoregressive coefficients is model selection consistent, estimation consistent and prediction consistent. Simulation study results are reported.     

Linear relations in time series models. II

In this paper an asymptotic theory is developed for a new time series model which was introduced in a previous paper [5]. An algorithm for computing estimates of the parameters of this time series model is given, and it is shown that these estimators are asymptotically efficient in the sense that they have the same asymptotic distribution as the maximum likelihood estimators.     

Testing the stability of the functional autoregressive process

The functional autoregressive process has become a useful tool in the analysis of functional time series data. It is defined by the equation , in which the observations X_n and errors ε_n are curves, and is an operator. To ensure meaningful inference and prediction based on this model, it is important to verify that the operator does not change with time. We propose a method for testing the constancy of against a change-point alternative which uses the functional principal component analysis. The test statistic is constructed to have a well-known asymptotic distribution, but the asymptotic justification of the procedure is very delicate. We develop a new truncation approach which together with Mensov’s inequality can be used in other problems of functional time series analysis. The estimation of the principal components introduces asymptotically non-negligible terms, which however cancel because of the special form of our test statistic (CUSUM type). The test is implemented using the R package fda, and its finite sample performance is examined by application to credit card transaction data.     

Approximating posterior probabilities in a linear model with possibly noninvertible moving average errors

The method of Laplace is used to approximate posterior probabilities for a collection of polynomial regression models when the errors follow a process with a noninvertible moving average component. These results are useful in the problem of period-change analysis of variable stars and in assessing the posterior probability that a time series with trend has been overdifferenced. The nonstandard covariance structure induced by a noninvertible moving average process can invalidate the standard Laplace method. A number of analytical tools is used to produce corrected Laplace approximations. These tools include viewing the covariance matrix of the observations as tending to a differential operator. The use of such an operator and its Green's function provides a convenient and systematic method of asymptotically inverting the covariance matrix.In certain cases there are two different Laplace approximations, and the appropriate one to use depends upon unknown parameters. This problem is dealt with by using a weighted geometric mean of the candidate approximations, where the weights are completely data-based and such that, asymptotically, the correct approximation is used. The new methodology is applied to an analysis of the prototypical long-period variable star known as Mira.     

On Nummelin splitting for continuous time Harris recurrent Markov processes and application to kernel estimation for multi-dimensional diffusions

We introduce a sequence of stopping times that allow us to study an analogue of a life-cycle decomposition for a continuous time Markov process, which is an extension of the well-known splitting technique of Nummelin to the continuous time case. As a consequence, we are able to give deterministic equivalents of additive functionals of the process and to state a generalisation of Chen’s inequality. We apply our results to the problem of non-parametric kernel estimation of the drift of multi-dimensional recurrent, but not necessarily ergodic, diffusion processes.     

Weak convergence of non-stationary multivariate marked processes with applications to martingale testing

This paper establishes the weak convergence of a class of marked empirical processes of possibly non-stationary and/or non-ergodic multivariate time series sequences under martingale conditions. The assumptions involved are similar to those in Brown's martingale central limit theorem. In particular, no mixing conditions are imposed. As an application, we propose a test statistic for the martingale hypothesis and we derive its asymptotic null distribution. Finally, a Monte Carlo study shows that the asymptotic results provide good approximations for small and moderate sample sizes. An application to the S&P 500 is also considered.