Wavelet methods for continuous-time prediction using Hilbert-valued autoregressive processes

We consider the prediction problem of a continuous-time stochastic process on an entire time-interval in terms of its recent past. The approach we adopt is based on the notion of autoregressive Hilbert processes that represent a generalization of the classical autoregressive processes to random variables with values in a Hilbert space. A careful analysis reveals, in particular, that this approach is related to the theory of function estimation in linear ill-posed inverse problems. In the deterministic literature, such problems are usually solved by suitable regularization techniques. We describe some recent approaches from the deterministic literature that can be adapted to obtain fast and feasible predictions. For large sample sizes, however, these approaches are not computationally efficient.With this in mind, we propose three linear wavelet methods to efficiently address the aforementioned prediction problem. We present regularization techniques for the sample paths of the stochastic process and obtain consistency results of the resulting prediction estimators. We illustrate the performance of the proposed methods in finite sample situations by means of a real-life data example which concerns with the prediction of the entire annual cycle of climatological El Niño-Southern Oscillation time series 1 year ahead. We also compare the resulting predictions with those obtained by other methods available in the literature, in particular with a smoothing spline interpolation method and with a SARIMA model.     

Model selection by sequentially normalized least squares

Model selection by means of the predictive least squares (PLS) principle has been thoroughly studied in the context of regression model selection and autoregressive (AR) model order estimation. We introduce a new criterion based on sequentially minimized squared deviations, which are smaller than both the usual least squares and the squared prediction errors used in PLS. We also prove that our criterion has a probabilistic interpretation as a model which is asymptotically optimal within the given class of distributions by reaching the lower bound on the logarithmic prediction errors, given by the so called stochastic complexity, and approximated by BIC. This holds when the regressor (design) matrix is non-random or determined by the observed data as in AR models. The advantages of the criterion include the fact that it can be evaluated efficiently and exactly, without asymptotic approximations, and importantly, there are no adjustable hyper-parameters, which makes it applicable to both small and large amounts of data.     

Departure from normality of increasing-dimension martingales

In this paper, we consider sequences of vector martingale differences of increasing dimension. We show that the Kantorovich distance from the distribution of the k(n)-dimensional average of n martingale differences to the corresponding Gaussian distribution satisfies certain inequalities. As a consequence, if the growth of k(n) is not too fast, then the Kantorovich distance converges to zero. Two applications of this result are presented. The first is a precise proof of the asymptotic distribution of the multivariate portmanteau statistic applied to the residuals of an autoregressive model and the second is a proof of the asymptotic normality of the estimates of a finite autoregressive model when the process is an AR(∞) and the order of the model grows with the length of the series.     

Generalized Levinson-Durbin sequences, binomial coefficients and autoregressive estimation

For a discrete time second-order stationary process, the Levinson-Durbin recursion is used to determine the coefficients of the best linear predictor of the observation at time k+1, given k previous observations, best in the sense of minimizing the mean square error. The coefficients determined by the recursion define a Levinson-Durbin sequence. We also define a generalized Levinson-Durbin sequence and note that binomial coefficients form a special case of a generalized Levinson-Durbin sequence. All generalized Levinson-Durbin sequences are shown to obey summation formulas which generalize formulas satisfied by binomial coefficients. Levinson-Durbin sequences arise in the construction of several autoregressive model coefficient estimators. The least squares autoregressive estimator does not give rise to a Levinson-Durbin sequence, but least squares fixed point processes, which yield least squares estimates of the coefficients unbiased to order 1/T, where T is the sample length, can be combined to construct a Levinson-Durbin sequence. By contrast, analogous fixed point processes arising from the Yule-Walker estimator do not combine to construct a Levinson-Durbin sequence, although the Yule-Walker estimator itself does determine a Levinson-Durbin sequence. The least squares and Yule-Walker fixed point processes are further studied when the mean of the process is a polynomial time trend that is estimated by least squares.     

Efficiencies of tests and estimators forp-order autoregressive processes when the error distribution is nonnormal

Summary We considerpth order autoregressive time series where the shocks need not be normal. By employing the concept of contiguity, we obtain the sysmptotic power for tests of hypothesis concerning the autoregressive parameters. Our approach allows consideration of the double exponential and other thicker-tailed distributions for the shocks. We derive a new result in the contiguity framework that leads directly to an expression for the Pitman efficiencies of tests as well as estimators. The numerical values of the efficiencies suggest a lack of robustness for the normal theory least squares estimators when the shock distribution is thick tailed or an outlier prone mixed normal. An important alternative test statistic is proposed that competes with the normal theory tests. This research was supported by the Office of Naval Research under Grant No. N00014-78-C-0722 and by the Army Research Office.     

Spatial autoregression and related spatio-temporal models

We propose a spatial autoregressive random field of order p on the spatial domain for p?2 in this paper, whose univariate margins are the continuous-time autoregression of order p on the real line, and introduce a class of semiparametric spatio-temporal covariance models stationary in space with the spatial autoregressive margin.     

Robust Bayesian prediction and estimation under a squared log error loss function

Robust Bayesian analysis is concerned with the problem of making decisions about some future observation or an unknown parameter, when the prior distribution belongs to a class Γ instead of being specified exactly. In this paper, the problem of robust Bayesian prediction and estimation under a squared log error loss function is considered. We find the posterior regret Γ-minimax predictor and estimator in a general class of distributions. Furthermore, we construct the conditional Γ-minimax, most stable and least sensitive prediction and estimation in a gamma model. A prequential analysis is carried out by using a simulation study to compare these predictors.     

Autoregressive frequency detection using Regularized Least Squares

Tracking of an unknown frequency embedded in noise is widely applied in a variety of applications. Unknown frequencies can be obtained by approximating generalized spectral density of a periodic process by an autoregressive (AR) model. The advantage is that an AR model has a simple structure and its parameters can be easily estimated iteratively, which is crucial for online (real-time) applications. Typically, the order of the AR approximation is chosen by information criteria. However, with an increase of a sample size, model order may change, which leads to re-estimation of all model parameters. We propose a new iterative procedure for frequency detection based on a regularization of an empirical information matrix. The suggested method enables to avoid the repeated model selection as well as parameter estimation steps and therefore optimize computational costs. The asymptotic properties of the proposed regularized AR (RAR) frequency estimates are derived and performance of RAR is evaluated by numerical examples.     

Autoregressive process modeling via the Lasso procedure

The Lasso is a popular model selection and estimation procedure for linear models that enjoys nice theoretical properties. In this paper, we study the Lasso estimator for fitting autoregressive time series models. We adopt a double asymptotic framework where the maximal lag may increase with the sample size. We derive theoretical results establishing various types of consistency. In particular, we derive conditions under which the Lasso estimator for the autoregressive coefficients is model selection consistent, estimation consistent and prediction consistent. Simulation study results are reported.     

On unit roots for spatial autoregressive models

In this paper we consider the unit root problem for one rather simple autoregressive model Y_t,s=aY_t-1,s+bY_t,s-1+?_t,s on a two-dimensional lattice. We show that the growth of variance of Y_t,s is essentially different from corresponding growth in the unit root case for AR(1) or AR(2) time series models. We also show that the dimension of the lattice plays an important role: the growth of variance of autoregressive field on a d-dimensional lattice is different for d=2,3 and d≥4.     

Testing the stability of the functional autoregressive process

The functional autoregressive process has become a useful tool in the analysis of functional time series data. It is defined by the equation , in which the observations X_n and errors ε_n are curves, and is an operator. To ensure meaningful inference and prediction based on this model, it is important to verify that the operator does not change with time. We propose a method for testing the constancy of against a change-point alternative which uses the functional principal component analysis. The test statistic is constructed to have a well-known asymptotic distribution, but the asymptotic justification of the procedure is very delicate. We develop a new truncation approach which together with Mensov’s inequality can be used in other problems of functional time series analysis. The estimation of the principal components introduces asymptotically non-negligible terms, which however cancel because of the special form of our test statistic (CUSUM type). The test is implemented using the R package fda, and its finite sample performance is examined by application to credit card transaction data.     

Tensorial products of functional ARMA processes

We study the structure of tensorial products for the autoregressive and moving average processes (X_n), with values in a Hilbert space H and with innovations that are martingale differences.The obtained models are ARMA(H⊗H) processes, possibly non standard. We provide criteria for the standardness of these models, we specify the results in the real case, give some examples and consider some applications.     

Asymptotic properties of general autoregressive models and strong consistency of least-squares estimates of their parameters

This paper establishes several almost sure asymptotic properties of general autoregressive processes. By making use of these properties, we obtain a proof of the strong consistency of the least-squares estimates of the parameters of the process without any assumption on the roots of the characteristic polynomial.     

Correlation analysis of nonstationary processes using prior information

The paper examines the applicability of correlation analysis to identification of some classes of nonstationary processes. Consistency conditions are obtained for the time-dependent 1st order autoregressive (AR) process and for the polynomial AR model of an arbitrary order. These results are applied for stable AR estimation and for linear prediction on given classes of nonstationary processes.Translated from Matematicheskoe Modelirovanie i Reshenie Obratnykh Zadach Matematicheskoi Fiziki, pp. 50–54, 1993.     

On the asymptotic properties of multivariate sample autocovariances

We show that if a process can be obtained by filtering an autoregressive process, then the asymptotic distribution of sample autocovariances of the former is the same as the asymptotic distribution of linear combinations of sample autocovariances of the latter. This result is used to show that for small lags the sample autocovariances of the filtered process have the same asymptotic distribution as estimators utilizing more information (observations on the associated autoregression process and knowledge of the parameters of the filter). In particular, for a Gaussian ARMA process the first few sample autocovariances are jointly asymptotically efficient.     

Measures of influence for the functional linear model with scalar response

This paper studies how to identify influential observations in the functional linear model in which the predictor is functional and the response is scalar. Measurement of the effects of a single observation on estimation and prediction when the model is estimated by the principal components method is undertaken. For that, three statistics are introduced for measuring the influence of each observation on estimation and prediction of the functional linear model with scalar response that are generalizations of the measures proposed for the standard regression model by [D.R. Cook, Detection of influential observations in linear regression, Technometrics 19 (1977) 15-18; D. Peña, A new statistic for influence in linear regression, Technometrics 47 (2005) 1-12] respectively. A smoothed bootstrap method is proposed to estimate the quantiles of the influence measures, which allows us to point out which observations have the larger influence on estimation and prediction. The behavior of the three statistics and the quantile estimation bootstrap based method is analyzed via a simulation study. Finally, the practical use of the proposed statistics is illustrated by the analysis of a real data example, which show that the proposed measures are useful for detecting heterogeneity in the functional linear model with scalar response.     

Asymptotic Analysis of a Multiple Frequency Estimation Method

A recently proposed method of multiple frequency estimation for mixed-spectrum time series is analyzed. The so-called PF method is a procedure that combines the autoregressive (AR) representation of superimposed sinusoids with the idea of parametric filtering. The gist of the method is to parametrize a linear filter in accord with a certain parametrization property, as suggested by the particular form of the bias encountered by Prony′s least-squares estimator for the AR model. It is shown that for any parametric filter with this property, the least-squares estimator obtained from the filtered data is almost surely contractive as a function of the filter parameter and has a unique multivariate fixed-point in the vicinity of the true AR parameter. The fixed-point, known as the PF estimator, is shown to be stronly consistent for estimating the AR model, and the chronic bias of Prony′s estimator is thus eliminated. The almost sure convergence of an iterative algorithm that calculates the fixed-point and the asymptotic normality of the PF estimator are also established. The all-pole filter is considered as an example and application of the developed theory.     

Functional data analysis for cash flow and transactions intensity continuous-time prediction using Hilbert-valued autoregressive processes

In this paper we use a functional autoregressive model as a robust predictor of the cash flow and intensity of transactions in a credit card payment systems. Intraday economic time series are treated as random continuous functions projected onto low dimensional subspace. Wavelet bases are considered for data smoothing. We compare two linear wavelet methods for the prediction problem of a continuous-time stochastic process on an entire time interval. Ex poste prediction is used to check the models.     

Change detection in autoregressive time series

Autoregressive time series models of order p have p+2 parameters, the mean, the variance of the white noise and the p autoregressive parameters. Change in any of these over time is a sign of disturbance that is important to detect. The methods of this paper can test for change in any one of these p+2 parameters separately, or in any collection of them. They are available in forms that make one-sided tests possible, furthermore, they can be used to test for a temporary change. The test statistics are based on the efficient score vector. The large sample properties of the change-point estimator are also explored.     

HAC estimation and strong linearity testing in weak ARMA models

In the framework of ARMA models, we consider testing the reliability of the standard asymptotic covariance matrix (ACM) of the least-squares estimator. The standard formula for this ACM is derived under the assumption that the errors are independent and identically distributed, and is in general invalid when the errors are only uncorrelated. The test statistic is based on the difference between a conventional estimator of the ACM of the least-squares estimator of the ARMA coefficients and its robust HAC-type version. The asymptotic distribution of the HAC estimator is established under the null hypothesis of independence, and under a large class of alternatives. The asymptotic distribution of the proposed statistic is shown to be a standard χ² under the null, and a noncentral χ² under the alternatives. The choice of the HAC estimator is discussed through asymptotic power comparisons. The finite sample properties of the test are analyzed via Monte Carlo simulation.