Multi-step prediction of time series with random missing data

Multi-step prediction is still an open challenge in time series prediction. Moreover, practical observations are often incomplete because of sensor failure or outliers causing missing data. Therefore, it is very important to carry out research on multi-step prediction of time series with random missing data. Based on nonlinear filters and multilayer perceptron artificial neural networks (ANNs), one novel approach for multi-step prediction of time series with random missing data is proposed in the study. With the basis of original nonlinear filters which do not consider the missing data, first we obtain the generalized nonlinear filters by using a sequence of independent Bernoulli random variables to model random interruptions. Then the multi-step prediction model of time series with random missing data, which can be fit for the online training of generalized nonlinear filters, is established by using the ANN’s weights to present the state vector and the ANN’s outputs to present the observation equation. The performance between the original nonlinear filters based ANN model for multi-step prediction of time series with missing data and the generalized nonlinear filters based ANN model for multi-step prediction of time series with missing data is compared. Numerical results have demonstrated that the generalized nonlinear filters based ANN are proportionally superior to the original nonlinear filters based ANN for multi-step prediction of time series with missing data.     

RCA models with GARCH innovations

Rapid developments of time series models and methods addressing volatility in computational finance and econometrics have been recently reported in the financial literature. The non-linear volatility theory either extends and complements existing time series methodology by introducing more general structures or provides an alternative framework (see Abraham and Thavaneswaran [B. Abraham, A. Thavaneswaran, A nonlinear time series model and estimation of missing observations, Ann. Inst. Statist. Math. 43 (1991) 493–504] and Granger [C.W.J. Granger, Overview of non-linear time series specification in Economics, Berkeley NSF-Symposia, 1998]). In this work, we consider Gaussian first-order linear autoregressive models with time varying volatility. General properties for process mean, variance and kurtosis are derived; examples illustrate the wide range of properties that can appear under the autoregressive assumptions. The results can be used in identifying some volatility models. The kurtosis of the classical RCA model of Nicholls and Quinn [D.F. Nicholls, B.G. Quinn, Random Coefficient Autoregressive Models: An Introduction, in: Lecture Notes in Statistics, vol. 11, Springer, New York, 1982] is shown to be a special case.     

非线性回归模型中的约束拟似然

在非线性回归模型中,拟得分函数是一类线性无偏估计函数中的最优者(GodambeandHeyde(1987),朱仲义(1996)),而由拟得分函数得到的拟似然估计在由线性无偏估计函数得到的估计类中具有渐近最优性(林路(1999)).本文则研究非线性回归模型中的有偏估计函数理论,构造了参数的约束拟似然估计,得到了约束拟似然的局部最优性,局部改进了拟似然估计,从而扩充了线性模型中的有偏估计理论.     

Problems with the estimation of stochastic differential equations using structural equations models

The present paper deals with the identification and maximum likelihood estimation of systems of linear stochastic differential equations using panel data. So we only have a sample of discrete observations over time of the relevant variables for each individual. A popular approach in the social sciences advocates the estimation of the “exact discrete model” after a reparameterization with LISREL or similar programs for structural equations models. The “exact discrete model” corresponds to the continuous time model in the sense that observations at equidistant points in time that are generated by the latter system also satisfy the former. In the LISREL approach the reparameterized discrete time model is estimated first without taking into account the nonlinear mapping from the continuous to the discrete time parameters. In a second step, using the inverse mapping, the fundamental system parameters of the continuous time system in which we are interested, are inferred. However, some severe problems arise with this “indirect approach”. First, an identification problem may arise in multiple equation systems, since the matrix exponential function denning some of the new parameters is in general not one‐to‐one, and hence the inverse mapping mentioned above does not exist. Second, usually some sort of approximation of the time paths of the exogenous variables is necessary before the structural parameters of the system can be estimated with discrete data. Two simple approximation methods are discussed. In both approximation methods the resulting new discrete time parameters are connected in a complicated way. So estimating the reparameterized discrete model by OLS without restrictions does not yield maximum likelihood estimates of the desired continuous time parameters as claimed by some authors. Third, a further limitation of estimating the reparameterized model with programs for structural equations models is that even simple restrictions on the original fundamental parameters of the continuous time system cannot be dealt with. This issue is also discussed in some detail. For these reasons the “indirect method” cannot be recommended. In many cases the approach leads to misleading inferences. We strongly advocate the direct estimation of the continuous time parameters. This approach is more involved, because the exact discrete model is nonlinear in the original parameters. A computer program by Hermann Singer that provides appropriate maximum likelihood estimates is described.     

A fusion of least squares and empirical likelihood for regression models with a missing binary covariate

Multiply robust inference has attracted much attention recently in the context of missing response data. An estimation procedure is multiply robust, if it can incorporate information from multiple candidate models, and meanwhile the resulting estimator is consistent as long as one of the candidate models is correctly specified. This property is appealing, since it provides the user a flexible modeling strategy with better protection against model misspecification. We explore this attractive property for the regression models with a binary covariate that is missing at random. We start from a reformulation of the celebrated augmented inverse probability weighted estimating equation, and based on this reformulation, we propose a novel combination of the least squares and empirical likelihood to separately handling each of the two types of multiple candidate models, one for the missing variable regression and the other for the missingness mechanism. Due to the separation, all the working models are fused concisely and effectively. The asymptotic normality of our estimator is established through the theory of estimating function with plugged-in nuisance parameter estimates. The finite-sample performance of our procedure is illustrated both through the simulation studies and the analysis of a dementia data collected by the national Alzheimer’s coordinating center.     

随机系统的状态与参数估计

本文根据Ｋａｌｍａｎ滤波与时间序列分析的理论，对非线性随机系统，导出了状与参数同时进行估计的递推算法，确定了它们的统计性质。为减少估计误差，用两层估计方法改进了估计精度。这个结果在ＳＩＳＯ系统中得到了证实。     

An EM Algorithm for Estimating Equations

This article presents an algorithm for accommodating missing data in situations where a natural set of estimating equations exists for the complete data setting. The complete data estimating equations can correspond to the score functions from a standard, partial, or quasi-likelihood, or they can be generalized estimating equations (GEEs). In analogy to the EM, which is a special case, the method is called the ES algorithm, because it iterates between an E-Step wherein functions of the complete data are replaced by their expected values, and an S-Step where these expected values are substituted into the complete-data estimating equation, which is then solved. Convergence properties of the algorithm are established by appealing to general theory for iterative solutions to nonlinear equations. In particular, the ES algorithm (and indeed the EM) are shown to correspond to examples of nonlinear Gauss-Seidel algorithms. An added advantage of the approach is that it yields a computationally simple method for estimating the variance of the resulting parameter estimates.     

Optimal time decay of the Boltzmann equation with frictional force

In this paper, we use the combination of energy method and Fourier analysis to obtain the optimal time decay of the Boltzmann equation with frictional force towards equilibrium. Precisely speaking, we decompose the equation into macroscopic and microscopic partitions and perform the energy estimation. Then, we construct a special solution operator to a linearized equation without source term and use Fourier analysis to obtain the optimal decay rate to this solution operator. Finally, combining the decay rate with the energy estimation for nonlinear terms, the optimal decay rate to the Boltzmann equation with frictional force is established.     

Nonparametric estimation of the spectral density of amplitude-modulated time series with missing observations

Consider a real-valued and second-order stationary time series with mean zero. The aim is to estimate its spectral density. A minimax solution of this problem is known when either the time series is observed directly, or some observations are missed according to an independent Bernoulli process, or for some special cases when the time series is multiplied by an amplitude-modulating time series with known distribution. It is shown that if a time series of interest, a Bernoulli time series defining missing mechanism, and an amplitude-modulating time series are mutually independent, then the shape of spectral density of an underlying time series of interest can be estimated with the minimax rate known for the case of direct observations. Furthermore, in some special cases the spectral density can be estimated with the minimax rate known for directly observed time series of interest.     

A general additive-multiplicative rates model for recurrent event data

In this article, we propose a general additive-multiplicative rates model for recurrent event data. The proposed model includes the additive rates and multiplicative rates models as special cases. For the inference on the model parameters, estimating equation approaches are developed, and asymptotic properties of the proposed estimators are established through modern empirical process theory. In addition, an illustration with multiple-infection data from a clinic study on chronic granulomatous disease is pr...     

Risk bounds for model selection via penalization

Performance bounds for criteria for model selection are developed using recent theory for sieves. The model selection criteria are based on an empirical loss or contrast function with an added penalty term motivated by empirical process theory and roughly proportional to the number of parameters needed to describe the model divided by the number of observations. Most of our examples involve density or regression estimation settings and we focus on the problem of estimating the unknown density or regression function. We show that the quadratic risk of the minimum penalized empirical contrast estimator is bounded by an index of the accuracy of the sieve. This accuracy index quantifies the trade-off among the candidate models between the approximation error and parameter dimension relative to sample size. If we choose a list of models which exhibit good approximation properties with respect to different classes of smoothness, the estimator can be simultaneously minimax rate optimal in each of those classes. This is what is usually called adaptation. The type of classes of smoothness in which one gets adaptation depends heavily on the list of models. If too many models are involved in order to get accurate approximation of many wide classes of functions simultaneously, it may happen that the estimator is only approximately adaptive (typically up to a slowly varying function of the sample size). We shall provide various illustrations of our method such as penalized maximum likelihood, projection or least squares estimation. The models will involve commonly used finite dimensional expansions such as piecewise polynomials with fixed or variable knots, trigonometric polynomials, wavelets, neural nets and related nonlinear expansions defined by superposition of ridge functions. Received: 7 July 1995 / Revised version: 1 November 1997     

Optimal homotopy analysis and control of error for solutions to the non-local Whitham equation

The Whitham equation is a non-local model for nonlinear dispersive water waves. Since this equation is both nonlinear and non-local, exact or analytical solutions are rare except for in a few special cases. As such, an analytical method which results in minimal error is highly desirable for general forms of the Whitham equation. We obtain approximate analytical solutions to the non-local Whitham equation for general initial data by way of the optimal homotopy analysis method, through the use of a partial differential auxiliary linear operator. A method to control the residual error of these approximate solutions, through the use of the embedded convergence control parameter, is discussed in the context of optimal homotopy analysis. We obtain residual error minimizing solutions, using relatively few terms in the solution series, in the case of several different kernels and associated initial data. Interestingly, we find that for a specific class of initial data, there exists an exact solution given by the first term in the homotopy expansion. A specific example of initial data which satisfies the condition producing an exact solution is included. These results demonstrate the applicability of optimal homotopy analysis to equations which are simultaneously nonlinear and non-local.     

Estimation of Lyapunov spectrum and model selection for a chaotic time series

The estimation of the Lyapunov spectrum for a chaotic time series is discussed in this study. Three models: the local linear (LL) model; the local polynomial (LP) model and the global radial basis function (RBF) model, are compared for estimating the Lyapunov spectrum in this study. The number of neighbors for training the LL model and the LP model; the number of centers for building the RBF model, have been determined by the generalized degree of freedom for a chaotic time series. The above models have been applied to three artificial chaotic time series and two real-world time series, the numerical results show that the model-chosen LL model provides more accurate estimation than other models for clean data set while the RBF model behaves more robust to noise than other models for noisy data set.     

Estimation in semiparametric models with missing data

This paper considers the problem of parameter estimation in a general class of semiparametric models when observations are subject to missingness at random. The semiparametric models allow for estimating functions that are non-smooth with respect to the parameter. We propose a nonparametric imputation method for the missing values, which then leads to imputed estimating equations for the finite dimensional parameter of interest. The asymptotic normality of the parameter estimator is proved in a general setting, and is investigated in detail for a number of specific semiparametric models. Finally, we study the small sample performance of the proposed estimator via simulations.     

Likelihood-based kernel estimation in semiparametric errors-in-covariables models with validation data

We present methods to handle error-in-variables models. Kernel-based likelihood score estimating equation methods are developed for estimating conditional density parameters. In particular, a semiparametric likelihood method is proposed for sufficiently using the information in the data. The asymptotic distribution theory is derived. Small sample simulations and a real data set are used to illustrate the proposed estimation methods.     

Markovian representation of a bilinear time series model and maximum likelihood estimation of the parameters

A bilinear time series (BLTS) model is expressed in the form of Akaike's Markovian representation in order to use the Kalman recursive estimation approach. It is shown that Akaike's Markovian representation of autoregressive moving average models of orderp and q (ARMA(p,q)) and that of the bilinear model are equivalent. This equivalence facilitates the maximum likelihood estimation of the parameters involved in the bilinear model, which otherwise is an unwieldy problem. The present approach can easily be extended to take into account missing observations     

An equivalent canonical form for multiple time series

In this paper, an equivalent canonical form VARTMA of multiple time series models is obtained by using the polynomial matrix theory, which may lead to converting the parameter estimation problem of the simultaneous equations into that of the single equation in modeling. It it proved that a VARMA model can be turned into the unique VARTMA form by means of the equivalence transformation, and the character of the transformed model is not changed at all.     

A preprocessing method for parameter estimation in ordinary differential equations

Parameter estimation for nonlinear differential equations is notoriously difficult because of poor or even no convergence of the nonlinear fit algorithm due to the lack of appropriate initial parameter values. This paper presents a method to gather such initial values by a simple estimation procedure. The method first determines the tangent slope and coordinates for a given solution of the ordinary differential equation (ODE) at randomly selected points in time. With these values the ODE is transformed into a system of equations, which is linear for linear appearance of the parameters in the ODE. For numerically generated data of the Lorenz attractor good estimates are obtained even at large noise levels. The method can be generalized to nonlinear parameter dependency. This case is illustrated using numerical data for a biological example. The typical problems of the method as well as their possible mitigation are discussed. Since a rigorous failure criterion of the method is missing, its results must be checked with a nonlinear fit algorithm. Therefore the method may serve as a preprocessing algorithm for nonlinear parameter fit algorithms. It can improve the convergence of the fit by providing initial parameter estimates close to optimal ones.     

Chaos and reproduction in sea level

Prediction of sea-level is an important task for navigation, coastal engineering and geodetic applications, as well as recreational activities. This study presents a comparison of Chaos theory and Auto-Regressive Integrated Moving Average (ARIMA) techniques for sea level modelling for daily, weekly, 10-day and monthly time scale at the Cocos (Keeling) islands from 1992 to 2001. The state space reconstruction of the unknown underlying process is directly employed from time series data, through Takens delay embedding method: optimal embedding dimension and delay time are obtained from false nearest neighbours and average mutual information techniques, respectively. Optimal values are then used for the estimation of the correlation dimension and the largest Lyapunov exponent, for inspecting possible signatures of chaotic dynamics. We find a positive Lyapunov exponent an evident feature of chaos. Indeed, the nonlinear prediction of sea level, in the period ranging from January 2001 to December 2001, is in an excellent agreement with the data for the same period, evidencing the nonlinear nature of the process. ARIMA method is also used for sea level modelling, for the same time scales; the performances of the two models are compared using such statistical indices as the root mean square error (RMSE) and correlation coefficient (CC). The comparative analyses show that the chaos theory model has a slight edge over ARIMA while both models are in principal acceptable.