Empirical likelihood for the smoothed LAD estimator in infinite variance autoregressive models

This paper proposes an empirical likelihood method to estimate the parameters of infinite variance autoregressive (IVAR) models and to construct confidence regions for the parameters. Simulation studies suggest that in small sample case, the empirical likelihood confidence regions may be more accurate than the confidence regions constructed by the normal approximation based on the self-weighted LAD estimator proposed by Ling (2005).     

Shrinkage estimation and variable selection in multiple regression models with random coefficient autoregressive errors

In this paper, we consider improved estimation strategies for the parameter vector in multiple regression models with first-order random coefficient autoregressive errors (RCAR(1)). We propose a shrinkage estimation strategy and implement variable selection methods such as lasso and adaptive lasso strategies. The simulation results reveal that the shrinkage estimators perform better than both lasso and adaptive lasso when and only when there are many nuisance variables in the model.     

Local Walsh-average-based estimation and variable selection for single-index models

We propose a robust estimation procedure based on local Walsh-average regression(LWR) for single-index models. Our novel method provides a root-n consistent estimate of the single-index parameter under some mild regularity conditions; the estimate of the unknown link function converges at the usual rate for the nonparametric estimation of a univariate covariate. We theoretically demonstrate that the new estimators show significant efficiency gain across a wide spectrum of non-normal error distributions and have almost no loss of efficiency for the normal error. Even in the worst case, the asymptotic relative efficiency(ARE) has a lower bound compared with the least squares(LS) estimates; the lower bounds of the AREs are 0.864 and 0.8896 for the single-index parameter and nonparametric function, respectively. Moreover, the ARE of the proposed LWR-based approach versus the ARE of the LS-based method has an expression that is closely related to the ARE of the signed-rank Wilcoxon test as compared with the t-test. In addition, to obtain a sparse estimate of the single-index parameter, we develop a variable selection procedure by combining the estimation method with smoothly clipped absolute deviation penalty; this procedure is shown to possess the oracle property. We also propose a Bayes information criterion(BIC)-type criterion for selecting the tuning parameter and further prove its ability to consistently identify the true model. We conduct some Monte Carlo simulations and a real data analysis to illustrate the finite sample performance of the proposed methods.     

方差无穷非线性自回归序列的自加权L_1估计

对具有无穷方差的非线性自回归序列x_t=φ(x_(t-1),x_(t-2),…,x_(t-p),θ) ε_t,E(ε_t~2)=∞,利用局部二次近似和连续函数空间C(R~q)上弱收敛随机过程最小点的渐近性质,证明了若存在δ≥1,使得E|ε_t|~δ<∞成立,则θ满足一定条件的自加权L_1估计θ_(L_1)是渐近正态估计,Wald检验统计量也具有通常的x~2分布,为模型的统计推断提供了理论基础.     

Efficient and robust estimation for autoregressive regression models using shape mixtures of skewt normal distribution

Multiple linear regression model based on normally distributed and uncorrelated errors is a popular statistical tool with application in various fields. But these assumptions of normality and no serial correlation are hardly met in real life. Hence, this study considers the linear regression time series model for series with outliers and autocorrelated errors. These autocorrelated errors are represented by a covariance-stationary autoregressive process where the independent innovations are driven by shape mixture of skew-t normal distribution. The shape mixture of skew-t normal distribution is a flexible extension of the skew-t normal with an additional shape parameter that controls skewness and kurtosis. With this error model, stochastic modeling of multiple outliers is possible with an adaptive robust maximum likelihood estimation of all the parameters. An Expectation Conditional Maximization Either algorithm is developed to carryout the maximum likelihood estimation. We derive asymptotic standard errors of the estimators through an information-based approximation. The performance of the estimation procedure developed is evaluated through Monte Carlo simulations and real life data analysis.

     

Efficient on-line estimation of autoregressive parameters

New procedures for estimating autoregressive parameters in AR(m) models are proposed. The proposed method allows for incorporation of auxiliary information into the estimation process and produces estimation procedures, which are consistent and asymptotically efficient under certain regularity conditions. Also, these procedures are naturally on-line and do not require storing all the data. Theoretical results are presented in the case when m = 1. Two important particular cases are considered in detail: linear procedures and likelihood procedures with the LS truncations. A specific example is also presented to briefly discuss some practical aspects of applications of the procedures of this type.     

Nonparametric density estimation for linear processes with infinite variance

We consider nonparametric estimation of marginal density functions of linear processes by using kernel density estimators. We assume that the innovation processes are i.i.d. and have infinite-variance. We present the asymptotic distributions of the kernel density estimators with the order of bandwidths fixed as h =  cn ^−1/5, where n is the sample size. The asymptotic distributions depend on both the coefficients of linear processes and the tail behavior of the innovations. In some cases, the kernel estimators have the same asymptotic distributions as for i.i.d. observations. In other cases, the normalized kernel density estimators converge in distribution to stable distributions. A simulation study is also carried out to examine small sample properties.     

Bayesian subset selection for threshold autoregressive moving-average models

Optimal subset selection among a general family of threshold autoregressive moving-average (TARMA) models is considered. The usual complexity of model/order selection is increased by capturing the uncertainty of unknown threshold levels and an unknown delay lag. The Monte Carlo method of Bayesian model averaging provides a possible way to overcome such model uncertainty. Incorporating with the idea of Bayesian model averaging, a modified stochastic search variable selection method is adapted to consider subset selection in TARMA models, by adding latent indicator variables for all potential model lags as part of the proposed Markov chain Monte Carlo sampling scheme. Metropolis–Hastings methods are employed to deal with the well-known difficulty of including moving-average terms in the model and a novel proposal mechanism is designed for this purpose. Bayesian comparison of two hyper-parameter settings is carried out via a simulation study. The results demonstrate that the modified method has favourable performance under reasonable sample size and appropriate settings of the necessary hyper-parameters. Finally, the application to four real datasets illustrates that the proposed method can provide promising and parsimonious models from more than 16 million possible subsets.     

Concave group methods for variable selection and estimation in high-dimensional varying coefficient models

The varying-coefficient model is flexible and powerful for modeling the dynamic changes of regression coefficients. We study the problem of variable selection and estimation in this model in the sparse, high-dimensional case. We develop a concave group selection approach for this problem using basis function expansion and study its theoretical and empirical properties. We also apply the group Lasso for variable selection and estimation in this model and study its properties. Under appropriate conditions, we show that the group least absolute shrinkage and selection operator (Lasso) selects a model whose dimension is comparable to the underlying model, regardless of the large number of unimportant variables. In order to improve the selection results, we show that the group minimax concave penalty (MCP) has the oracle selection property in the sense that it correctly selects important variables with probability converging to one under suitable conditions. By comparison, the group Lasso does not have the oracle selection property. In the simulation parts, we apply the group Lasso and the group MCP. At the same time, the two approaches are evaluated using simulation and demonstrated on a data example.     

Smoothed estimates for models with random coefficients and infinite variance innovations

Infinite variance processes have attracted growing interest in recent years due to its applications in many areas of statistics (see [1] and references therein). For example, ARIMA time-series models with infinite variance innovations are widely used in financial modelling. However, a little attention has been paid to incorporate infinite variance innovations for time-series models with random coefficients introduced by [2]. This paper considers the problem of nonparametric estimation for some time-series models using the smoothed least absolute deviation (SLAD) estimating function approach. We introduce a class of kernels in order to smooth the LAD estimators. It is also shown that this new SLAD estimators are superior than some existing ones.     

Empirical likelihood for LAD estimators in infinite variance ARMA models

In this paper, we use an empirical likelihood method to construct confidence regions for the stationary ARMA(p,q) models with infinite variance. An empirical log-likelihood ratio is derived by the estimating equation of the self-weighted LAD estimator. It is proved that the proposed statistic has an asymptotic standard chi-squared distribution. Simulation studies show that in a small sample case, the performance of empirical likelihood method is better than that of normal approximation of the LAD estimator in terms of the coverage accuracy.     

Empirical likelihood-based subset selection for partially linear autoregressive models

Based on the empirical likelihood method, the subset selection and hypothesis test for parameters in a partially linear autoregressive model are investigated. We show that the empirical log-likelihood ratio at the true parameters converges to the standard chi-square distribution. We then present the definitions of the empirical likelihood-based Bayes information criteria （EBIC） and Akaike information criteria （EAIC）. The results show that EBIC is consistent at selecting subset variables while EAIC is not. Simulation studies demonstrate that the proposed empirical likelihood confidence regions have better coverage probabilities than the least square method, while EBIC has a higher chance to select the true model than EAIC.     

Parameter estimation for a misspecified arma model with infinite variance innovations

We consider a general linear model , where the innovations Z_t belong to the domain of attraction of an α-stable law for α<2, so that neither Z_t nor X_t have a finite variance. We do not assume that (X_t) is a standardARMA process of the form φ(B)X_t=ϕ(B)Z_t, but we fit anARMA process of a given order to the data X₁,...,X_n by estimating the coefficients of φ and ϕ. Given that (X_t) is anARMA process, it has been proved that the Whittle estimator is a consistent estimator of the true coefficients of ϕ and φ. Moreover, it then has a heavytailed limit distribution and the rate of convergence is (n/logn)^1/α, which compares favorably with the L² situation with rate . In this note we study the limit properties of the Whittle estimator when the underlying model is not necessarily anARMA process. Under general conditions we show that the Whittle estimate converges in probability. It converges weakly to a distribution which does not have a finite moment of order a and the rate of convergence is again (n/logn)^1/α. We also give an analytic expression for the limit distribution. Proceedings of the XVI Seminar on Stability Problems for Stochastic Models, Part II, Eger, Hungary, 1994.     

On parameter estimation of threshold autoregressive models

This paper studies the threshold estimation of a TAR model when the underlying threshold parameter is a random variable. It is shown that the Bayesian estimator is consistent and its limit distribution is expressed in terms of a limit likelihood ratio. Furthermore, convergence of moments of the estimators is also established. The limit distribution can be computed via explicit simulations from which testing and inference for the threshold parameter can be conducted. The obtained results are illustrated with numerical simulations.     

Semi-parametric quantile estimation for double threshold autoregressive models with heteroskedasticity

Compared to the conditional mean or median, conditional quantiles provide a more comprehensive picture of a variable in various scenarios. A semi-parametric quantile estimation method for a double threshold auto-regression with exogenous regressors and heteroskedasticity is considered, allowing representation of both asymmetry and volatility clustering. As such, GARCH dynamics with nonlinearity are added to a nonlinear time series regression model. An adaptive Bayesian Markov chain Monte Carlo scheme, exploiting the link between the quantile loss function and the asymmetric-Laplace distribution, is employed for estimation and inference, simultaneously estimating and accounting for nonlinear heteroskedasticity plus unknown threshold limits and delay lags. A simulation study illustrates sampling properties of the method. Two data sets are considered in the empirical applications: modelling daily maximum temperatures in Melbourne, Australia; and exploring dynamic linkages between financial markets in the US and Hong Kong.     

Discrete time parametric models with long memory and infinite variance

We study a large class of infinite variance time series that display long memory. They can be represented as linear processes (infinite order moving averages) with coefficients that decay slowly to zero and with innovations that are in the domain of attraction of a stable distribution with index 1 < α < 2 (stable fractional ARIMA is a particular example). Assume that the coefficients of the linear process depend on an unknown parameter vector β which is to be estimated from a series of length n. We show that a Whittle-type estimator β_n for β is consistent (β_n converges to the true value β₀ in probability as n → ∞), and, under some additional conditions, we characterize the limiting distribution of the rescaled differences (n/logn)^1/ga(β_n − β₀).     

Simultaneous variable selection for heteroscedastic regression models

In this paper, we propose a new criterion, named PICa, to simultaneously select explanatory variables in the mean model and variance model in heteroscedastic linear models based on the model structure. We show that the new criterion can select the true mean model and a correct variance model with probability tending to 1 under mild conditions. Simulation studies and a real example are presented to evaluate the new criterion, and it turns out that the proposed approach performs well.