在线Logistic回归

针对连续数据流分类问题,基于在线学习理论,提出一种在线logistic回归算法.研究带有正则项的在线logistic回归,提出了在线logistic-l2回归模型,并给出了理论界估计.最终实验结果表明,随着在线迭代次数的增加,提出的模型与算法能够达到离线预测的分类结果.本文工作为处理海量流数据分类问题提供了一种新的有效方法.     

AR模型辨识的稀疏结构迭代算法

模型估计是机器学习领域一个重要的研究内容,动态数据的模型估计是系统辨识和系统控制的基础.针对AR时间序列模型辨识问题,证明了在给定阶数下AR模型参数的最小二乘估计本质上也是一种矩估计.根据结构风险最小化原理,通过对模型拟合度和模型复杂度的折衷,提出了基于稀疏结构迭代的AR序列模型估计算法,并讨论了基于广义岭估计的最优正则化参数选取规则.数值结果表明,方法能以节省参数的方式有效地实现AR模型的辨识,比矩估计法结果有明显改善.     

Asymptotic Analysis of a Multiple Frequency Estimation Method

A recently proposed method of multiple frequency estimation for mixed-spectrum time series is analyzed. The so-called PF method is a procedure that combines the autoregressive (AR) representation of superimposed sinusoids with the idea of parametric filtering. The gist of the method is to parametrize a linear filter in accord with a certain parametrization property, as suggested by the particular form of the bias encountered by Prony′s least-squares estimator for the AR model. It is shown that for any parametric filter with this property, the least-squares estimator obtained from the filtered data is almost surely contractive as a function of the filter parameter and has a unique multivariate fixed-point in the vicinity of the true AR parameter. The fixed-point, known as the PF estimator, is shown to be stronly consistent for estimating the AR model, and the chronic bias of Prony′s estimator is thus eliminated. The almost sure convergence of an iterative algorithm that calculates the fixed-point and the asymptotic normality of the PF estimator are also established. The all-pole filter is considered as an example and application of the developed theory.     

Asymptotic distribution of residual autocorrelations from estimation of ARMA processes by Gram-Schmidt orthogonalization

One computationally efficient procedure for obtaining maximum likelihood parameter estimates for an ARMA process is based on the Gram-Schmidt orthogonalization of the space generated by the finite series of observations. This paper shows that the asymptotic distribution of the autocorrelations of the resulting residuals coincides with that for least-square residuals.     

Inference in binomial AR(1) models

This paper studies inference methods for stationary time series with binomial distributions. Such series describe, for example, the number of rainy days in consecutive weeks. First, we formulate the renewal sequence version of the model that seemingly generates a new class of stationary binomial series. The model is shown to obey an AR(1) recursion in cases where the renewal lifetime has a constant hazard rate past lag one. Explicit asymptotic variances of the parameter estimators in the AR(1) case are derived from conditional least squares methods; likelihood techniques are also considered.     

Detecting changes in the ar parameters of a nonstationary arma process

We present a method for detecting changes in the AR parameters of an ARMA process with arbitrarily time varying MA parameters. Assuming that a collection of observations and a set of nominal time invariant AR parameters are given, we test if the observations are generated by the nominal AR parameters or by a different set of time invariant AR parameters. The detection method is derived by using a local asymptotic approach and it is based on an estimation procedure which was shown to be consistent under nonstationarities.     

Applicability of the Interval Taylor Model to the Computational Proof of Existence of Periodic Trajectories in Systems of Ordinary Differential Equations

We consider the construction of the interval Taylor model used to prove the existence of periodic trajectories in systems of ordinary differential equations. Our model differs from the ones available in the literature in the method for describing the algorithms for the computation of arithmetic operations over Taylor models. In the framework of the current model, this permits reducing the computational expenditures for obtaining interval estimates on computers. We prove an assertion that permits establishing the existence of a periodic solution of a system of ordinary differential equations by verifying the convergence of the Picard iterations in the sense of embedding of the proposed Taylor models. An example illustrating how the resulting assertion can be used to prove the existence of a closed trajectory in the van der Pol system is given.     

Generalised parameters technique for identification of seasonal ARMA (SARMA) and non seasonal ARMA (NSARMA) models

Times series modeling plays an important role in the field of engineering, Statistics, Biomedicine etc. Model identification is one of crucial steps in the modeling of an AutoRegreesive Moving Average (ARMA(p,q)) process for real world problems. Many techniques have been developed in the literature (Salas et al., McLeod et al. etc.) for the identification of an ARMA(p,q) Model. In this paper, a new technique called The Generalised Parameters Technique is formulated for seasonal and non-seasonal ARMA model identification. This technique is very simple and can be applied to any given time series. Initial estimates of the AR parameters of the ARMA model are also obtained by this method. This model identification technique is validated through many theoretical and simulated examples.     

Inexact-Restoration Algorithm for Constrained Optimization1

We introduce a new model algorithm for solving nonlinear programming problems. No slack variables are introduced for dealing with inequality constraints. Each iteration of the method proceeds in two phases. In the first phase, feasibility of the current iterate is improved; in second phase, the objective function value is reduced in an approximate feasible set. The point that results from the second phase is compared with the current point using a nonsmooth merit function that combines feasibility and optimality. This merit function includes a penalty parameter that changes between consecutive iterations. A suitable updating procedure for this penalty parameter is included by means of which it can be increased or decreased along consecutive iterations. The conditions for feasibility improvement at the first phase and for optimality improvement at the second phase are mild, and large-scale implementation of the resulting method is possible. We prove that, under suitable conditions, which do not include regularity or existence of second derivatives, all the limit points of an infinite sequence generated by the algorithm are feasible, and that a suitable optimality measure can be made as small as desired. The algorithm is implemented and tested against the LANCELOT algorithm using a set of hard-spheres problems.     

ESTIMATION OF THE MIXED AR AND HIDDEN PERIODIC MODEL

ThisresearchissupportedbytheNationalNaturalScienceFoundationofChina.1.IntroductionGeneralizedhiddenperiodicmodelhasthefollowingformwhereacisthesetofallpositiveintegers,('~{((t);tEac}isastationarysequencewithzeromeanandcontinuousspectraldensity,i=n,qisanonnegativeinteger,'f=0,X=(Al,Az,',A,)isarealvectorwith--T相似文献   

Autoregressive frequency detection using Regularized Least Squares

Tracking of an unknown frequency embedded in noise is widely applied in a variety of applications. Unknown frequencies can be obtained by approximating generalized spectral density of a periodic process by an autoregressive (AR) model. The advantage is that an AR model has a simple structure and its parameters can be easily estimated iteratively, which is crucial for online (real-time) applications. Typically, the order of the AR approximation is chosen by information criteria. However, with an increase of a sample size, model order may change, which leads to re-estimation of all model parameters. We propose a new iterative procedure for frequency detection based on a regularization of an empirical information matrix. The suggested method enables to avoid the repeated model selection as well as parameter estimation steps and therefore optimize computational costs. The asymptotic properties of the proposed regularized AR (RAR) frequency estimates are derived and performance of RAR is evaluated by numerical examples.     

A COMPARISON OF FORECASTING MODELS OF THE VOLATILITY IN SHENZHEN STOCK MARKET

Based on the weekly closing price of Shenzhen Integrated Index, this article studies the volatility of Shenzhen Stock Market using three different models: Logistic, AR(1) and AR(2). The time-variable parameters of Logistic regression model is estimated by using both the index smoothing method and the time-variable parameter estimation method. And both the AR(1) model and the AR(2) model of zero-mean series of the weekly closing price and its zero-mean series of volatility rate are established based on the analysis results of zero-mean series of the weekly closing price. Six common statistical methods for error prediction are used to test the predicting results. These methods are: mean error (ME), mean absolute error (MAE), root mean squared error (RMSE), mean absolute percentage error (MAPE), Akaike's information criterion (AIC), and Bayesian information criterion (BIC). The investigation shows that AR(1) model exhibits the best predicting result, whereas AR(2) model exhibits predicting results that is intermediate between AR(1) model and the Logistic regression model.     

Estimation for regression with infinite variance errors

This paper addresses the problem of modelling time series with nonstationarity from a finite number of observations. Problems encountered with the time varying parameters in regression type models led to the smoothing techniques. The smoothing methods basically rely on the finiteness of the error variance, and thus, when this requirement fails, particularly when the error distribution is heavy tailed, the existing smoothing methods due to [1], are no longer optimal. In this paper, we propose a penalized minimum dispersion method for time varying parameter estimation when a regression model generated by an infinite variance stable process with characteristic exponent α ε (1, 2). Recursive estimates are evaluated and it is shown that these estimates for a nonstationary process with normal errors is a special case.     

Generalized Rank Estimates For An Autoregressive Time Series: A U-Statistic Approach

A class of weighted rank-based estimates for estimating the parameter vector of an autoregressive time series is considered. This class of estimates is similar to, and contains, the class proposed by Terpstra et al. [54]. Asymptotic linearity properties are derived for the so called GR-estimates. Based on these properties, the GR-estimates are shown to be asymptotically normal at rate n ^1/2. The theory of U-statistics along with a characterization of weak dependence that is inherent in stationary AR(p) models are the primary tools used to obtain the results. The so called pair-wise slopes estimator, which is a special case of this class of estimates, is discussed in an AR(1) context. This revised version was published online in June 2006 with corrections to the Cover Date.     

Breathers for the Discrete Nonlinear Schrödinger Equation with Nonlinear Hopping

We discuss the existence of breathers and lower bounds on their power, in nonlinear Schrödinger lattices with nonlinear hopping. Our methods extend from a simple variational approach to fixed-point arguments, deriving lower bounds for the power which can serve as a threshold for the existence of breather solutions. Qualitatively, the theoretical results justify non-existence of breathers below the prescribed lower bounds of the power which depend on the dimension, the parameters of the lattice as well as of the frequency of breathers. In the case of supercritical power nonlinearities we investigate the interplay of these estimates with the optimal constant of the discrete interpolation inequality. Improvements of the general estimates, taking into account the localization of the true breather solutions are derived. Numerical studies in the one-dimensional lattice corroborate the theoretical bounds and illustrate that in certain parameter regimes of physical significance, the estimates can serve as accurate predictors of the breather power and its dependence on the various system parameters.     

Parameter estimation of partially observed continuous time stochastic processes via the EM algorithm

An algorithm is presented for the problem of maximum likelihood (ML) estimation of parameters of partially observed continuous time random processes. This algorithm is an extension of the EM algorithm [3] used in the time series literature, and preserves its main features. It is then applied to the problem of parameter estimation of continuous time, finite state or infinite state (diffusions) Markov processes observed via a noisy sensor. The algorithm in general involves iterations of non-linear smoothing with known parameters and then a non-stochastic maximization. For special cases, including linear models and AR/ARMA processes observed in white noise, each iteration is easily performed with finite dimensional filters. Finally, the algorithm is applied to parameter estimation of “randomly slowly varying” linear systems observed in white noise, and explicit results are derived.     

Estimation of spatial ar models

In this paper, some results in [1] on parameter estimation of spatial AR models are improved. Moreover, using some recent results on convergence rate of sample autocorrelations, we give strongly consistent order estimates for spatial AR models and iterated logarithmic convergence rate for AR parameter estimates, which improve the result in [2] of weakly consistent order estimation and strongly consistent parameter estimation.This project is supported by the Doctoral Programme Foundation of Institute of Higher Education and by the National Natural Science Foundation of China.     

Improved threshold diagnostic plots for extreme value analyses

A crucial aspect of threshold-based extreme value analyses is the level at which the threshold is set. For a suitably high threshold asymptotic theory suggests that threshold excesses may be modelled by a generalized Pareto distribution. A common threshold diagnostic is a plot of estimates of the generalized Pareto shape parameter over a range of thresholds. The aim is to select the lowest threshold above which the estimates are judged to be approximately constant, taking into account sampling variability summarized by pointwise confidence intervals. This approach doesn’t test directly the hypothesis that the underlying shape parameter is constant above a given threshold, but requires the user subjectively to combine information from many dependent estimates and confidence intervals. We develop tests of this hypothesis based on a multiple-threshold penultimate model that generalizes a two-threshold model proposed recently. One variant uses only the model fits from the traditional parameter stability plot. This is particularly beneficial when many datasets are analysed and enables assessment of the properties of the test on simulated data. We assess and illustrate these tests on river flow rate data and 72 series of significant wave heights.     

On same-realization prediction in an infinite-order autoregressive process

Let observations come from an infinite-order autoregressive (AR) process. For predicting the future of the observed time series (referred to as the same-realization prediction), we use the least-squares predictor obtained by fitting a finite-order AR model. We also allow the order to become infinite as the number of observations does in order to obtain a better approximation. Moment bounds for the inverse sample covariance matrix with an increasing dimension are established under various conditions. We then apply these results to obtain an asymptotic expression for the mean-squared prediction error of the least-squares predictor in same-realization and increasing-order settings. The second-order term of this expression is the sum of two terms which measure both the goodness of fit and model complexity. It forms the foundation for a companion paper by Ing and Wei (Order selection for same-realization predictions in autoregressive processes, Technical report C-00-09, Institute of Statistical Science, Academia Sinica, Taipei, Taiwan, ROC, 2000) which provides the first theoretical verification that AIC is asymptotically efficient for same-realization predictions. Finally, some comparisons between the least-squares predictor and the ridge regression predictor are also given.     

Convex combinations of long memory estimates from different sampling rates

This paper investigates convex combinations of long memory estimates from both the original and sub-sampled data. Sub-sampling is carried out by decreasing the sampling rate, which leaves the long memory parameter unchanged. Any convex combination of these sub-sample estimates requires a preliminary correction for the bias observed at lower sampling rates, reported by Souza and Smith (2002). Through Monte Carlo simulations, we investigate the bias and the standard deviation of the combined estimates, as well as the root mean squared error (RMSE). Combining estimates can significantly lower the RMSE of a standard estimator (by about 30% on average for ARFIMA (0, d, 0) series), at the cost of inducing some bias.