Recursive computational formulas of the least squares criterion functions for scalar system identification

The paper discusses recursive computation problems of the criterion functions of several least squares type parameter estimation methods for linear regression models, including the well-known recursive least squares (RLS) algorithm, the weighted RLS algorithm, the forgetting factor RLS algorithm and the finite-data-window RLS algorithm without or with a forgetting factor. The recursive computation formulas of the criterion functions are derived by using the recursive parameter estimation equations. The proposed recursive computation formulas can be extended to the estimation algorithms of the pseudo-linear regression models for equation error systems and output error systems. Finally, the simulation example is provided.     

Recursive Estimation of Regression Functions by Local Polynomial Fitting

The recursive estimation of the regression function m(x) = E(Y/X = x) and its derivatives is studied under dependence conditions. The examined method of nonparametric estimation is a recursive version of the estimator based on locally weighted polynomial fitting, that in recent articles has proved to be an attractive technique and has advantages over other popular estimation techniques. For strongly mixing processes, expressions for the bias and variance of these estimators are given and asymptotic normality is established. Finally, a simulation study illustrates the proposed estimation method.     

һ��Coxģ�Ͳ����ľ���Bayes��˫�����

In this paper, the empirical Bayes (EB) two-sided test for parameter of Cox models is investigated under square loss functions. At first by using recursive kernel estimation of probability function the empirical Bayes two-sided test rule is constructed. It proves that the proposed empirical Bayes test rule is asymptotic optimal and convergence rates are obtained under suitable conditions. Finally an example of satisfying theorem conditions is given.     

Maximum likelihood least squares identification for systems with autoregressive moving average noise

Maximum likelihood methods are important for system modeling and parameter estimation. This paper derives a recursive maximum likelihood least squares identification algorithm for systems with autoregressive moving average noises, based on the maximum likelihood principle. In this derivation, we prove that the maximum of the likelihood function is equivalent to minimizing the least squares cost function. The proposed algorithm is different from the corresponding generalized extended least squares algorithm. The simulation test shows that the proposed algorithm has a higher estimation accuracy than the recursive generalized extended least squares algorithm.     

Three-stage recursive least squares parameter estimation for controlled autoregressive autoregressive systems

A three-stage recursive least squares parameter estimation algorithm is derived for controlled autoregressive autoregressive (CARAR) systems. The basic idea is to decompose a CARAR system into three subsystems, one of which contains one parameter vector, and to identify the parameters of each subsystem one by one. Compared with the recursive generalized least squares algorithm, the dimensions of the involved covariance matrices in each subsystem become small and thus the proposed algorithm has a high computational efficiency. Finally, we verify the proposed algorithm with a simulation example.     

Least-squares linear estimation of signals from observations with Markovian delays

The least-squares linear estimation of signals from randomly delayed measurements is addressed when the delay is modeled by a homogeneous Markov chain. To estimate the signal, recursive filtering and fixed-point smoothing algorithms are derived, using an innovation approach, assuming that the covariance functions of the processes involved in the observation equation are known. Recursive formulas for filtering and fixed-point smoothing error covariance matrices are obtained to measure the goodness of the proposed estimators.     

Recursive parameter estimation: convergence

We consider estimation procedures which are recursive in the sense that each successive estimator is obtained from the previous one by a simple adjustment. We propose a wide class of recursive estimation procedures for the general statistical model and study convergence.      

Online EM Algorithm for Hidden Markov Models

Online (also called “recursive” or “adaptive”) estimation of fixed model parameters in hidden Markov models is a topic of much interest in times series modeling. In this work, we propose an online parameter estimation algorithm that combines two key ideas. The first one, which is deeply rooted in the Expectation-Maximization (EM) methodology, consists in reparameterizing the problem using complete-data sufficient statistics. The second ingredient consists in exploiting a purely recursive form of smoothing in HMMs based on an auxiliary recursion. Although the proposed online EM algorithm resembles a classical stochastic approximation (or Robbins–Monro) algorithm, it is sufficiently different to resist conventional analysis of convergence. We thus provide limited results which identify the potential limiting points of the recursion as well as the large-sample behavior of the quantities involved in the algorithm. The performance of the proposed algorithm is numerically evaluated through simulations in the case of a noisily observed Markov chain. In this case, the algorithm reaches estimation results that are comparable to those of the maximum likelihood estimator for large sample sizes. The supplemental material for this article available online includes an appendix with the proofs of Theorem 1 and Corollary 1 stated in Section 4 as well as the MATLAB/OCTAVE code used to implement the algorithm in the case of a noisily observed Markov chain considered in Section 5.     

Recursive state estimation for hybrid systems

The paper deals with recursive state estimation for hybrid systems. An unobservable state of such systems is changed both in a continuous and a discrete way. Fast and efficient online estimation of hybrid system state is desired in many application areas. The presented paper proposes to look at this problem via Bayesian filtering in the factorized (decomposed) form. General recursive solution is proposed as the probability density function, updated entry-wise. The paper summarizes general factorized filter specialized for (i) normal state-space models; (ii) multinomial state-space models with discrete observations; and (iii) hybrid systems. Illustrative experiments and comparison with one of the counterparts are provided.     

On consistency of recursive least squares identification algorithms for controlled auto-regression models

The recursive least squares (RLS) algorithms is a popular parameter estimation one. Its consistency has received much attention in the identification literature. This paper analyzes convergence of the RLS algorithms for controlled auto-regression models (CAR models), and gives the convergence theorems of the parameter estimation by the RLS algorithms, and derives the conditions that the parameter estimates consistently converge to the true parameters under noise time-varying variance and unbounded condition number. This relaxes the assumptions that the noise variance is constant and that high-order moments are existent. Finally, the proposed algorithms are tested with two example systems, including an experimental water-level system.     

One-step estimation for varying coefficient models

A one-step method is proposed to estimate the unknown functions in the varying coefficient models, in which the unknown functions admit different degrees of smoothness. In this method polynomials of different orders are used to approximate unknown functions with different degrees of smoothness. As only one minimization operation is employed, the required computation burden is much less than that required by the existing two-step estimation method. It is shown that the one-step estimators also achieve the optimal convergence rate. Moreover this property is obtained under conditions milder than that imposed in the two-step estimation method. More importantly, as only one minimization operation is employed, the full asymptotic properties, not only the asymptotic bias and variance, but also the asymptotic distributions of the estimators can be derived. The asymptotic distribution results will play a key role for making statistical inference.     

Robust Estimation Using the Huber Function With a Data-Dependent Tuning Constant

Robust estimation often relies on a dispersion function that is more slowly varying at large values than the square function. However, the choice of tuning constant in dispersion functions may impact the estimation efficiency to a great extent. For a given family of dispersion functions such as the Huber family, we suggest obtaining the “best” tuning constant from the data so that the asymptotic efficiency is maximized. This data-driven approach can automatically adjust the value of the tuning constant to provide the necessary resistance against outliers. Simulation studies show that substantial efficiency can be gained by this data-dependent approach compared with the traditional approach in which the tuning constant is fixed. We briefly illustrate the proposed method using two datasets.     

舍入数据下Lomax分布形状参数的经验Bayes检验问题

本文研究了舍入数据下Lomax分布形状参数的经验Bayes (EB)单侧检验问题.利用密度函数的递归核估计构造了参数的EB检验函数,并在适当的条件下证明了所提出的EB检验函数的渐近最优性,获得了它的收敛速度.最后,给出一个有关本文主要结果的例子.     

A Class of Learning/Estimation Algorithms Using Nominal Values: Asymptotic Analysis and Applications

A class of estimation/learning algorithms using stochastic approximation in conjunction with two kernel functions is developed. This algorithm is recursive in form and uses known nominal values and other observed quantities. Its convergence analysis is carried out; the rate of convergence is also evaluated. Applications to a nonlinear chemical engineering system are examined through simulation study. The estimates obtained will be useful in process operation and control, and in on-line monitoring and fault detection.     

Combined state and least squares parameter estimation algorithms for dynamic systems

The control theory and automation technology cast the glory of our era. Highly integrated computer chip and automation products are changing our lives. Mathematical models and parameter estimation are basic for automatic control. This paper discusses the parameter estimation algorithm of establishing the mathematical models for dynamic systems and presents an estimated states based recursive least squares algorithm, and the states of the system are computed through the Kalman filter using the estimated parameters. A numerical example is provided to confirm the effectiveness of the proposed algorithm.     

On the Consistency of Prony's Method and Related Algorithms

Abstract

Modifications of Prony's classical technique for estimating rate constants in exponential fitting problems have many contemporary applications. In this article the consistency of Prony's method and of related algorithms based on maximum likelihood is discussed as the number of observations n → ∞ by considering the simplest possible models for fitting sums of exponentials to observed data. Two sampling regimes are relevant, corresponding to transient problems and problems of frequency estimation, each of which is associated with rather different kinds of behavior. The general pattern is that the stronger results are obtained for the frequency estimation problem. However, the algorithms considered are all scaling dependent and consistency is not automatic. A new feature that emerges is the importance of an appropriate choice of scale in order to ensure consistency of the estimates in certain cases. The tentative conclusion is that algorithms referred to as Objective function Reweighting Algorithms (ORA's) are superior to their exact maximum likelihood counterparts, referred to as Gradient condition Reweighting Algorithms (GRA's), especially in the frequency estimation problem. This conclusion does not extend to fitting other families of functions such as rational functions.     

Acoustic estimation of suspended sediment concentration

In this paper, the acoustic estimation of suspended sediment concentration is discussed and two estimation methods of suspended sediment concentration are presented. The first method is curve fitting method, in which, according to the acoustic backscattering theory we assume that the fitting factor K₁ (r) between the concentration M(r) obtained by acoustic observation and the concentration M₀ ( r) obtained by sampling water is a high order power function of distancer. Using least-square algorithm, we can determine the coefficients of the high order power function by minimizing the difference betweenM( r) and M₀ ( r) in the whole water profile. To the absorption coefficient of sound due to the suspension in water we do not give constraint in the first method. The second method is recursive fitting method, in which we take M₀ ( r) as the conditions of initialization and decision and give rational constraints to some parameters. The recursive process is stable. We analyzed the two methods with a lot of experimental data. The analytical results show that the estimate error of the first method is less than that of the second method and the latter can not only estimate the concentration of suspended sediment but also give the absorption coefficient of sound. Good results have been obtained with the two methods.     

线性定常系统特征模型的特征参量辨识

说明线性定常系统特征模型的特征参量是一组由高阶线性定常系统的相关信息压缩而成,于是不能简单的作为与状态无关的慢时变参数来处理. 基于特征建模思想,建立了线性定常系统特征模型的特征参量与子空间方法之间的联系,给出了一种该特征模型的特征参量 的合成辨识算法.同时证明了在用于子空间辨识的样本量充分大和用于状态估计的时间充分长的情况下, 特征参量的估计值与真值之间的误差达到充分小. 最后,对于一个六阶的单输入单输出线性定常系统的仿真例子,对投影的带遗忘因子最小二乘算法和合成辨识算法进行了比较,验证了合成辨识算法的有效性.     

Signal estimation using an array of recorders

This paper considers the problem of signal estimation in the case where the signal is received by an array of recorders. Because of the spatial configuration of the array the individual recorders will, at any instant of time, receive lagged forms of the signal. Moreover the lags in question will often be frequency dependent. An estimation procedure is proposed and its asymptotic properties investigated. The optimum orientation of such arrays is also discussed.     

Orthogonality correction in the conjugate-gradient method

This paper describes a simple algorithm for calculating the carryover term β in the conjugate-gradient method. The proposed algorithm incorporates an orthogonality correction as well as an automatic restart. Its performance is compared with alternate β forms reported, using five test functions and two cases of parameter estimation.