Three-stage recursive least squares parameter estimation for controlled autoregressive autoregressive systems

A three-stage recursive least squares parameter estimation algorithm is derived for controlled autoregressive autoregressive (CARAR) systems. The basic idea is to decompose a CARAR system into three subsystems, one of which contains one parameter vector, and to identify the parameters of each subsystem one by one. Compared with the recursive generalized least squares algorithm, the dimensions of the involved covariance matrices in each subsystem become small and thus the proposed algorithm has a high computational efficiency. Finally, we verify the proposed algorithm with a simulation example.     

Auxiliary model identification method for multirate multi-input systems based on least squares

This paper derives state-space models for multirate multi-input sampled-data systems. Based on the corresponding transfer function models, an auxiliary model based recursive least squares algorithm is presented to identify the parameters of the multirate systems from the multirate input–output data. Further, convergence properties of the proposed algorithm are analyzed. Finally, an illustrative example is given.     

Two-stage least squares based iterative estimation algorithm for CARARMA system modeling

For stochastic systems described by the controlled autoregressive autoregressive moving average (CARARMA) models, a new-type two-stage least squares based iterative algorithm is proposed for identifying the system model parameters and the noise model parameters. The basic idea is based on the interactive estimation theory and to estimate the parameter vectors of the system model and the noise model, respectively. The simulation results indicate that the proposed algorithm is effective.     

Maximum likelihood least squares identification for systems with autoregressive moving average noise

Maximum likelihood methods are important for system modeling and parameter estimation. This paper derives a recursive maximum likelihood least squares identification algorithm for systems with autoregressive moving average noises, based on the maximum likelihood principle. In this derivation, we prove that the maximum of the likelihood function is equivalent to minimizing the least squares cost function. The proposed algorithm is different from the corresponding generalized extended least squares algorithm. The simulation test shows that the proposed algorithm has a higher estimation accuracy than the recursive generalized extended least squares algorithm.     

Combined state and least squares parameter estimation algorithms for dynamic systems

The control theory and automation technology cast the glory of our era. Highly integrated computer chip and automation products are changing our lives. Mathematical models and parameter estimation are basic for automatic control. This paper discusses the parameter estimation algorithm of establishing the mathematical models for dynamic systems and presents an estimated states based recursive least squares algorithm, and the states of the system are computed through the Kalman filter using the estimated parameters. A numerical example is provided to confirm the effectiveness of the proposed algorithm.     

A least squares identification algorithm for a state space model with multi-state delays

A parameter estimator is presented for a state space model with time delay based on the given input–output data. The basic idea is to expand the state equations and to eliminate some state variables, and to substitute the state equation into the output equation to obtain the identification model which contains the information vector and parameter vector. A least squares algorithm is developed to estimate the system parameter vectors. Finally, an illustrative example is provided to verify the effectiveness of the proposed algorithm.     

Convergence of HLS estimation algorithms for multivariable ARX-like systems

A hierarchical least squares (HLS) algorithm is derived in details for identifying MIMO ARX-like systems based on the hierarchical identification principle. It is shown that the parameter estimation errors by the HLS algorithm consistently converge to zero for bounded noise variances by using the stochastic martingale theory. A numerical example is given.     

Recursive computational formulas of the least squares criterion functions for scalar system identification

The paper discusses recursive computation problems of the criterion functions of several least squares type parameter estimation methods for linear regression models, including the well-known recursive least squares (RLS) algorithm, the weighted RLS algorithm, the forgetting factor RLS algorithm and the finite-data-window RLS algorithm without or with a forgetting factor. The recursive computation formulas of the criterion functions are derived by using the recursive parameter estimation equations. The proposed recursive computation formulas can be extended to the estimation algorithms of the pseudo-linear regression models for equation error systems and output error systems. Finally, the simulation example is provided.     

The residual based interactive stochastic gradient algorithms for controlled moving average models

This paper derives a residual based interactive stochastic gradient (ISG) parameter estimation algorithm for controlled moving average (CMA) models and studied the performance of the residual based ISG algorithm under weaker conditions on statistical properties of the noise. Compared with the residual based extended stochastic gradient algorithm for identifying CMA models, the proposed ISG algorithm can give highly accurate parameter estimates by the simulation example.     

Parameter and state estimation algorithm for single-input single-output linear systems using the canonical state space models

This paper presents a new parameter and state estimation algorithm for single-input single-output systems based on canonical state space models from the given input–output data. Difficulties of identification for state space models lie in that there exist unknown noise terms in the formation vector and unknown state variables. By means of the hierarchical identification principle, those noise terms in the information vector are replaced with the estimated residuals and a new least squares algorithm is proposed for parameter estimation and the system states are computed by using the estimated parameters. Finally, an example is provided.     

Hierarchical least squares based iterative estimation algorithm for multivariable Box-Jenkins-like systems using the auxiliary model

This paper presents a hierarchical least squares iterative algorithm to estimate the parameters of multivariable Box-Jenkins-like systems by combining the hierarchical identification principle and the auxiliary model identification idea. The key is to decompose a multivariable systems into two subsystems by using the hierarchical identification principle. As there exist the unmeasurable noise-free outputs and noise terms in the information vector, the solution is using the auxiliary model identification idea to replace the unmeasurable variables with the outputs of the auxiliary model and the estimated residuals. A numerical example is given to show the performance of the proposed algorithm.     

On the consistency of a least squares identification procedure in linear evolution systems

Conditions for the convergence of parameter estimates to the true value applicable in continuous time linear stochastic evolution systems are presented. A special case, continuous time linear stochastic systems with delays, is also considered. A persistent excitation property is proved by control theory methods     

Implicitly-weighted total least squares

In a total least squares (TLS) problem, we estimate an optimal set of model parameters X, so that (A-ΔA)X=B-ΔB, where A is the model matrix, B is the observed data, and ΔA and ΔB are corresponding corrections. When B is a single vector, Rao (1997) and Paige and Strakoš (2002) suggested formulating standard least squares problems, for which ΔA=0, and data least squares problems, for which ΔB=0, as weighted and scaled TLS problems. In this work we define an implicitly-weighted TLS formulation (ITLS) that reparameterizes these formulations to make computation easier. We derive asymptotic properties of the estimates as the number of rows in the problem approaches infinity, handling the rank-deficient case as well. We discuss the role of the ratio between the variances of errors in A and B in choosing an appropriate parameter in ITLS. We also propose methods for computing the family of solutions efficiently and for choosing the appropriate solution if the ratio of variances is unknown. We provide experimental results on the usefulness of the ITLS family of solutions.     

Bias compensation methods for stochastic systems with colored noise

For ARX-like systems, this paper derives a bias compensation based recursive least squares identification algorithm by means of the prefilter idea and bias compensation principle. The proposed algorithm can give the unbiased estimates of the system model parameters in the presence of colored noises, and can be on-line implemented. Finally, the advantages of the proposed bias compensation recursive least squares algorithm are shown by simulation tests.     

Hierarchical multi-innovation stochastic gradient algorithm for Hammerstein nonlinear system modeling

This paper decomposes a Hammerstein nonlinear system into two subsystems, one containing the parameters of the linear dynamical block and the other containing the parameters of the nonlinear static block, and presents a hierarchical multi-innovation stochastic gradient identification algorithm for Hammerstein systems based on the hierarchical identification principle. The proposed algorithm is simple in principle and easy to implement on-line. A simulation example is provided to test the effectiveness of the proposed algorithm.     

On the perturbation of least squares problems with equality constraints

Some new perturbation results are presented for least squares problems with equality constraints, in which relative errors are obtained on perturbed solutions, least squares residuals, and vectors of Lagrange multipliers of the problem, based on the equivalence of the problem to a usual least squares problem and optimal perturbation results for orthogonal projections.     

On equivalence of the Gauss-Newton technique,the parameter influence coefficient technique,and the quasilinearization technique in dynamic system identification by least squares

The literature presents, among others, three general techniques for system identification by least squares. These techniques are the Gauss-Newton technique, the parameter influence technique and the quasilinearization technique. The purpose of this paper is to show the equivalence of all three techniques for the general case.     

Preconditioned conjugate gradient method for generalized least squares problems

A variant of the preconditioned conjugate gradient method to solve generalized least squares problems is presented. If the problem is min (Ax − b)^TW⁻¹(Ax − b) with A ∈ R^m×n and W ∈ R^m×m symmetric and positive definite, the method needs only a preconditioner A₁ ∈ R^n×n, but not the inverse of matrix W or of any of its submatrices. Freund's comparison result for regular least squares problems is extended to generalized least squares problems. An error bound is also given.