Hierarchical least squares based iterative estimation algorithm for multivariable Box-Jenkins-like systems using the auxiliary model

This paper presents a hierarchical least squares iterative algorithm to estimate the parameters of multivariable Box-Jenkins-like systems by combining the hierarchical identification principle and the auxiliary model identification idea. The key is to decompose a multivariable systems into two subsystems by using the hierarchical identification principle. As there exist the unmeasurable noise-free outputs and noise terms in the information vector, the solution is using the auxiliary model identification idea to replace the unmeasurable variables with the outputs of the auxiliary model and the estimated residuals. A numerical example is given to show the performance of the proposed algorithm.     

Parameter and state estimation algorithm for single-input single-output linear systems using the canonical state space models

This paper presents a new parameter and state estimation algorithm for single-input single-output systems based on canonical state space models from the given input–output data. Difficulties of identification for state space models lie in that there exist unknown noise terms in the formation vector and unknown state variables. By means of the hierarchical identification principle, those noise terms in the information vector are replaced with the estimated residuals and a new least squares algorithm is proposed for parameter estimation and the system states are computed by using the estimated parameters. Finally, an example is provided.     

Gradient based iterative parameter identification for Wiener nonlinear systems

This paper focuses on the identification problem of Wiener nonlinear output error systems. The application of the key-term decomposition technique provides a special form of the Wiener model with polynomials, where all the model parameters to be estimated are separated. To solve the identification problem of Wiener nonlinear output error systems with the unmeasurable variables in the information vector, an auxiliary model-based gradient iterative algorithm is presented by replacing the unmeasurable variables with their corresponding iterative estimates. The performances of the proposed algorithm are analyzed and compared by using numerical examples.     

A NARMAX model-based state-space self-tuning control for nonlinear stochastic hybrid systems

A novel state-space self-tuning control methodology for a nonlinear stochastic hybrid system with stochastic noise/disturbances is proposed in this paper. via the optimal linearization approach, an adjustable NARMAX-based noise model with estimated states can be constructed for the state-space self-tuning control in nonlinear continuous-time stochastic systems. Then, a corresponding adaptive digital control scheme is proposed for continuous-time multivariable nonlinear stochastic systems, which have unknown system parameters, measurement noise/external disturbances, and inaccessible system states. The proposed method enables the development of a digitally implementable advanced control algorithm for nonlinear stochastic hybrid systems.     

Multi-innovation stochastic gradient algorithm for multiple-input single-output systems using the auxiliary model

In order to reduce computational burden and improve the convergence rate of identification algorithms, an auxiliary model based multi-innovation stochastic gradient (AM-MISG) algorithm is derived for the multiple-input single-output systems by means of the auxiliary model identification idea and multi-innovation identification theory. The basic idea is to replace the unknown outputs of the fictitious subsystems in the information vector with the outputs of the auxiliary models and to present an auxiliary model based stochastic gradient algorithm, and then to derive the AM-MISG algorithm by expanding the scalar innovation to innovation vector and introducing the innovation length. The simulation example shows that the proposed algorithms work quite well.     

Two-stage least squares based iterative estimation algorithm for CARARMA system modeling

For stochastic systems described by the controlled autoregressive autoregressive moving average (CARARMA) models, a new-type two-stage least squares based iterative algorithm is proposed for identifying the system model parameters and the noise model parameters. The basic idea is based on the interactive estimation theory and to estimate the parameter vectors of the system model and the noise model, respectively. The simulation results indicate that the proposed algorithm is effective.     

Combining evolutionary and stochastic gradient techniques for system identification

In the present contribution, a novel method combining evolutionary and stochastic gradient techniques for system identification is presented. The method attempts to solve the AutoRegressive Moving Average (ARMA) system identification problem using a hybrid evolutionary algorithm which combines Genetic Algorithms (GAs) and the Least Mean Squares LMS algorithm. More precisely, LMS is used in the step of the evaluation of the fitness function in order to enhance the chromosomes produced by the GA. Experimental results demonstrate that the proposed method manages to identify unknown systems, even in cases with high additive noise. Furthermore, it is observed that, in most cases, the proposed method finds the correct order of the unknown system without using a lot of a priori information, compared to other system identification methods presented in the literature. So, the proposed hybrid evolutionary algorithm builds models that not only have small MSE, but also are very similar to the real systems. Except for that, all models derived from the proposed algorithm are stable.     

Natural gradient algorithm for stochastic distribution systems with output feedback

In this paper, we use natural gradient algorithm to control the shape of the conditional output probability density function for the stochastic distribution systems from the viewpoint of information geometry. The considered system here is of multi-input and single output with an output feedback and a stochastic noise. Based on the assumption that the probability density function of the stochastic noise is known, we obtain the conditional output probability density function whose shape is only determined by the control input vector under the condition that the output feedback is known at any sample time. The set of all the conditional output probability density functions forms a statistical manifold (M), and the control input vector and the output feedback are considered as the coordinate system. The Kullback divergence acts as the distance between the conditional output probability density function and the target probability density function. Thus, an iterative formula for the control input vector is proposed in the sense of information geometry. Meanwhile, we consider the convergence of the presented algorithm. At last, an illustrative example is utilized to demonstrate the effectiveness of the algorithm.     

The regularizing effect of the Golub-Kahan iterative bidiagonalization and revealing the noise level in the data

Regularization techniques based on the Golub-Kahan iterative bidiagonalization belong among popular approaches for solving large ill-posed problems. First, the original problem is projected onto a lower dimensional subspace using the bidiagonalization algorithm, which by itself represents a form of regularization by projection. The projected problem, however, inherits a part of the ill-posedness of the original problem, and therefore some form of inner regularization must be applied. Stopping criteria for the whole process are then based on the regularization of the projected (small) problem. In this paper we consider an ill-posed problem with a noisy right-hand side (observation vector), where the noise level is unknown. We show how the information from the Golub-Kahan iterative bidiagonalization can be used for estimating the noise level. Such information can be useful for constructing efficient stopping criteria in solving ill-posed problems.     

Randomized Kaczmarz solver for noisy linear systems

The Kaczmarz method is an iterative algorithm for solving systems of linear equations Ax=b. Theoretical convergence rates for this algorithm were largely unknown until recently when work was done on a randomized version of the algorithm. It was proved that for overdetermined systems, the randomized Kaczmarz method converges with expected exponential rate, independent of the number of equations in the system. Here we analyze the case where the system Ax=b is corrupted by noise, so we consider the system Ax≈b+r where r is an arbitrary error vector. We prove that in this noisy version, the randomized method reaches an error threshold dependent on the matrix A with the same rate as in the error-free case. We provide examples showing our results are sharp in the general context.     

Investigation of the effect of the measurement with the abrupt noise and the number of approximated terms in conductional boundary estimation problem

A direct error vector analysis of inverse heat conduction problem is presented to detect the measured noise. Based on the reverse matrix approach that the inverse problem is solved directly in a linear domain, and the error vector is formulated from the difference between the measured temperature and the estimated temperature. There is no prior knowledge on the exact solution while the error vector is constructed. The error vector is used to investigate the consistence of the measured data in the domain and lead to detect the noise data. Furthermore, the proper number of the undetermined variable is able to suggest based on the mean value of the error vector and the value of the condition number of the reverse matrix. In the first example, a test problem with the measurement noise is presented. The estimated result is influent by the noise globally. The result shows that the value of error vector is changed significantly at the coordinate of the measurement noise appeared. In other words, the error vector analysis is able to identify the noise data. In the second example, the proper number of series expansion terms is investigated. From the result, it shows that the number of expansion terms with the small mean value and condition number can better approximate to the unknown condition. It means that the proposed method is able to suggest a proper number of expansion terms when the function of the recovered boundary is unknown.     

An iterative least squares estimation algorithm for controlled moving average systems based on matrix decomposition

An iterative least squares parameter estimation algorithm is developed for controlled moving average systems based on matrix decomposition. The proposed algorithm avoids repeatedly computing the inverse of the data product moment matrix with large sizes at each iteration and has a high computational efficiency. A numerical example indicates that the proposed algorithm is effective.     

Gradient based estimation algorithm for Hammerstein systems with saturation and dead-zone nonlinearities

This paper focuses on identification problems for Hammerstein systems with saturation and dead-zone nonlinearities. An appropriate switching function is introduced to derive an identification model with fewer parameters and all the unknown parameters can be estimated by using an iterative method. A numerical simulation is carried out to show the effectiveness of the proposed method.     

Iterative algorithms for solving a class of complex conjugate and transpose matrix equations

This paper is concerned with iterative solutions to a class of complex matrix equations, which include some previously investigated matrix equations as special cases. By applying the hierarchical identification principle, an iterative algorithm is constructed to solve this class of matrix equations. A sufficient condition is presented to guarantee that the proposed algorithm is convergent for an arbitrary initial matrix with a real representation of a complex matrix as tools. By using some properties of the real representation, a convergence condition that is easier to compute is also given in terms of original coefficient matrices. A numerical example is employed to illustrate the effectiveness of the proposed methods.     

非对称线性代数方程组的块向量算法

预条件广义共轭余量法并行和向量计算的关键是预条件计算是否可并行和向量计算，我们利用分而治之的原则，构造了一处块预条件矩阵Ｍ，这里的矩阵Ｍ是通过对线性代数方程组Ａｘ＝ｆ的矩阵Ａ进行块分解，在块分解中利用近似逆技术。这样分解形成的预条件矩阵Ｍ在迭代计算时，可向量或并行计算。     

Data-driven recursive least squares methods for non-affined nonlinear discrete-time systems

Aiming at identifying nonlinear systems, one of the most challenging problems in system identification, a class of data-driven recursive least squares algorithms are presented in this work. First, a full form dynamic linearization based linear data model for nonlinear systems is derived. Consequently, a full form dynamic linearization-based data-driven recursive least squares identification method for estimating the unknown parameter of the obtained linear data model is proposed along with convergence analysis and prediction of the outputs subject to stochastic noises. Furthermore, a partial form dynamic linearization-based data-driven recursive least squares identification algorithm is also developed as a special case of the full form dynamic linearization based algorithm. The proposed two identification algorithms for the nonlinear nonaffine discrete-time systems are flexible in applications without relying on any explicit mechanism model information of the systems. Additionally, the number of the parameters in the obtained linear data model can be tuned flexibly to reduce computation complexity. The validity of the two identification algorithms is verified by rigorous theoretical analysis and simulation studies.     

Message‐Passing Algorithms for Synchronization Problems over Compact Groups

Various alignment problems arising in cryo‐electron microscopy, community detection, time synchronization, computer vision, and other fields fall into a common framework of synchronization problems over compact groups such as ℤ/L, U(1), or SO(3). The goal in such problems is to estimate an unknown vector of group elements given noisy relative observations. We present an efficient iterative algorithm to solve a large class of these problems, allowing for any compact group, with measurements on multiple “frequency channels” (Fourier modes, or more generally, irreducible representations of the group). Our algorithm is a highly efficient iterative method following the blueprint of approximate message passing (AMP), which has recently arisen as a central technique for inference problems such as structured low‐rank estimation and compressed sensing. We augment the standard ideas of AMP with ideas from representation theory so that the algorithm can work with distributions over general compact groups. Using standard but nonrigorous methods from statistical physics, we analyze the behavior of our algorithm on a Gaussian noise model, identifying phases where we believe the problem is easy, (computationally) hard, and (statistically) impossible. In particular, such evidence predicts that our algorithm is information‐theoretically optimal in many cases, and that the remaining cases exhibit statistical‐to‐computational gaps. © 2018 Wiley Periodicals, Inc.     

Iterative parameter identification methods for nonlinear functions

This paper considers identification problems of nonlinear functions fitting or nonlinear systems modelling. A gradient based iterative algorithm and a Newton iterative algorithm are presented to determine the parameters of a nonlinear system by using the negative gradient search method and Newton method. Furthermore, two model transformation based iterative methods are proposed in order to enhance computational efficiencies. By means of the model transformation, a simpler nonlinear model is achieved to simplify the computation. Finally, the proposed approaches are analyzed using a numerical example.     

Reconstruction of external actions under incomplete information in a linear stochastic equation

The problem of reconstructing unknown external actions in a linear stochastic differential equation is investigated on the basis of the approach of the theory of dynamic inversion. We consider the statement when the simultaneous reconstruction of disturbances in the deterministic and stochastic terms of the equation is performed with the use of discrete information on a number of realizations of a part of coordinates of the stochastic process. The problem is reduced to an inverse problem for systems of ordinary differential equations describing the mathematical expectation and covariance matrix of the original process. A finite-step software-oriented solution algorithm based on the method of auxiliary controlled models is proposed. We derive an estimate for its convergence rate with respect to the number of measured realizations.     

On a reconstruction algorithm for the trajectory and control in a delay system

We discuss a problem of the dynamic reconstruction of unmeasured coordinates of the phase vector and unknown controls in nonlinear vector equations with delay. A regularizing algorithm is proposed for the reconstruction of both controls and unmeasured coordinates simultaneously with the processes. The algorithm is stable with respect to information noises and computational errors.