Gradient-Based Identification Methods for Hammerstein Nonlinear ARMAX Models

Two identification algorithms, an iterative gradient and a recursive stochastic gradient based, are developed for a Hammerstein nonlinear ARMAX model, a linear dynamical block following a memoryless nonlinear block. The basic idea is to use the gradient search principle, to replace unmeasurable noise terms in the information vectors by their estimates, and to compute iteratively or recursively the noise estimates based on the obtained parameter estimates. Convergence analysis of the recursive stochastic gradient algorithm indicates that the parameter estimation error consistently converges to zero under certain conditions. The simulation results show the effectiveness of the proposed algorithms. An erratum to this article is available at .     

Least squares algorithm for an input nonlinear system with a dynamic subspace state space model

For a Hammerstein input nonlinear system with a subspace state space linear element, this paper transforms the system into a bilinear identification model by using the property of the shift operator to the state space model and presents a recursive and an iterative least squares algorithms to generate parameter estimates and state estimates by using the hierarchical identification principle and by replacing the unknown state variables with their estimates. The proposed approaches are computationally more efficient than the over-parameterization model based least squares method.     

MILM hybrid identification method of fractional order neural-fuzzy Hammerstein model

Aiming at the difficult identification of fractional order Hammerstein nonlinear systems, including many identification parameters and coupling variables, unmeasurable intermediate variables, difficulty in estimating the fractional order, and low accuracy of identification algorithms, a multiple innovation Levenberg–Marquardt algorithm (MILM) hybrid identification method based on the fractional order neuro-fuzzy Hammerstein model is proposed. First, a fractional order discrete neuro-fuzzy Hammerstein system model is constructed; secondly, the neuro-fuzzy network structure and network parameters are determined based on fuzzy clustering, and the self-learning clustering algorithm is used to determine the antecedent parameters of the neuro-fuzzy network model; then the multiple innovation principle is combined with the Levenberg–Marquardt algorithm, and the MILM hybrid algorithm is used to estimate the linear module parameters and fractional order. Finally, the academic example of the fractional order Hammerstein nonlinear system and the example of a flexible manipulator are identified to prove the effectiveness of the proposed algorithm.

     

Newton iterative identification method for an input nonlinear finite impulse response system with moving average noise using the key variables separation technique

This paper studies parameter identification problems for input nonlinear finite impulse response systems with moving average noise (i.e., input nonlinear finite impulse response moving average systems). Since the identification model of the system contains the product of the parameters of the nonlinear part and the linear part, we use the key variables separation technique and express the output of the system as the linear combination of all parameters, and then derive a Newton iterative identification method. The simulation results show that the proposed algorithm is effective.     

Recursive least squares parameter estimation algorithm for dual-rate sampled-data nonlinear systems

This paper focuses on the identification problem of Hammerstein systems with dual-rate sampling. Using the key-term separation principle, we derive a regression identification model with different input updating and output sampling rates. To solve the identification problem of the dual-rate Hammerstein systems with the unmeasurable variables in the information vector, an auxiliary model-based recursive least squares algorithm is presented by replacing the unmeasurable variables with their corresponding recursive estimates. Convergence properties of the algorithm are analyzed. Simulation results show that the proposed algorithm can estimate the parameters of a class of nonlinear systems.     

Identification of nonlinear Hammerstein system using mixed integer-real coded particle swarm optimization: application to the electric daily peak-load forecasting

This paper investigates the modeling of a class of dynamic systems using nonlinear Hammerstein (NLH) model composed of a memory-less polynomial block cascaded to an autoregressive with exogenous input (ARX) time-series block. The model thus defined is known as NLHARX. Both the integer orders and the real coefficients of the model are identified simultaneously in a unified framework using a new algorithm based on a mixed coded integer-real particle swarm optimization. Unlike classical identification methods which assume the orders to be known in advance, the proposed approach is new since it estimates both the real and integer design parameters while minimizing the error between the outputs of the system and the model. The usefulness and the effectiveness of the proposed approach have been demonstrated through extensive simulations. Two illustrative examples are included in this paper: an empirical example and an application to the forecasting of the daily peak-load of Hail region, Saudi Arabia. Future works will be devoted to the identification of more complex dynamic systems, such as Hammerstein–Wiener and the application to the prediction of time-series related to water and energy.     

Multistage least squares based iterative estimation for feedback nonlinear systems with moving average noises using the hierarchical identification principle

This paper develops a multistage least squares based iterative algorithm to estimate the parameters of feedback nonlinear systems with moving average noise from input–output data. Since that the identification model is bilinear on the unknown parameter space, the solution is to decompose a system into several subsystems with each of which is linear about its parameter vector, then to replace the unknown noise terms in the information vectors with their corresponding estimates at the previous iteration of each subsystem, and estimate each subsystem, respectively. The simulation results show that the proposed algorithm can work well.     

Maximum likelihood Newton recursive and the Newton iterative estimation algorithms for Hammerstein CARAR systems

This paper discusses the identification problems of Hammerstein controlled autoregressive autoregressive (CARAR) systems using the maximum likelihood principle and Newton optimization method. A Newton recursive algorithm and a Newton iterative algorithm using the maximum likelihood principle are presented. The simulation results show that the proposed algorithms can effectively estimate the parameters of the Hammerstein CARAR systems.     

Gradient-based iterative identification for Wiener nonlinear systems with non-uniform sampling

This paper focuses on the identification problem of Wiener nonlinear systems with non-uniform sampling. The mathematical model for the Wiener nonlinear system is established from the non-uniformly sampled input–output data. In order to solve the identification problem of the Wiener nonlinear system with the unmeasurable variables in the information vector, the gradient-based iterative algorithm is presented by replacing the unmeasurable variables with their corresponding iterative estimates. Finally, the simulation results indicate that the proposed algorithm is effective.     

Identification for disturbed MIMO Wiener systems

The identification of Multi-input Multi-output (MIMO) Wiener systems is concerned in this paper. The system presented is comprised of a multi-dimensional linear subsystem and a memory-less nonlinear block which is made of discontinuous asymmetric piece-wise linear functions. A recursive algorithm is proposed to estimate all the unknown parameters of the system with interference noises. It is shown that the recursive algorithm for the disturbed MIMO Wiener system is convergent. Finally, some simulation results illustrate the identification accuracy and the convergence rate.     

Variable knot-based spline approximation recursive Bayesian algorithm for the identification of Wiener systems with process noise

Wiener systems consist of a linear dynamic block in cascade with static nonlinearity. One of the challenging issues in the identification of a process noise disturbed Wiener system is that the influence of noise is difficult to eliminate. For Wiener systems with process noise, traditional algorithms will result in biased estimates. To solve this problem, a novel recursive Bayesian algorithm based on variable knot spline approximation is proposed in this paper. First, a spline function is taken to approximate the inverse function of the nonlinear part, which can achieve excellent extrapolation and eliminate oscillatory behaviors. A knot selection method is then presented to achieve accurate estimates. Furthermore, a knot variation strategy to improve the accuracy of the spline approximation is described. Finally, the proposed algorithm is validated through a numerical simulation.     

Online system identification using fractional-order Hammerstein model with noise cancellation

Slow convergence and low accuracy are two main drawbacks in nonlinear system identification methods. It becomes more complicated when time delay and noises are considered. In this paper, considering a fractional-order Hammerstein model, an online identification method is proposed. A combination of an evolutionary optimization method and recursive least square algorithm is used to estimate the system parameters and orders in the presence of unknown noises. Finally, simulation results are taken to prove the effectiveness of the proposed algorithm.

     

Hierarchical recursive least squares algorithm for Hammerstein systems using the filtering method

A hierarchical recursive least squares algorithm is presented in the paper to estimate the parameters of Hammerstein nonlinear systems by combining the filtering method and least squares search principle. The key is to decompose the Hammerstein system into two subsystems by adopting the hierarchical idea. Numerical examples are given to illustrate the performance of the proposed algorithm.     

A novel APSO-aided maximum likelihood identification method for Hammerstein systems

Identification of Hammerstein nonlinear models has received much attention due to its ability to describe a wide variety of nonlinear systems. In this paper the maximum likelihood estimator which was originally derived for linear systems is extended to work for Hammerstein nonlinear systems in colored-noise environment. The maximum likelihood estimate is known to be statistically efficient, but can lead to complex nonlinear multidimensional optimization problem; traditional methods solve this problem at the computational cost of evaluating second derivatives. To overcome these shortcomings, a particle swarm optimization (PSO) aided maximum likelihood identification algorithm (Maximum Likelihood-Particle Swarm Optimization, ML-PSO) is first proposed to integrate PSO’s simplicity in implementation and computation, and its ability to quickly converge to a reasonably good solution. Furthermore, a novel adaptive strategy using the evolution state estimation technique is proposed to improve PSO’s performance (maximum likelihood-adaptive particle swarm optimization, ML-APSO). A simulation example shows that ML-APSO method outperforms ML-PSO and traditional recursive least square method in various noise conditions, and thus proves the effectiveness of the proposed identification scheme.     

Highly computationally efficient parameter estimation algorithms for a class of nonlinear multivariable systems by utilizing the state estimates

This paper investigates the parameter estimation issue for an input nonlinear multivariable state-space system. First, the canonical form of the input nonlinear multivariable state-space system is obtained through the linear transformation and the over-parameterization identification model of the considered system is derived. Second, by cutting down the redundant parameter estimates and extracting the unique parameter estimates from the parameter estimation vector in the least-squares identification method, we present an over-parameterization-based partially coupled average recursive extended least-squares parameter estimation algorithm to estimate the parameters. As for the unknown states in the parameter estimation algorithm, a new state estimator is designed to generate the state estimates. Third, in order to improve the computational efficiency of the parameter estimation algorithm, an over-parameterization-based multi-stage partially coupled average recursive extended least-squares algorithm is proposed. Finally, the computational efficiency analysis and the simulation examples are given to verify the effectiveness of the proposed algorithms.

     

Several gradient-based iterative estimation algorithms for a class of nonlinear systems using the filtering technique

This paper considers iterative identification problems for a class of nonlinear systems with colored noises, which can be described by a linear-in-parameters output error autoregressive model. A gradient-based iterative (GI) algorithm, a filtered GI algorithm, and a filtered three-stage GI algorithm are developed using the decomposition technique and filtering technique, and their computational efficiencies are analyzed and compared. The simulation results indicate that the proposed algorithms can estimate effectively the parameters of nonlinear systems.     

Newton iterative identification for a class of output nonlinear systems with moving average noises

This paper discusses iterative identification problems for a class of output nonlinear systems (i.e., Wiener nonlinear systems) with moving average noises from input–output measurement data, based on the Newton iterative method. The basic idea is to decompose a nonlinear system into two subsystems, to replace the unknown variables in the information vectors with their corresponding estimates at the previous iteration, and to present a Newton iterative identification method using the hierarchical identification principle. The numerical simulation results indicate that the proposed algorithms are effective.     

A modified time domain subspace method for nonlinear identification based on nonlinear separation strategy

Nonlinear factors existing in engineering structures have drawn considerable attention, and nonlinear identification is a competent technique to understand the dynamic characteristics of nonlinear structures. Therefore, in this paper, a novel nonlinear separation subspace identification (NSSI) algorithm based on subspace algorithm and nonlinear separation strategy is proposed to conduct nonlinear parameter identification of nonlinear structures. For the proposed NSSI algorithm, the low-level excitation test is firstly conducted to obtain the transfer matrix in the linear response formula. Then, the obtained transfer matrix is used in the high-level excitation test to calculate the nonlinear response part by the proposed nonlinear separation strategy, and the subspace algorithm is utilized to identify the nonlinear parameter on the modified state-space model including only the nonlinear part. The proposed NSSI algorithm can reduce the coupling error caused by simultaneously processing both the large number part (corresponding to the linear part) and small number part (corresponding to the nonlinear part) in the traditional nonlinear subspace identification (NSI) algorithm. At last, two numerical experiments are given to validate the effectiveness of the developed novel nonlinear identification method. Furthermore, some influence factors are discussed to show the stability of the identification algorithm, and some comparisons between the proposed NSSI method and traditional NSI method are also conducted to demonstrate the advantages of the novel method.     

Filtering based recursive least squares algorithm for Hammerstein FIR-MA systems

We consider the parameter estimation problem for Hammerstein finite impulse response (FIR) systems. An estimated noise transfer function is used to filter the input–output data of the Hammerstein system. By combining the key-term separation principle and the filtering theory, a recursive least squares algorithm and a filtering-based recursive least squares algorithm are presented. The proposed filtering-based recursive least squares algorithm can estimate the noise and system models. The given examples confirm that the proposed algorithm can generate more accurate parameter estimates and has a higher computational efficiency than the recursive least squares algorithm.     

Instrumental variables approach to identification of a class of MIMO Wiener systems

A new approach to identification of multi-input multi-output (MIMO) Wiener systems using the instrumental variables method is presented. It is assumed that static nonlinear elements are invertible and their inverse characteristics can be expressed or approximated by polynomials of known orders. It is also assumed that the linear part of the Wiener system can be represented by a matrix polynomial form. Based on these assumptions, the Wiener system is transformed introducing a new parameterization and its parameters are estimated using a linear-in-parameters model. To solve the problem of non-consistency of least squares parameter estimates, an instrumental variables method is employed. A numerical example is included to show the effectiveness and the practical feasibility of the presented approach.