Parameter and state estimation algorithm for single-input single-output linear systems using the canonical state space models

This paper presents a new parameter and state estimation algorithm for single-input single-output systems based on canonical state space models from the given input–output data. Difficulties of identification for state space models lie in that there exist unknown noise terms in the formation vector and unknown state variables. By means of the hierarchical identification principle, those noise terms in the information vector are replaced with the estimated residuals and a new least squares algorithm is proposed for parameter estimation and the system states are computed by using the estimated parameters. Finally, an example is provided.     

基于固定效应的纵向数据分位点回归模型的参数估计及CDM和MSOM的等价性

研究了基于固定效应的纵向数据模分位点回归模型的参数估计及统计诊断问题.首先给出了参数估计的MM迭代算法,然后讨论了统计诊断中数据删除模型(CDM)和均值移模型(MSOM)的等价性问题,最后利用消炎镇痛药数据说明了方法的应用.     

AR模型辨识的稀疏结构迭代算法

模型估计是机器学习领域一个重要的研究内容,动态数据的模型估计是系统辨识和系统控制的基础.针对AR时间序列模型辨识问题,证明了在给定阶数下AR模型参数的最小二乘估计本质上也是一种矩估计.根据结构风险最小化原理,通过对模型拟合度和模型复杂度的折衷,提出了基于稀疏结构迭代的AR序列模型估计算法,并讨论了基于广义岭估计的最优正则化参数选取规则.数值结果表明,方法能以节省参数的方式有效地实现AR模型的辨识,比矩估计法结果有明显改善.     

Recursive computational formulas of the least squares criterion functions for scalar system identification

The paper discusses recursive computation problems of the criterion functions of several least squares type parameter estimation methods for linear regression models, including the well-known recursive least squares (RLS) algorithm, the weighted RLS algorithm, the forgetting factor RLS algorithm and the finite-data-window RLS algorithm without or with a forgetting factor. The recursive computation formulas of the criterion functions are derived by using the recursive parameter estimation equations. The proposed recursive computation formulas can be extended to the estimation algorithms of the pseudo-linear regression models for equation error systems and output error systems. Finally, the simulation example is provided.     

双边定时截尾样本下广义逆指数分布形状参数的估计

先给出了广义逆指数分布在双边定时截尾样本下形状参数的最大似然估计,并不能得到估计的显式表达式,但证明了参数在(0,+∞)上最大似然估计是唯一存在的.其次提出用EM算法求出形状参数的估计且该估计具有良好的收敛性,还给出了形状参数的EM估计的渐近方差和近似置信区间;最后通过数值模拟,对形状参数的最大似然估计和EM估计的效果进行了比较,说明了用EM算法求形状参数的估计是可行的,并且模拟效果相对比较好.     

Global optimization of nonlinear least-squares problems by branch-and-bound and optimality constraints

We study a simple, yet unconventional approach to the global optimization of unconstrained nonlinear least-squares problems. Non-convexity of the sum of least-squares objective in parameter estimation problems may often lead to the presence of multiple local minima. Here, we focus on the spatial branch-and-bound algorithm for global optimization and experiment with one of its implementations, BARON (Sahinidis in J. Glob. Optim. 8(2):201–205, 1996), to solve parameter estimation problems. Through the explicit use of first-order optimality conditions, we are able to significantly expedite convergence to global optimality by strengthening the relaxation of the lower-bounding problem that forms a crucial part of the spatial branch-and-bound technique. We analyze the results obtained from 69 test cases taken from the statistics literature and discuss the successes and limitations of the proposed idea. In addition, we discuss software implementation for the automation of our strategy.     

Mixed Model-Based Hazard Estimation

This article proposes a new method for estimation of the hazard function from a set of censored failure time data, with a view to extending the general approach to more complicated models. The approach is based on a mixed model representation of penalized spline hazard estimators. One payoff is the automation of the smoothing parameter choice through restricted maximum likelihood. Another is the option to use standard mixed model software for automatic hazard estimation.     

An implicit least squares algorithm for nonlinear rational model parameter estimation

In this study a new insight into least squares regression is identified and immediately applied to estimating the parameters of nonlinear rational models. From the beginning the ordinary explicit expression for linear in the parameters model is expanded into an implicit expression. Then a generic algorithm in terms of least squares error is developed for the model parameter estimation. It has been proved that a nonlinear rational model can be expressed as an implicit linear in the parameters model, therefore, the developed algorithm can be comfortably revised for estimating the parameters of the rational models. The major advancement of the generic algorithm is its conciseness and efficiency in dealing with the parameter estimation problems associated with nonlinear in the parameters models. Further, the algorithm can be used to deal with those regression terms which are subject to noise. The algorithm is reduced to an ordinary least square algorithm in the case of linear or linear in the parameters models. Three simulated examples plus a realistic case study are used to test and illustrate the performance of the algorithm.     

Learning curve parameter estimation beyond traditional statistics

Due to decreasing order quantities, increasing product variety and fluctuating production orders, manufacturing companies have been encountering an increased occurrence of repetitive learning-forgetting phenomenon. In this paper, deterministic methods for the learning curve parameter estimation from the limited production data available from the unstable production environment are studied. Two main learning curve models: cumulative average (Wright) and unit (Crawford) were considered and several different mathematically proven methods were proposed for the parameter estimation. The calculation results illustrated that learning curve parameters can be unequivocally estimated from the limited production data (single random sample) by using deterministic methods for both of the learning curve models, although more accurate estimation was provided by the cumulative average model based methods. Newly proposed methods enable sufficiently accurate parameter estimation from the limited production data where traditional statistical parameter estimation methods cannot be applied.     

Parameter estimation of chaotic system with time-delay: A differential evolution approach

It is an important issue to estimate parameters of chaotic system in nonlinear science. In this paper, parameter estimation problem of chaotic system with time-delay is considered. Parameters and time-delay are estimated together by treating time-delay as an additional parameter. The parameter estimation problem is converted to an multi-dimensional optimization problem. A differential evolution (DE) algorithm, which possess a powerful searching capability for finding the solutions for a given optimization problem, is applied to solve this optimization problem. Two illustrative examples are given to verify the effectiveness of the proposed method.     

Filter‐based portfolio strategies in an HMM setting with varying correlation parametrizations

We consider portfolio optimization in a regime‐switching market. The assets of the portfolio are modeled through a hidden Markov model (HMM) in discrete time, where drift and volatility of the single assets are allowed to switch between different states. We consider different parametrizations of the involved asset covariances: statewise uncorrelated assets (though linked through the common Markov chain), assets correlated in a state‐independent way, and assets where the correlation varies from state to state. As a benchmark, we also consider a model without regime switches. We utilize a filter‐based expectation‐maximization (EM) algorithm to obtain optimal parameter estimates within this multivariate HMM and present parameter estimators in all three HMM settings. We discuss the impact of these different models on the performance of several portfolio strategies. Our findings show that for simulated returns, our strategies in many settings outperform naïve investment strategies, like the equal weights strategy. Information criteria can be used to detect the best model for estimation as well as for portfolio optimization. A second study using real data confirms these findings.     

The residual based interactive stochastic gradient algorithms for controlled moving average models

This paper derives a residual based interactive stochastic gradient (ISG) parameter estimation algorithm for controlled moving average (CMA) models and studied the performance of the residual based ISG algorithm under weaker conditions on statistical properties of the noise. Compared with the residual based extended stochastic gradient algorithm for identifying CMA models, the proposed ISG algorithm can give highly accurate parameter estimates by the simulation example.     

A Soft-Computing Algorithm-based Cognitive Observer for an Elastic Robotic Arm

This paper proposes the use of Support Vector Machine (SVM) algorithm for modeling and states estimation of an elastic robotic arm. Due to the complexity of the elastic robotic arm, an accurate mathematical model and large number of sensors are needed to achieve accurate estimation. Initially, it is assumed that all system states are measurable for a short period of time. Additionally, the system output and input signals are being continuously measured. The modeling module builds two models, the internal model which will capture the dynamics of the elastic robotic arm and the data-driven observer model which will estimate the system states. The internal model and data-driven observer are obtained based on the experimental measurements using SVM algorithm in contrast to classical methods that uses the mathematical system model to drive an observer. The internal model is able to make multi-steps ahead predictions of all system states; therefore it can be used to generate a suitable control strategy. Once the models are ready, the main sensors can be removed or turned off. The proposed modeling module will eliminate the need for mathematical modeling and reduces the number of permanent sensors needed. If the main sensors are removed completely, the hardware price can be radically reduced. The simulation result demonstrates the efficiency and high performance of the modeling module. (© 2014 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim)     

Convergence properties of the least squares estimation algorithm for multivariable systems

This paper focuses on the convergence properties of the least squares parameter estimation algorithm for multivariable systems that can be parameterized into a class of multivariate linear regression models. The performance analysis of the algorithm by using the stochastic process theory and the martingale convergence theorem indicates that the parameter estimation errors converge to zero under weak conditions. The simulation results validate the proposed theorem.     

Efficient Algorithm for Computing Maximum Likelihood Estimates in Linear Transformation Models

Linear transformation models, which have been extensively studied in survival analysis, include the two special cases: the proportional hazards model and the proportional odds model. Nonparametric maximum likelihood estimation is usually used to derive the efficient estimators. However, due to the large number of nuisance parameters, calculation of the nonparametric maximum likelihood estimator is difficult in practice, except for the proportional hazards model. We propose an efficient algorithm for computing the maximum likelihood estimates, where the dimensionality of the parameter space is dramatically reduced so that only a finite number of equations need to be solved. Moreover, the asymptotic variance is automatically estimated in the computing procedure. Extensive simulation studies indicate that the proposed algorithm works very well for linear transformation models. A real example is presented for an illustration of the new methodology.     

用函数求根法解系统控制问题

提供了一大类系统控制问题的求解路线.当所考察的问题可转化为参数估计时,可以把问题进一步转化为未知回归函数求根(根即待估参数)的问题,而扩展截尾的随机逼近算法是解决这类求根问题的恰当工具.给出了算法的一般收敛定理,它已在一系列系统控制问题中得到应用.以ARMA过程的辨识,Hammerstein系统的适应调节为例,展示了上述求解路线的具体实现,并附有相应的模拟计算实例.这种方法提供的估计是递推的,并且以概率1收敛到真值.     

Parameter estimation of delay differential equations: An integration-free LS-SVM approach

This paper introduces an estimation method based on Least Squares Support Vector Machines (LS-SVMs) for approximating time-varying as well as constant parameters in deterministic parameter-affine delay differential equations (DDEs). The proposed method reduces the parameter estimation problem to an algebraic optimization problem. Thus, as opposed to conventional approaches, it avoids iterative simulation of the given dynamical system and therefore a significant speedup can be achieved in the parameter estimation procedure. The solution obtained by the proposed approach can be further utilized for initialization of the conventional nonconvex optimization methods for parameter estimation of DDEs. Approximate LS-SVM based models for the state and its derivative are first estimated from the observed data. These estimates are then used for estimation of the unknown parameters of the model. Numerical results are presented and discussed for demonstrating the applicability of the proposed method.     

不完全信息下型截尾Weibull分布参数的极大似然估计及贝叶斯估计

给出了不完全信息下 型截尾weibull分布参数的极大似然估计、无信息先验Bayes估计及多层Bayes估计,并指出针对一些具体模型还可以通过随机模拟来比较其估计精度.     

A random model approach for the LASSO

The least absolute selection and shrinkage operator (LASSO) is a method of estimation for linear models similar to ridge regression. It shrinks the effect estimates, potentially shrinking some to be identically zero. The amount of shrinkage is governed by a single parameter. Using a random model formulation of the LASSO, this parameter can be specified as the ratio of dispersion parameters. These parameters are estimated using an approximation to the marginal likelihood of the observed data. The observed score equations from the approximation are biased and hence are adjusted by subtracting an empirical estimate of the expected value. After estimation, the model effects can be tested (via simulation) as the distribution of the observed data given that all model effects are zero is known. Two related simulation studies are presented that show that dispersion parameter estimation results in effect estimates that are competitive with other estimation methods (including other LASSO methods).     

Experimental design for distributed parameter vector systems

We formulate an optimal design problem for the selection of best states to observe and optimal sampling times and locations for parameter estimation or inverse problems involving complex nonlinear partial differential systems. An iterative algorithm for implementation of the resulting methodology is proposed.