Generalized least squares and maximum likelihood estimations of multivariate polychoric correlations

This paper investigates the generalized least squares estimation and the maximum likelihood estimation of the parameters in a multivariate polychoric correlations model, based on data from a multidimensional contingency table. Asymptotic properties of the estimators are discussed. An iterative procedure based on the Gauss-Newton algorithm is implemented to produce the generalized least squares estimates and the standard errors estimates. It is shown that via an iteratively reweighted method, the algorithm produces the maximum likelihood estimates as well. Numerical results on the finite sample behaviors of the methods are reported.     

Three-stage recursive least squares parameter estimation for controlled autoregressive autoregressive systems

A three-stage recursive least squares parameter estimation algorithm is derived for controlled autoregressive autoregressive (CARAR) systems. The basic idea is to decompose a CARAR system into three subsystems, one of which contains one parameter vector, and to identify the parameters of each subsystem one by one. Compared with the recursive generalized least squares algorithm, the dimensions of the involved covariance matrices in each subsystem become small and thus the proposed algorithm has a high computational efficiency. Finally, we verify the proposed algorithm with a simulation example.     

Recursive computational formulas of the least squares criterion functions for scalar system identification

The paper discusses recursive computation problems of the criterion functions of several least squares type parameter estimation methods for linear regression models, including the well-known recursive least squares (RLS) algorithm, the weighted RLS algorithm, the forgetting factor RLS algorithm and the finite-data-window RLS algorithm without or with a forgetting factor. The recursive computation formulas of the criterion functions are derived by using the recursive parameter estimation equations. The proposed recursive computation formulas can be extended to the estimation algorithms of the pseudo-linear regression models for equation error systems and output error systems. Finally, the simulation example is provided.     

Recursive prediction error identification of fractional order models

This paper deals with time domain identification of fractional order systems. A new identification technique is developed providing recursive parameters estimation of fractional order models. The identification model is defined by a generalized ARX structure obtained by discretization of a continuous fractional order differential equation. The parameters are then estimated using the recursive least squares and the recursive instrumental variable algorithms extended to fractional order cases. Finally, the quality of the proposed technique is illustrated and compared through the identification of simulated fractional order systems.     

Method of Moments Using Monte Carlo Simulation

Abstract

We present a computational approach to the method of moments using Monte Carlo simulation. Simple algebraic identities are used so that all computations can be performed directly using simulation draws and computation of the derivative of the log-likelihood. We present a simple implementation using the Newton-Raphson algorithm with the understanding that other optimization methods may be used in more complicated problems. The method can be applied to families of distributions with unknown normalizing constants and can be extended to least squares fitting in the case that the number of moments observed exceeds the number of parameters in the model. The method can be further generalized to allow “moments” that are any function of data and parameters, including as a special case maximum likelihood for models with unknown normalizing constants or missing data. In addition to being used for estimation, our method may be useful for setting the parameters of a Bayes prior distribution by specifying moments of a distribution using prior information. We present two examples—specification of a multivariate prior distribution in a constrained-parameter family and estimation of parameters in an image model. The former example, used for an application in pharmacokinetics, motivated this work. This work is similar to Ruppert's method in stochastic approximation, combines Monte Carlo simulation and the Newton-Raphson algorithm as in Penttinen, uses computational ideas and importance sampling identities of Gelfand and Carlin, Geyer, and Geyer and Thompson developed for Monte Carlo maximum likelihood, and has some similarities to the maximum likelihood methods of Wei and Tanner.     

Nonlinear estimation applying an unscented transformation in systems with correlated uncertain observations

An approximation to the least squares filter is proposed for discrete signals whose evolution is governed by nonlinear functions, when the estimation is based on nonlinear observations with additive noise which can consist only of random noise; this uncertainty in the observation process is modelled by Bernoulli random variables which are correlated at consecutive time instants and are otherwise independent. The proposed recursive approximation is based on the unscented principle; successive applications of the unscented transformation to a suitable augmented state vector enable us to approximate the one-stage state and observation predictors from the state filter at the previous time instant. The performance of the proposed algorithm is compared with that of an extended algorithm in a numerical simulation example.     

Monte Carlo maximum likelihood circle fitting using circular density functions

Finding the “best-fitting” circle to describe a set of points in two dimensions is discussed in terms of maximum likelihood estimation. Several combinations of distributions are proposed to describe the stochastic nature of points in the plane, as the points are considered to have a common, typically unknown center, a random radius, and random angular orientation. A Monte Carlo search algorithm over part of the parameter space is suggested for finding the maximum likelihood parameter estimates. Examples are presented, and comparisons are drawn between circles fit by this proposed method, least squares, and other maximum likelihood methods found in the literature.     

Computationally Efficient Estimation for the Generalized Odds Rate Mixture Cure Model With Interval-Censored Data

For semiparametric survival models with interval-censored data and a cure fraction, it is often difficult to derive nonparametric maximum likelihood estimation due to the challenge in maximizing the complex likelihood function. In this article, we propose a computationally efficient EM algorithm, facilitated by a gamma-Poisson data augmentation, for maximum likelihood estimation in a class of generalized odds rate mixture cure (GORMC) models with interval-censored data. The gamma-Poisson data augmentation greatly simplifies the EM estimation and enhances the convergence speed of the EM algorithm. The empirical properties of the proposed method are examined through extensive simulation studies and compared with numerical maximum likelihood estimates. An R package “GORCure” is developed to implement the proposed method and its use is illustrated by an application to the Aerobic Center Longitudinal Study dataset. Supplementary material for this article is available online.     

Functional relationships having many independent variables and errors with multivariate normal distribution

This paper deals with maximum likelihood estimation of linear or nonlinear functional relationships assuming that replicated observations have been made on p variables at n points. The joint distribution of the pn errors is assumed to be multivariate normal. Existing results are extended in two ways: first, from known to unknown error covariance matrix; second, from the two variate to the multivariate case.For the linear relationship it is shown that the maximum likelihood point estimates are those obtained by the method of generalized least squares. The present method, however, has the advantage of supplying estimates of the asymptotic covariances of the structural parameter estimates.     

Auxiliary model based parameter estimation for dual-rate output error systems with colored noise

The dual-rate sampled-data systems can offer better quality of control than the systems with single sampling rate in practice. However, the conventional identification methods run in the single-rate scheme. This paper focuses on the parameter estimation problems of the dual-rate output error systems with colored noises. Based on the dual-rate sampled and noise-contaminated data, two direct estimation algorithms are addressed: the auxiliary model based recursive extended least squares algorithm and the recursive prediction error method. The auxiliary model is employed to estimate the noise-free system output. An example is given to test and illustrate the proposed algorithms.     

面板数据空间误差分量模型的GMM估计及其应用

本文研究面板数据空间误差分量模型(Spatial Error Components Model,SEC)的估计方法。为克服极大似然法在SEC模型估计中运算的困难,本文提出基于广义矩估计的可行广义最小二乘法(GMM-GLS),证明了估计量的一致性及有限样本下的有效性;并应用此模型,研究2000-2007年中国30个省(西藏除外)的物质资本存量、人力资本存量及能源消耗对实际GDP的影响,结果表明,采用SEC模型所得估计结果更为符合经济现实。     

A modified candecomp algorithm for fitting the latent class model: Implementation and evaluation

In this paper an implementation is discussed of a modified CANDECOMP algorithm for fitting Lazarsfeld's latent class model. The CANDECOMP algorithm is modified such that the resulting parameter estimates are non-negative and ‘best asymptotically normal’. In order to achieve this, the modified CANDECOMP algorithm minimizes a weighted least squares function instead of an unweighted least squares function as the traditional CANDECOMP algorithm does. To evaluate the new procedure, the modified CANDECOMP procedure with different weighting schemes is compared on five published data sets with the widely-used iterative proportional fitting procedure for obtaining maximum likelihood estimates of the parameters in the latent class model. It is found that, with appropriate weights, the modified CANDECOMP algorithm yields solutions that are nearly identical with those obtained by means of the maximum likelihood procedure. While the modified CANDECOMP algorithm tends to be computationally more intensive than the maximum likelihood method, it is very flexible in that it easily allows one to try out different weighting schemes.     

Second-order risk comparison of SLSE with GLSE and MLE in a regression with serial correlation

First, the second-order bias of the estimator of the autoregressive parameter based on the ordinary least squares residuals in a linear model with serial correlation is given. Second, the second-order expansion of the risk matrix of a generalized least squares estimator with the above estimated parameter is obtained. This expansion is the same as that based on a suitable estimator of the autoregressive parameter independent of the sample. Third, it is shown that the risk matrix of the generalized least squares estimator is asymptotically equivalent to that of the maximum likelihood estimator up to the second order. Last, a sufficient condition is given for the term due to the estimation of the autoregressive parameter in this expansion to vanish under Grenander's condition for the explanatory variates.     

The exponentiated generalized power Lindley distribution: Properties and applications

In this paper, we introduce a new extension of the power Lindley distribution, called the exponentiated generalized power Lindley distribution. Several mathematical properties of the new model such as the shapes of the density and hazard rate functions, the quantile function, moments, mean deviations, Bonferroni and Lorenz curves and order statistics are derived.Moreover, we discuss the parameter estimation of the new distribution using the maximum likelihood and diagonally weighted least squares methods. A simulation study is performed to evaluate the estimators. We use two real data sets to illustrate the applicability of the new model. Empirical findings show that the proposed model provides better fits than some other well-known extensions of Lindley distributions.     

Combined state and least squares parameter estimation algorithms for dynamic systems

The control theory and automation technology cast the glory of our era. Highly integrated computer chip and automation products are changing our lives. Mathematical models and parameter estimation are basic for automatic control. This paper discusses the parameter estimation algorithm of establishing the mathematical models for dynamic systems and presents an estimated states based recursive least squares algorithm, and the states of the system are computed through the Kalman filter using the estimated parameters. A numerical example is provided to confirm the effectiveness of the proposed algorithm.     

State of charge estimation for lithium-ion batteries based on adaptive dual Kalman filter

Accurate estimation of the battery state of charge (SOC) is of great significance for enhancing its service life and safety. In this study, based on the fractional-order equivalent circuit model of lithium-ion battery, the SOC estimation methods using dual Kalman filter (DKF) and dual extended Kalman filter (DEKF) are simulated and compared, in terms of model accuracy and SOC estimation accuracy. Then, combining the advantages of the DKF and DEKF algorithms, an SOC estimation algorithm based on adaptive double Kalman filter is proposed. This algorithm uses the recursive least squares (RLS) method to update the battery model parameters online in real time, and employs the DKF algorithm to filter the SOC twice to reduce the interferences from the battery model error and the current measurement error. In the experimental studies, the measured SOC values are compared with the estimated SOC values produced by the proposed algorithm. The comparison results show that SOC estimation error of the proposed algorithm is within the range of ±0.01 under most test conditions, and it can automatically correct SOC to true value in the presence of system errors. Thus, the validity, accuracy, robustness and adaptability of the proposed algorithm under different operation conditions are verified.     

Bias compensation methods for stochastic systems with colored noise

For ARX-like systems, this paper derives a bias compensation based recursive least squares identification algorithm by means of the prefilter idea and bias compensation principle. The proposed algorithm can give the unbiased estimates of the system model parameters in the presence of colored noises, and can be on-line implemented. Finally, the advantages of the proposed bias compensation recursive least squares algorithm are shown by simulation tests.     

Estimation of Parameters of the Generalized Pareto\Distribution by the Least Squares

Traditional estimations of parameters of the generalized Pareto distribution (GPD) are generally constrained by the shape parameter of GPD. Such as: the method-of-moments (MOM), the probability-weighted moments (PWM), L-moments (LM), the maximum likelihood estimation (MLE) and so on. In this paper we use the fact that GPD can be transformed into the exponential distribution and use the results of parameters estimation for the exponential distribution, than we propose parameters estimators of the two-parameter or three-parameter GPD by the least squares method. Some asymptotic results are provided and the proposed method not constrained by the shape parameter of GPD. A simulation study is carried out to evaluate the performance of the proposed method and to compare them with other methods suggested in this paper. The simulation results indicate that the proposed method performs better than others in some common situation.     

Exponential family non-linear models for categorical data with errors of observation

We present a general framework for treating categorical data with errors of observation. We show how both latent class models and models for doubly sampled data can be treated as exponential family nonlinear models. These are extended generalized linear models with the link function substituted by an observationwise defined non-linear function of the model parameters. The models are formulated in terms of structural probabilities and conditional error probabilities, thus allowing natural constraints when modelling errors of observation. We use an iteratively reweighted least squares procedure for obtaining maximum likelihood estimates. This is faster than the traditionally used EM algorithm and the computations can be made in GLIM.¹ As examples we analyse three sets of categorical data with errors of observation which have been analysed before by Ashford and Sowden,² Goodman³ and Chen,⁴ respectively.     

An efficient generalized least squares algorithm for periodic trended regression with autoregressive errors

Time series data with periodic trends like daily temperatures or sales of seasonal products can be seen in periods fluctuating between highs and lows throughout the year. Generalized least squares estimators are often computed for such time series data as these estimators have minimum variance among all linear unbiased estimators. However, the generalized least squares solution can require extremely demanding computation when the data is large. This paper studies an efficient algorithm for generalized least squares estimation in periodic trended regression with autoregressive errors. We develop an algorithm that can substantially simplify generalized least squares computation by manipulating large sets of data into smaller sets. This is accomplished by coining a structured matrix for dimension reduction. Simulations show that the new computation methods using our algorithm can drastically reduce computing time. Our algorithm can be easily adapted to big data that show periodic trends often pertinent to economics, environmental studies, and engineering practices.