Maximum integrated likelihood estimator of the interest parameter when the nuisance parameter is location or scale

The problem of estimation of an interest parameter in the presence of a nuisance parameter, which is either location or scale, is studied. Two estimators are considered: the usual maximum likelihood estimator and the estimator based on maximization of the integrated likelihood function. The estimators are compared, asymptotically, with respect to the bias and with respect to the mean squared error. The examples are given.     

Approximations to the Log-Likelihood Function in the Nonlinear Mixed-Effects Model

Abstract

Nonlinear mixed-effects models have received a great deal of attention in the statistical literature in recent years because of the flexibility they offer in handling the unbalanced repeated-measures data that arise in different areas of investigation, such as pharmacokinetics and economics. Several different methods for estimating the parameters in nonlinear mixed-effects model have been proposed. We concentrate here on two of them—maximum likelihood and restricted maximum likelihood. A rather complex numerical issue for (restricted) maximum likelihood estimation in nonlinear mixed-effects models is the evaluation of the log-likelihood function of the data, because it involves the evaluation of a multiple integral that, in most cases, does not have a closed-form expression. We consider here four different approximations to the log-likelihood, comparing their computational and statistical properties. We conclude that the linear mixed-effects (LME) approximation suggested by Lindstrom and Bates, the Laplacian approximation, and Gaussian quadrature centered at the conditional modes of the random effects are quite accurate and computationally efficient. Gaussian quadrature centered at the expected value of the random effects is quite inaccurate for a smaller number of abscissas and computationally inefficient for a larger number of abscissas. Importance sampling is accurate, but quite inefficient computationally.     

Parameter Estimation in Hidden Markov Models With Intractable Likelihoods Using Sequential Monte Carlo

We propose sequential Monte Carlo-based algorithms for maximum likelihood estimation of the static parameters in hidden Markov models with an intractable likelihood using ideas from approximate Bayesian computation. The static parameter estimation algorithms are gradient-based and cover both offline and online estimation. We demonstrate their performance by estimating the parameters of three intractable models, namely the α-stable distribution, g-and-k distribution, and the stochastic volatility model with α-stable returns, using both real and synthetic data.     

Monotonicity of quadratic-approximation algorithms

It is desirable that a numerical maximization algorithm monotonically increase its objective function for the sake of its stability of convergence. It is here shown how one can adjust the Newton-Raphson procedure to attain monotonicity by the use of simple bounds on the curvature of the objective function. The fundamental tool in the analysis is the geometric insight one gains by interpreting quadratic-approximation algorithms as a form of area approximation. The statistical examples discussed include maximum likelihood estimation in mixture models, logistic regression and Cox's proportional hazards regression.The second author's research was partially supported by the National Science Foundation under Grant DMS-8402735.     

Skew normal measurement error models

In this paper we define a class of skew normal measurement error models, extending usual symmetric normal models in order to avoid data transformation. The likelihood function of the observed data is obtained, which can be maximized by using existing statistical software. Inference on the parameters of interest can be approached by using the observed information matrix, which can also be computed by using existing statistical software, such as the Ox program. Bayesian inference is also discussed for the family of asymmetric models in terms of invariance with respect to the symmetric normal distribution showing that early results obtained for the normal distribution also holds for the asymmetric family. Results of a simulation study and an analysis of a real data set analysis are provided.     

Estimation of Smooth Time-Varying Parameters in State Space Models

In this article, we propose a penalized likelihood method to estimate time-varying parameters in standard linear state space models. The time-varying parameter is modeled as a smoothing spline and then expressed as a state space model. The maximum likelihood method is used to estimate the smoothing parameter. The proposed method is assessed by a simulation study and applied to virological response data from an HIV-infected patient receiving antiretroviral treatment.     

A statistical approach to graduation by mathematical formula

Graduation by mathematical formula is recast as problem of statistical estimation. The method of maximum likelihood is used to determine the estimates of the parameters. Theory is developed to allow for estimation without resorting to the usual ‘exposure’ formulas. Both single and multiple decrement models are considered. Theoretical results are obtained for some specific mortality models. Numerical procedures to obtain the estimates are considered.     

非线性回归模型中的约束拟似然

在非线性回归模型中,拟得分函数是一类线性无偏估计函数中的最优者(GodambeandHeyde(1987),朱仲义(1996)),而由拟得分函数得到的拟似然估计在由线性无偏估计函数得到的估计类中具有渐近最优性(林路(1999)).本文则研究非线性回归模型中的有偏估计函数理论,构造了参数的约束拟似然估计,得到了约束拟似然的局部最优性,局部改进了拟似然估计,从而扩充了线性模型中的有偏估计理论.     

Model-Based Recursive Partitioning

Recursive partitioning is embedded into the general and well-established class of parametric models that can be fitted using M-type estimators (including maximum likelihood). An algorithm for model-based recursive partitioning is suggested for which the basic steps are: (1) fit a parametric model to a dataset; (2) test for parameter instability over a set of partitioning variables; (3) if there is some overall parameter instability, split the model with respect to the variable associated with the highest instability; (4) repeat the procedure in each of the daughter nodes. The algorithm yields a partitioned (or segmented) parametric model that can be effectively visualized and that subject-matter scientists are used to analyzing and interpreting.     

Maximum likelihood least squares identification for systems with autoregressive moving average noise

Maximum likelihood methods are important for system modeling and parameter estimation. This paper derives a recursive maximum likelihood least squares identification algorithm for systems with autoregressive moving average noises, based on the maximum likelihood principle. In this derivation, we prove that the maximum of the likelihood function is equivalent to minimizing the least squares cost function. The proposed algorithm is different from the corresponding generalized extended least squares algorithm. The simulation test shows that the proposed algorithm has a higher estimation accuracy than the recursive generalized extended least squares algorithm.     

On a Small Sample Adjustment for the Profile Score Function in Semiparametric Smoothing Models

We consider the profile score function in models with smooth and parametric components. If local respectively weighted likelihood estimation is used for fitting the smooth component, the resulting profile likelihood estimate for the parametric component is asymptotically efficient as shown in T. A. Severini and W. H. Wong (1992, Ann. Statist.20, 1768–1802). However, as in solely parametric models the profile score function is not unbiased. We propose a small sample bias adjustment which results by extending the correction suggested in P. McCullagh and R. Tibshirani (1990, J. Roy. Statist. Soc. Ser. B52, 325–344) to the framework of semiparametric models.     

最大似然方法和Bayes方法在结构方程模型分析中的讨论(英文)

本文通过模拟研究,讨论了最大似然方法和Bayes方法在分析结构方程模型中的相似点和不同之处。     

Second- and third-order bias reduction for one-parameter family models

In this paper we derive second- and third-order bias-corrected maximum likelihood estimates in general uniparametric models. We compare the corrected estimates and the usual maximum likelihood estimate in terms of their mean squared errors. We also obtain closed-form expressions for bias-corrected estimates in one-parameter exponential family models. Our results cover many important and commonly used distributions. Simulation results are also given.     

Estimation of parameters in the beta binomial model

This paper contains some alternative methods for estimating the parameters in the beta binomial and truncated beta binomial models. These methods are compared with maximum likelihood on the basis of Asymptotic Relative Efficiency (ARE). For the beta binomial distribution a simple estimator based on moments or ratios of factorial moments has high ARE for most of the parameter space and it is an attractive and viable alternative to computing the maximum likelihood estimator. It is also simpler to compute than an estimator based on the mean and zeros, proposed by Chatfield and Goodhart (1970,Appl. Statist.,19, 240–250), and has much higher ARE for most part of the parameter space. For the truncated beta binomial, the simple estimator based on two moment relations does not behave quite as well as for the BB distribution, but a simple estimator based on two linear relations involving the first three moments and the frequency of ones has extremely high ARE. Some examples are provided to illustrate the procedure for the two models.     

Logspline Density Estimation for Censored Data

Abstract

Logspline density estimation is developed for data that may be right censored, left censored, or interval censored. A fully automatic method, which involves the maximum likelihood method and may involve stepwise knot deletion and either the Akaike information criterion (AIC) or Bayesian information criterion (BIC), is used to determine the estimate. In solving the maximum likelihood equations, the Newton–Raphson method is augmented by occasional searches in the direction of steepest ascent. Also, a user interface based on S is described for obtaining estimates of the density function, distribution function, and quantile function and for generating a random sample from the fitted distribution.     

Fast Implementation for Normal Mixed Effects Models With Censored Response

We propose an EM algorithm for computing the maximum likelihood and restricted maximum likelihood for linear and nonlinear mixed effects models with censored response. In contrast with previous developments, this algorithm uses closed-form expressions at the E-step, as opposed to Monte Carlo simulation. These expressions rely on formulas for the mean and variance of a truncated multinormal distribution, and can be computed using available software. This leads to an improvement in the speed of computation of up to an order of magnitude. A wide class of mixed effects models is considered, including the Laird–Ware model, and extensions to different structures for the variance components, heteroscedastic and autocorrelated errors, and multilevel models. We apply the methodology to two case studies from our own biostatistical practice, involving the analysis of longitudinal HIV viral load in two recent AIDS studies.

The proposed algorithm is implemented in the R package lmec. An appendix which includes further mathematical details, the R code, and datasets for examples and simulations are available as the online supplements.     

Ergodicity of observation-driven time series models and consistency of the maximum likelihood estimator

This paper deals with a general class of observation-driven time series models with a special focus on time series of counts. We provide conditions under which there exist strict-sense stationary and ergodic versions of such processes. The consistency of the maximum likelihood estimators is then derived for well-specified and misspecified models.     

Comparing the Likelihood Functions of Phylogenetic Trees

DNA sequence data provide a good source of information on the evolutionary history of organisms. Among the proposed methods, the maximum likelihood methods require an explicit probabilistic model of nucleotide substitution that makes the assumption clear. However, procedures for testing hypotheses on topologies have not been well developed. We propose a revised version of the maximum likelihood estimator of a tree and derive some of its properties. Then we present tests to compare given trees and to derive the most likely candidates for the true topology, applying to maximum likelihoods the notion of contrast, as defined in the framework of the analysis of variance, and the procedures used in multiple comparison. Finally, an example is presented.     

Estimators of the regression parameters of the zeta distribution

The zeta distribution with regression parameters has been rarely used in statistics because of the difficulty of estimating the parameters by traditional maximum likelihood. We propose an alternative method for estimating the parameters based on an iteratively reweighted least-squares algorithm. The quadratic distance estimator (QDE) obtained is consistent, asymptotically unbiased and normally distributed; the estimate can also serve as the initial value required by an algorithm to maximize the likelihood function. We illustrate the method with a numerical example from the insurance literature; we compare the values of the estimates obtained by the quadratic distance and maximum likelihood methods and their approximate variance–covariance matrix. Finally, we calculate the bias, variance and the asymptotic efficiency of the QDE compared to the maximum likelihood estimator (MLE) for some values of the parameters.