An asymptotic expansion for the distribution of asymptotic maximum likelihood estimators of vector parameters

It is shown that the probability that a suitably standardized asymptotic maximum likelihood estimator of a vector parameter (i.e., an estimator which approximates the solution of the likelihood equation in a reasonably good way) lies in a measurable convex set can be approximated by an integral involving a multidimensional normal density function and a series in $\text{n}{}^{\mathrm{<mtext>?1</mtext><mtext>2</mtext>}}$ with certain polynomials as coefficients.     

A matrix formula for the skewness of maximum likelihood estimators

We give a general matrix formula for computing the second-order skewness of maximum likelihood estimators. The formula was firstly presented in a tensorial version by Bowman and Shenton (1998). Our matrix formulation has numerical advantages, since it requires only simple operations on matrices and vectors. We apply the second-order skewness formula to a normal model with a generalized parametrization and to an ARMA model.     

Higher order asymptotic theory for normalizing transformations of maximum likelihood estimators

Suppose thatX _n =(X ₁,...X _n) is a collection ofm-dimensional random vectorsX _i forming a stochastic process with a parameter . Let be the MLE of . We assume that a transformationA( ) of has thek-thorder Edgeworth expansion (k=2,3). IfA extinguishes the terms in the Edgeworth expansion up tok-th-order (k2), then we say thatA is thek-th-order normalizing transformation. In this paper, we elucidate thek-th-order asymptotics of the normalizing transformations. Some conditions forA to be thek-th-order normalizing transformation will be given. Our results are very general, and can be applied to the i.i.d. case, multivariate analysis and time series analysis. Finally, we also study thek-th-order asymptotics of a modified signed log likelihood ratio in terms of the Edgeworth approximation.Research supported by the Office of Naval Research Contract N00014-91-J-1020.     

Third order asymptotic properties of maximum likelihood estimators for Gaussian ARMA processes

In this paper we investigate various third-order asymptotic properties of maximum likelihood estimators for Gaussian ARMA processes by the third-order Edgeworth expansions of the sampling distributions. We define a third-order asymptotic efficiency by the highest probability concentration around the true value with respect to the third-order Edgeworth expansion. Then we show that the maximum likelihood estimator is not always third-order asymptotically efficient in the class A₃ of third-order asymptotically median unbiased estimators. But, if we confine our discussions to an appropriate class D (⊂ A₃) of estimators, we can show that appropriately modified maximum likelihood estimator is always third-order asymptotically efficient in D.     

Convergence results for maximum likelihood type estimators in multivariable ARMA models

General convergence results for maximum likelihood type estimators in multivariable ARMA-models under very weak assumptions are given. This extends results by Dunsmuir and Hannan (1976, Advan. Appl. Probab. 8 339–364) and Deistler, Dunsmuir, and Hannan (1978, Advan. Appl. Probab. 10 360–372). In particular it is shown that consistency can be achieved without imposing a certain assumption used in Dunsmuir and Hannan which is related to the zeroes of the spectral density if one is willing to make stronger assumptions concerning the probabilistic structure of the process.     

Convergence results for maximum likelihood type estimators in multivariable ARMA models II

The consistency proof for the (Gaussian quasi) maximum likelihood estimator in multivariable ARMA models as given in Dunsmuir and Hannan (1976, Adv, in Appl. Probab. 8, 339–364) rests on a certain property of the underlying parameter space, called B6 in their paper. It is not known whether the usual parameter spaces like the manifold M(n) or the parameter spaces corresponding to echelon forms satisfy condition B6, since the argument given by Dunsmuir and Hannan to establish this fact is inconclusive. In Pötscher (1987, J. Multivariate Anal. 21 29–52) it was shown how consistency can be proved without relying on B6 if the data generating process is Gaussian. In this note we show that the Gaussianity assumption can be replaced by ergodicity thus restoring Dunsmuir and Hannan's consistency proof to its full generality and extending it to parameter spaces which do not satisfy condition B6.     

Sharp rates of convergence of maximum likelihood estimators in nonparametric models

Summary Sharp rates of convergence of maximum likelihood estimators are established in models which are defined by probability densities having bounded derivatives. This result is achieved by making use of local properties of the empirical distribution function.     

Asymptotic properties of particle filter-based maximum likelihood estimators for state space models

We study the asymptotic performance of approximate maximum likelihood estimators for state space models obtained via sequential Monte Carlo methods. The state space of the latent Markov chain and the parameter space are assumed to be compact. The approximate estimates are computed by, firstly, running possibly dependent particle filters on a fixed grid in the parameter space, yielding a pointwise approximation of the log-likelihood function. Secondly, extensions of this approximation to the whole parameter space are formed by means of piecewise constant functions or B-spline interpolation, and approximate maximum likelihood estimates are obtained through maximization of the resulting functions. In this setting we formulate criteria for how to increase the number of particles and the resolution of the grid in order to produce estimates that are consistent and asymptotically normal.     

Donsker-type theorems for nonparametric maximum likelihood estimators

Let ${\mathcal{P}}$ be a nonparametric probability model consisting of smooth probability densities and let ${\hat{p}_{n}}$ be the corresponding maximum likelihood estimator based on n independent observations each distributed according to the law ${\mathbb{P}}$ . With $\hat{\mathbb{P}}_{n}$ denoting the measure induced by the density ${\hat{p}_{n}}$ , define the stochastic process ${\hat{\nu}}_{n}: f\longmapsto \sqrt{n} \int fd({\hat{\mathbb{P}}}_{n} -\mathbb{P})$ where f ranges over some function class ${\mathcal{F}}$ . We give a general condition for Donsker classes ${\mathcal{F}}$ implying that the stochastic process $\hat{\nu}_{n}$ is asymptotically equivalent to the empirical process in the space ${\ell ^{\infty }(\mathcal{F})}$ of bounded functions on ${ \mathcal{F}}$ . This implies in particular that $\hat{\nu}_{n}$ converges in law in ${\ell ^{\infty }(\mathcal{F})}$ to a mean zero Gaussian process. We verify the general condition for a large family of Donsker classes ${\mathcal{ F}}$ . We give a number of applications: convergence of the probability measure ${\hat{\mathbb{P}}_{n}}$ to ${\mathbb{P}}$ at rate ${\sqrt{n}}$ in certain metrics metrizing the topology of weak(-star) convergence; a unified treatment of convergence rates of the MLE in a continuous scale of Sobolev-norms; ${\sqrt{n}}$ -efficient estimation of nonlinear functionals defined on ${\mathcal{P}}$ ; limit theorems at rate ${\sqrt{n}}$ for the maximum likelihood estimator of the convolution product ${\mathbb{P\ast P}}$ .     

Improving on the maximum likelihood estimators of the means in Poisson decomposable graphical models

In this article we study the simultaneous estimation of the means in Poisson decomposable graphical models. We derive some classes of estimators which improve on the maximum likelihood estimator under the normalized squared losses. Our estimators are based on the argument in Chou [Simultaneous estimation in discrete multivariate exponential families, Ann. Statist. 19 (1991) 314-328.] and shrink the maximum likelihood estimator depending on the marginal frequencies of variables forming a complete subgraph of the conditional independence graph.     

Finding maximum likelihood estimators for the three-parameter Weibull distribution

Much work has been devoted to the problem of finding maximum likelihood estimators for the three-parameter Weibull distribution. This problem has not been clearly recognized as a global optimization one and most methods from the literature occasionally fail to find a global optimum. We develop a global optimization algorithm which uses first order conditions and projection to reduce the problem to a univariate optimization one. Bounds on the resulting function and its first order derivative are obtained and used in a branch-and-bound scheme. Computational experience is reported. It is also shown that the solution method we propose can be extended to the case of right censored samples.     

Limit theorems for maximum likelihood estimators in the Curie-Weiss-Potts model

The Curie-Weiss-Potts model, a model in statistical mechanics, is parametrized by the inverse temperature β and the external magnetic field h. This paper studies the asymptotic behavior of the maximum likelihood estimator of the parameter β when h = 0 and the asymptotic behavior of the maximum likelihood estimator of the parameter h when β is known and the true value of h is 0. The limits of these maximum likelihood estimators reflect the phase transition in the model; i.e., different limits depending on whether β < β_c, β = β_c or β > β_c, where β_c ε (0, ∞) is the critical inverse temperature of the model.     

On the consistency of conditional maximum likelihood estimators

Let {P _, : , H} be a family of probability measures admitting a sufficient statistic for the nuisance parameter . The paper presents conditions for consistency of (asymptotic) conditional maximum likelihood estimators for . An application to the Rasch-model (a stochastic model for psychological tests) yields a condition on the sequence of nuisance parameters which is sufficient for strong consistency of conditional maximum likelihood estimators, and necessary for the existence of any weakly consistent estimator-sequence.     

The maximum likelihood estimators in a multivariate normal distribution with AR(1) covariance structure for monotone data

The maximum likelihood estimators are uniquely obtained in a multivariate normal distribution with AR(1) covariance structure for monotone data. The maximum likelihood estimator of mean is unbiased.