On a fundamental optimality of the maximum likelihood estimator

It is well-known that the rate of exponential convergence for any consistent estimator is less than or equal to the Bahadur bound. In this paper we have proven, for the one-dimensional case, that the rate of exponential convergence for the maximum likelihood estimator (m.l.e.) attains the Bahadur bound if and only if the underlying distribution is a member of the exponential family of distributions.     

Bahadur efficiency of the maximum likelihood estimator and one-step estimator for quasi-arithmetic means of the Cauchy distribution

Annals of the Institute of Statistical Mathematics - Some quasi-arithmetic means of random variables easily give unbiased strongly consistent closed-form estimators of the joint of the location and...     

A third-order optimum property of the maximum likelihood estimator

Let θ⁽ⁿ⁾ denote the maximum likelihood estimator of a vector parameter, based on an i.i.d. sample of size n. The class of estimators θ⁽ⁿ⁾ + n^?1q(θ⁽ⁿ⁾), with q running through a class of sufficiently smooth functions, is essentially complete in the following sense: For any estimator T⁽ⁿ⁾ there exists q such that the risk of θ⁽ⁿ⁾ + n^?1q(θ⁽ⁿ⁾) exceeds the risk of T⁽ⁿ⁾ by an amount of order o(n^?1) at most, simultaneously for all loss functions which are bounded, symmetric, and neg-unimodal. If $\text{q}{}^1$ is chosen such that $\text{θ}{}^{\mathrm{(n)}}\text{+ n}{}^{\mathrm{?1}}\text{q}{}^1\text{(θ}{}^{\mathrm{(n)}}\text{)}$ is unbiased up to $\text{o(n}{}^{\mathrm{<mtext>?1</mtext><mtext>2</mtext>}}\text{)}$, then this estimator minimizes the risk up to an amount of order o(n^?1) in the class of all estimators which are unbiased up to $\text{o(n}{}^{\mathrm{<mtext>?1</mtext><mtext>2</mtext>}}\text{)}$.The results are obtained under the assumption that T⁽ⁿ⁾ admits a stochastic expansion, and that either the distributions have—roughly speaking—densities with respect to the lebesgue measure, or the loss functions are sufficiently smooth.     

On large deviation expansion of distribution of maximum likelihood estimator and its application in large sample estimation

For estimating an unknown parameter , the likelihood principle yields the maximum likelihood estimator. It is often favoured especially by the applied statistician, for its good properties in the large sample case. In this paper, a large deviation expansion for the distribution of the maximum likelihood estimator is obtained. The asymptotic expansion provides a useful tool to approximate the tail probability of the maximum likelihood estimator and to make statistical inference. Theoretical and numerical examples are given. Numerical results show that the large deviation approximation performs much better than the classical normal approximation.This work is supported in part by the Natural Science and Engineering Research Council of Canada under grant NSERC A-9216.This author is also partially supported by the National Science Foundation of China.     

A note on the asymptotic distribution of the maximum likelihood estimator in a non-regular case

We consider the asymptotic distribution of the maximum likelihood estimator (MLE), when the log-likelihood ratio statistic weakly converges to the non-degenerated Gaussian process. We provide a simple expression for the density function of the asymptotic distribution by fundamental stochastic results. This note is helpful to investigate asymptotic properties of the MLE in a certain non-regular case.     

Empirical likelihood bivariate nonparametric maximum likelihood estimator with right censored data

This article considers the estimation for bivariate distribution function (d.f.) \(F_0(t, z)\) of survival time \(T\) and covariate variable \(Z\) based on bivariate data where \(T\) is subject to right censoring. We derive the empirical likelihood-based bivariate nonparametric maximum likelihood estimator \(\hat{F}_n(t,z)\) for \(F_0(t,z)\) , which has an explicit expression and is unique in the sense of empirical likelihood. Other nice features of \(\hat{F}_n(t,z)\) include that it has only nonnegative probability masses, thus it is monotone in bivariate sense. We show that under \(\hat{F}_n(t,z)\) , the conditional d.f. of \(T\) given \(Z\) is of the same form as the Kaplan–Meier estimator for the univariate case, and that the marginal d.f. \(\hat{F}_n(\infty ,z)\) coincides with the empirical d.f. of the covariate sample. We also show that when there is no censoring, \(\hat{F}_n(t,z)\) coincides with the bivariate empirical d.f. For discrete covariate \(Z\) , the strong consistency and weak convergence of \(\hat{F}_n(t,z)\) are established. Some simulation results are presented.     

Asymptotic properties of maximum likelihood estimator for two-step logit models

Two-step logit models are extensions of the ordinary logistic regression model, which are designed for complex ordinal outcomes commonly seen in practice. In this paper, we establish some asymptotic properties of the maximum likelihood estimator (MLE) of the regression parameter vector under some mild conditions, which include existence of the MLE, convergence rate and asymptotic normality of the MLE. We relax the boundedness condition of the regressors required in most existing theoretical results, and all conditions are easy to verify.     

The extent of the maximum likelihood estimator for the extreme value index

In extreme value analysis, staring from Smith (1987) [1], the maximum likelihood procedure is applied in estimating the shape parameter of tails—the extreme value index γ. For its theoretical properties, Zhou (2009) [12] proved that the maximum likelihood estimator eventually exists and is consistent for γ>−1 under the first order condition. The combination of Zhou (2009) [12] and Drees et al (2004) [11] provides the asymptotic normality under the second order condition for γ>−1/2. This paper proves the asymptotic normality for −1<γ≤−1/2 and the non-consistency for γ<−1. These results close the discussion on the theoretical properties of the maximum likelihood estimator.     

Existence and consistency of the maximum likelihood estimator for the extreme value index

The paper is about the asymptotic properties of the maximum likelihood estimator for the extreme value index. Under the second order condition, Drees et al. [H. Drees, A. Ferreira, L. de Haan, On maximum likelihood estimation of the extreme value index, Ann. Appl. Probab. 14 (2004) 1179-1201] proved asymptotic normality for any solution of the likelihood equations (with shape parameter γ>−1/2) that is not too far off the real value. But they did not prove that there is a solution of the equations satisfying the restrictions.In this paper, the existence is proved, even for γ>−1. The proof just uses the domain of attraction condition (first order condition), not the second order condition. It is also proved that the estimator is consistent. When the second order condition is valid, following the current proof, the existence of a solution satisfying the restrictions in the above-cited reference is a direct consequence.     

Geometrical expansions for the distributions of the score vector and the maximum likelihood estimator

In the present note, asymptotic expansions for conditional and unconditional distributions of the score vector are derived. Our aim is to consider these expansions in the light of differential geometry, particularly the theory of derivative strings. Expansions for the distributions of the maximum likelihood estimator are obtained from those for the score vector via transformation, with a view to interpreting from the standpoint of differential geometry the various terms entering the expansions.The present work was carried out at the Department of Theoretical Statistics, University of Aarhus, Denmark, with support from the Danish-French Cultural Exchange Programme.     

Exact maximum likelihood estimator of the structure of a stratified population

An exact expression for the extreme values of the integer vector that maximize the function

Exact maximum likelihood estimator for drift fractional Brownian motion at discrete observation

This paper deals with the problems of consistency and strong consistency of the maximum likelihood estimators of the mean and variance of the drift fractional Brownian motions observed at discrete time instants. Both the central limit theorem and the Berry-Ess′een bounds for these estimators are obtained by using the Stein’s method via Malliavin calculus.