The breakdown behavior of the maximum likelihood estimator in the logistic regression model

In this note we discuss the breakdown behavior of the maximum likelihood (ML) estimator in the logistic regression model. We formally prove that the ML-estimator never explodes to infinity, but rather breaks down to zero when adding severe outliers to a data set. An example confirms this behavior.     

A third-order optimum property of the maximum likelihood estimator

Let θ⁽ⁿ⁾ denote the maximum likelihood estimator of a vector parameter, based on an i.i.d. sample of size n. The class of estimators θ⁽ⁿ⁾ + n^?1q(θ⁽ⁿ⁾), with q running through a class of sufficiently smooth functions, is essentially complete in the following sense: For any estimator T⁽ⁿ⁾ there exists q such that the risk of θ⁽ⁿ⁾ + n^?1q(θ⁽ⁿ⁾) exceeds the risk of T⁽ⁿ⁾ by an amount of order o(n^?1) at most, simultaneously for all loss functions which are bounded, symmetric, and neg-unimodal. If $\text{q}{}^1$ is chosen such that $\text{θ}{}^{\mathrm{(n)}}\text{+ n}{}^{\mathrm{?1}}\text{q}{}^1\text{(θ}{}^{\mathrm{(n)}}\text{)}$ is unbiased up to $\text{o(n}{}^{\mathrm{<mtext>?1</mtext><mtext>2</mtext>}}\text{)}$, then this estimator minimizes the risk up to an amount of order o(n^?1) in the class of all estimators which are unbiased up to $\text{o(n}{}^{\mathrm{<mtext>?1</mtext><mtext>2</mtext>}}\text{)}$.The results are obtained under the assumption that T⁽ⁿ⁾ admits a stochastic expansion, and that either the distributions have—roughly speaking—densities with respect to the lebesgue measure, or the loss functions are sufficiently smooth.     

On a fundamental optimality of the maximum likelihood estimator

It is well-known that the rate of exponential convergence for any consistent estimator is less than or equal to the Bahadur bound. In this paper we have proven, for the one-dimensional case, that the rate of exponential convergence for the maximum likelihood estimator (m.l.e.) attains the Bahadur bound if and only if the underlying distribution is a member of the exponential family of distributions.     

Empirical likelihood bivariate nonparametric maximum likelihood estimator with right censored data

This article considers the estimation for bivariate distribution function (d.f.) \(F_0(t, z)\) of survival time \(T\) and covariate variable \(Z\) based on bivariate data where \(T\) is subject to right censoring. We derive the empirical likelihood-based bivariate nonparametric maximum likelihood estimator \(\hat{F}_n(t,z)\) for \(F_0(t,z)\) , which has an explicit expression and is unique in the sense of empirical likelihood. Other nice features of \(\hat{F}_n(t,z)\) include that it has only nonnegative probability masses, thus it is monotone in bivariate sense. We show that under \(\hat{F}_n(t,z)\) , the conditional d.f. of \(T\) given \(Z\) is of the same form as the Kaplan–Meier estimator for the univariate case, and that the marginal d.f. \(\hat{F}_n(\infty ,z)\) coincides with the empirical d.f. of the covariate sample. We also show that when there is no censoring, \(\hat{F}_n(t,z)\) coincides with the bivariate empirical d.f. For discrete covariate \(Z\) , the strong consistency and weak convergence of \(\hat{F}_n(t,z)\) are established. Some simulation results are presented.     

Approximate maximum likelihood estimation in linear regression

The application of the ML method in linear regression requires a parametric form for the error density. When this is not available, the density may be parameterized by its cumulants ( _i ) and the ML then applied. Results are obtained when the standardized cumulants ( _i ) satisfy _i = _i+2/ ₂ ^(i+2)/2 =O(v ⁱ ) asv 0 fori>0.Research financed in part by the Research Center of the Athens University of Economics and Business.     

On the maximum likelihood estimator for the Generalized Extreme-Value distribution

The vanilla method in univariate extreme-value theory consists of fitting the three-parameter Generalized Extreme-Value (GEV) distribution to a sample of block maxima. Despite claims to the contrary, the asymptotic normality of the maximum likelihood estimator has never been established. In this paper, a formal proof is given using a general result on the maximum likelihood estimator for parametric families that are differentiable in quadratic mean but whose supports depend on the parameter. An interesting side result concerns the (lack of) differentiability in quadratic mean of the GEV family.     

Exact maximum likelihood estimator of the structure of a stratified population

An exact expression for the extreme values of the integer vector that maximize the function

Asymptotic properties of maximum likelihood estimator for two-step logit models

Two-step logit models are extensions of the ordinary logistic regression model, which are designed for complex ordinal outcomes commonly seen in practice. In this paper, we establish some asymptotic properties of the maximum likelihood estimator (MLE) of the regression parameter vector under some mild conditions, which include existence of the MLE, convergence rate and asymptotic normality of the MLE. We relax the boundedness condition of the regressors required in most existing theoretical results, and all conditions are easy to verify.     

The extent of the maximum likelihood estimator for the extreme value index

In extreme value analysis, staring from Smith (1987) [1], the maximum likelihood procedure is applied in estimating the shape parameter of tails—the extreme value index γ. For its theoretical properties, Zhou (2009) [12] proved that the maximum likelihood estimator eventually exists and is consistent for γ>−1 under the first order condition. The combination of Zhou (2009) [12] and Drees et al (2004) [11] provides the asymptotic normality under the second order condition for γ>−1/2. This paper proves the asymptotic normality for −1<γ≤−1/2 and the non-consistency for γ<−1. These results close the discussion on the theoretical properties of the maximum likelihood estimator.     

A note on the asymptotic distribution of the maximum likelihood estimator in a non-regular case

We consider the asymptotic distribution of the maximum likelihood estimator (MLE), when the log-likelihood ratio statistic weakly converges to the non-degenerated Gaussian process. We provide a simple expression for the density function of the asymptotic distribution by fundamental stochastic results. This note is helpful to investigate asymptotic properties of the MLE in a certain non-regular case.     

Bahadur efficiency of the maximum likelihood estimator and one-step estimator for quasi-arithmetic means of the Cauchy distribution

Annals of the Institute of Statistical Mathematics - Some quasi-arithmetic means of random variables easily give unbiased strongly consistent closed-form estimators of the joint of the location and...     

Shrinkage estimator in normal mean vector estimation based on conditional maximum likelihood estimators

Estimation of normal mean vector has broad applications such as small area estimation, estimation of nonparametric functions and estimation of wavelet coefficients. In this paper, we propose a new shrinkage estimator based on conditional maximum likelihood estimator incorporating with Stein’s risk unbiased estimator (SURE) when data have the normality. We present some theoretical work and provide numerical studies to compare with some existing methods.     

Asymptotic properties of conditional maximum likelihood estimator in a certain exponential model

The conditional maximum likelihood estimator is suggested as an alternative to the maximum likelihood estimator and is favorable for an estimator of a dispersion parameter in the normal distribution, the inverse-Gaussian distribution, and so on. However, it is not clear whether the conditional maximum likelihood estimator is asymptotically efficient in general. Consider the case where it is asymptotically efficient and its asymptotic covariance depends only on an objective parameter in an exponential model. This remand implies that the exponential model possesses a certain parallel foliation. In this situation, this paper investigates asymptotic properties of the conditional maximum likelihood estimator and compares the conditional maximum likelihood estimator with the maximum likelihood estimator. We see that the bias of the former is more robust than that of the latter and that two estimators are very close, especially in the sense of bias-corrected version. The mean Pythagorean relation is also discussed.     

Existence and consistency of the maximum likelihood estimator for the extreme value index

The paper is about the asymptotic properties of the maximum likelihood estimator for the extreme value index. Under the second order condition, Drees et al. [H. Drees, A. Ferreira, L. de Haan, On maximum likelihood estimation of the extreme value index, Ann. Appl. Probab. 14 (2004) 1179-1201] proved asymptotic normality for any solution of the likelihood equations (with shape parameter γ>−1/2) that is not too far off the real value. But they did not prove that there is a solution of the equations satisfying the restrictions.In this paper, the existence is proved, even for γ>−1. The proof just uses the domain of attraction condition (first order condition), not the second order condition. It is also proved that the estimator is consistent. When the second order condition is valid, following the current proof, the existence of a solution satisfying the restrictions in the above-cited reference is a direct consequence.