Inequality constrained maximum likelihood estimation

Summary The existence of an estimator constrained to lie in a certain type of bounded set is established for a fairly wide class of probability density functions. The necessary and sufficient conditions thus obtained provide a convenient means of finding such an estimator by mathematical programming methods. This result is a generalization of Cramer’s demonstration of the existence of an unconstrained maximum likelihood estimator and of Aitchison and Silvey’s demonstration of the existence of a maximum likelihood estimator constrained to satisfy certain equations.     

Local polynomial maximum likelihood estimation for Pareto-type distributions

We discuss the estimation of the tail index of a heavy-tailed distribution when covariate information is available. The approach followed here is based on the technique of local polynomial maximum likelihood estimation. The generalized Pareto distribution is fitted locally to exceedances over a high specified threshold. The method provides nonparametric estimates of the parameter functions and their derivatives up to the degree of the chosen polynomial. Consistency and asymptotic normality of the proposed estimators will be proven under suitable regularity conditions. This approach is motivated by the fact that in some applications the threshold should be allowed to change with the covariates due to significant effects on scale and location of the conditional distributions. Using the asymptotic results we are able to derive an expression for the asymptotic mean squared error, which can be used to guide the selection of the bandwidth and the threshold. The applicability of the method will be demonstrated with a few practical examples.     

Approximate maximum likelihood estimation in linear regression

The application of the ML method in linear regression requires a parametric form for the error density. When this is not available, the density may be parameterized by its cumulants ( _i ) and the ML then applied. Results are obtained when the standardized cumulants ( _i ) satisfy _i = _i+2/ ₂ ^(i+2)/2 =O(v ⁱ ) asv 0 fori>0.Research financed in part by the Research Center of the Athens University of Economics and Business.     

A delay-time model for repairable machinery: maximum likelihood estimation of optimum inspection intervals

A delay-time model is presented for a complex repairable systemin which defects arise according to a nonhomogeneous Poissonprocess. The focus of this paper is not on the resulting probabilisticmodel itself, but on the estimation of the model parametersand their errors from records of failure times and number ofdefects found at inspections of a machine which has been operatedfor some time under some (usually suboptimal) inspection regime.The likelihood function of the observed failure and inspectiondata is derived, and a general approach for estimating the optimuminspection interval is discussed. An example relating to themaintenance of medical equipment is used to illustrate the techniquesdescribed in the paper.     

极大似然估计算法研究

将解一元方程的二分法推广至求解多元非线性方程组.以第K个变元Xk为参数,则κ元方程组就可以看作曲线s(前κ-1个方程)和κ-1维曲面C(第κ个方程),于是κ元方程组的解就可以看作寻找曲线s和曲面C的交点.对参数Xk作二分法,重复迭代,直到找到满足误差要求的方程组的解.最后给出了用多元二分法的算法求解极大似然估计的数值解.     

A note on maximum likelihood estimation for the complex-valued first-order autoregressive process

The maximum likelihood estimation of the parameters of a complex-valued zero-mean normal stationary first-order autoregressive process is investigated. It is shown that the likelihood function corresponding to independent replicated series is uniquely maximized at a point in the interior of the parameter space. A closed-form expression is given for the estimator.     

Sieve maximum likelihood estimation for doubly semiparametric zero-inflated Poisson models

For nonnegative measurements such as income or sick days, zero counts often have special status. Furthermore, the incidence of zero counts is often greater than expected for the Poisson model. This article considers a doubly semiparametric zero-inflated Poisson model to fit data of this type, which assumes two partially linear link functions in both the mean of the Poisson component and the probability of zero. We study a sieve maximum likelihood estimator for both the regression parameters and the nonparametric functions. We show, under routine conditions, that the estimators are strongly consistent. Moreover, the parameter estimators are asymptotically normal and first order efficient, while the nonparametric components achieve the optimal convergence rates. Simulation studies suggest that the extra flexibility inherent from the doubly semiparametric model is gained with little loss in statistical efficiency. We also illustrate our approach with a dataset from a public health study.     

Optimal tuning parameter estimation in maximum penalized likelihood method

In maximum penalized or regularized methods, it is important to select a tuning parameter appropriately. This paper proposes a direct plug-in method for tuning parameter selection. The tuning parameters selected using a generalized information criterion (Konishi and Kitagawa, Biometrika, 83, 875–890, 1996) and cross-validation (Stone, Journal of the Royal Statistical Society, Series B, 58, 267–288, 1974) are shown to be asymptotically equivalent to those selected using the proposed method, from the perspective of estimation of an optimal tuning parameter. Because of its directness, the proposed method is superior to the two selection methods mentioned above in terms of computational cost. Some numerical examples which contain the penalized spline generalized linear model regressions are provided.     

On maximum likelihood estimation of a Pareto mixture

In this paper we deal with maximum likelihood estimation (MLE) of the parameters of a Pareto mixture. Standard MLE procedures are difficult to apply in this setup, because the distributions of the observations do not have common support. We study the properties of the estimators under different hypotheses; in particular, we show that, when all the parameters are unknown, the estimators can be found maximizing the profile likelihood function. Then we turn to the computational aspects of the problem, and develop three alternative procedures: an EM-type algorithm, a Simulated Annealing and an algorithm based on Cross-Entropy minimization. The work is motivated by an application in the operational risk measurement field: we fit a Pareto mixture to operational losses recorded by a bank in two different business lines. Under the assumption that each population follows a Pareto distribution, the appropriate model is a mixture of Pareto distributions where all the parameters have to be estimated.     

面板数据公共变点的非参数最大似然估计

Gao Pengli;Xia Zhiming(School of Mathematics,Northwest University,Xi'an 710127,China)     

On normal approximation for maximum likelihood estimation from single server queues

This paper is concerned with the rate of convergence of the distribution of the maximum likelihood estimators of the arrival and the service rates in a GI/G/1 queueing system. This revised version was published online in June 2006 with corrections to the Cover Date.     

A note on Sheppard's corrections for grouping and maximum likelihood estimation

Sheppard's corrections for grouping can, in the case of an underlying normal distribution, be interpreted as a first step to the solution of the maximum likelihood equations which incorporate the grouping problem. This result of Lindley (for the univariate) and Haitovsky (for the bivariate) is generalized to the multivariate normal distribution, making use of recent results in matrix algebra. Also, formulae concerning the efficiency lost in grouping are generalized to the multivariate case.     

An adjusted maximum likelihood method for solving small area estimation problems

For the well-known Fay-Herriot small area model, standard variance component estimation methods frequently produce zero estimates of the strictly positive model variance. As a consequence, an empirical best linear unbiased predictor of a small area mean, commonly used in small area estimation, could reduce to a simple regression estimator, which typically has an overshrinking problem. We propose an adjusted maximum likelihood estimator of the model variance that maximizes an adjusted likelihood defined as a product of the model variance and a standard likelihood (e.g., a profile or residual likelihood) function. The adjustment factor was suggested earlier by Carl Morris in the context of approximating a hierarchical Bayes solution where the hyperparameters, including the model variance, are assumed to follow a prior distribution. Interestingly, the proposed adjustment does not affect the mean squared error property of the model variance estimator or the corresponding empirical best linear unbiased predictors of the small area means in a higher order asymptotic sense. However, as demonstrated in our simulation study, the proposed adjustment has a considerable advantage in small sample inference, especially in estimating the shrinkage parameters and in constructing the parametric bootstrap prediction intervals of the small area means, which require the use of a strictly positive consistent model variance estimate.     

The asymptotic equivalence of Bayes and maximum likelihood estimation

Let (X, $\text{A}$) be a measurable space, Θ ? $\text{R}$ an open interval and P_Ω ∥ $\text{A}$, Ω ? Θ, a family of probability measures fulfilling certain regularity conditions. Let $\text{Ω}{}_{\mathrm{n}}$ be the maximum likelihood estimate for the sample size n. Let λ be a prior distribution on Θ and let $\text{R}{}_{\mathrm{<mtext>n,</mtext><mtext>x</mtext>}}$ be the posterior distribution for the sample size n given $\text{x}\text{? X}{}^{\mathrm{n}}$. $\text{L:}\text{Θ}\text{× Θ →}\text{R}$ denotes a loss function fulfilling certain regularity conditions and T_n denotes the Bayes estimate relative to λ and L for the sample size n. It is proved that for every compact K ? Θ there exists c_K ≥ 0 such that

This theorem improves results of Bickel and Yahav [3], and Ibragimov and Has'minskii [4], as far as the speed of convergence is concerned.     

On fixed-width confidence intervals associated with maximum likelihood estimation

A sequential procedure is proposed to determine the sample size for a fixed-width confidence interval for an unknown parameter with its maximum likelihood estimator as the center of the interval. It is established that the sequential procedure is asymptotically consistent and efficient.     

An efficient ECM algorithm for maximum likelihood estimation in mixtures of t-factor analyzers

Mixture of t factor analyzers (MtFA) have been shown to be a sound model-based tool for robust clustering of high-dimensional data. This approach, which is deemed to be one of natural parametric extensions with respect to normal-theory models, allows for accommodation of potential noise components, atypical observations or data with longer-than-normal tails. In this paper, we propose an efficient expectation conditional maximization (ECM) algorithm for fast maximum likelihood estimation of MtFA. The proposed algorithm inherits all appealing properties of the ordinary EM algorithm such as its stability and monotonicity, but has a faster convergence rate since its CM steps are governed by a much smaller fraction of missing information. Numerical experiments based on simulated and real data show that the new procedure outperforms the commonly used EM and AECM algorithms substantially in most of the situations, regardless of how the convergence speed is assessed by the computing time or number of iterations.     

Likelihood landscape and maximum likelihood estimation for the discrete orbit recovery model

We study the nonconvex optimization landscape for maximum likelihood estimation in the discrete orbit recovery model with Gaussian noise. This is a statistical model motivated by applications in molecular microscopy and image processing, where each measurement of an unknown object is subject to an independent random rotation from a known rotational group. Equivalently, it is a Gaussian mixture model where the mixture centers belong to a group orbit. We show that fundamental properties of the likelihood landscape depend on the signal-to-noise ratio and the group structure. At low noise, this landscape is “benign” for any discrete group, possessing no spurious local optima and only strict saddle points. At high noise, this landscape may develop spurious local optima, depending on the specific group. We discuss several positive and negative examples, and provide a general condition that ensures a globally benign landscape at high noise. For cyclic permutations of coordinates on (multireference alignment), there may be spurious local optima when , and we establish a correspondence between these local optima and those of a surrogate function of the phase variables in the Fourier domain. We show that the Fisher information matrix transitions from resembling that of a single Gaussian distribution in low noise to having a graded eigenvalue structure in high noise, which is determined by the graded algebra of invariant polynomials under the group action. In a local neighborhood of the true object, where the neighborhood size is independent of the signal-to-noise ratio, the landscape is strongly convex in a reparametrized system of variables given by a transcendence basis of this polynomial algebra. We discuss implications for optimization algorithms, including slow convergence of expectation-maximization, and possible advantages of momentum-based acceleration and variable reparametrization for first- and second-order descent methods. © 2021 Wiley Periodicals LLC.     

An approximate maximum likelihood estimation for non-Gaussian non-minimum phase moving average processes

An approximate maximum likelihood procedure is proposed for the estimation of parameters in possibly nonminimum phase (noninvertible) moving average processes driven by independent and identically distributed non-Gaussian noise. Under appropriate conditions, parameter estimates that are solutions of likelihood-like equations are consistent and are asymptotically normal. A simulation study for MA(2) processes illustrates the estimation procedure.     

Nonparametric maximum likelihood density estimation and simulation-based minimum distance estimators

Indirect inference estimators (i.e., simulation-based minimum distance estimators) in a parametric model that are based on auxiliary nonparametric maximum likelihood density estimators are shown to be asymptotically normal. If the parametricmodel is correctly specified, it is furthermore shown that the asymptotic variance-covariance matrix equals the inverse of the Fisher-information matrix. These results are based on uniform-in-parameters convergence rates and a uniform-inparameters Donsker-type theorem for nonparametric maximum likelihood density estimators.