Estimation of the entropy of a multivariate normal distribution

Motivated by problems in molecular biosciences wherein the evaluation of entropy of a molecular system is important for understanding its thermodynamic properties, we consider the efficient estimation of entropy of a multivariate normal distribution having unknown mean vector and covariance matrix. Based on a random sample, we discuss the problem of estimating the entropy under the quadratic loss function. The best affine equivariant estimator is obtained and, interestingly, it also turns out to be an unbiased estimator and a generalized Bayes estimator. It is established that the best affine equivariant estimator is admissible in the class of estimators that depend on the determinant of the sample covariance matrix alone. The risk improvements of the best affine equivariant estimator over the maximum likelihood estimator (an estimator commonly used in molecular sciences) are obtained numerically and are found to be substantial in higher dimensions, which is commonly the case for atomic coordinates in macromolecules such as proteins. We further establish that even the best affine equivariant estimator is inadmissible and obtain Stein-type and Brewster–Zidek-type estimators dominating it. The Brewster–Zidek-type estimator is shown to be generalized Bayes.     

NM-QELE for ARMA-GARCH models with non-Gaussian innovations

Although the quasi maximum likelihood estimator based on Gaussian density (Gaussian-QMLE) is widely used to estimate parameters in ARMA models with GARCH innovations (ARMA-GARCH models), it does not perform successfully when error distribution of ARMA-GARCH models is either skewed or leptokurtic. In order to circumvent such defects, Lee and Lee (submitted for publication) proposed the quasi maximum estimated-likelihood estimator using Gaussian mixture-based likelihood (NM-QELE) for GARCH models. In this paper, we adopt the NM-QELE method for estimating parameters in ARMA-GARCH models and demonstrate the validity of NM-QELE by verifying its consistency.     

Nonparametric estimation of distributions with categorical and continuous data

In this paper we consider the problem of estimating an unknown joint distribution which is defined over mixed discrete and continuous variables. A nonparametric kernel approach is proposed with smoothing parameters obtained from the cross-validated minimization of the estimator's integrated squared error. We derive the rate of convergence of the cross-validated smoothing parameters to their ‘benchmark’ optimal values, and we also establish the asymptotic normality of the resulting nonparametric kernel density estimator. Monte Carlo simulations illustrate that the proposed estimator performs substantially better than the conventional nonparametric frequency estimator in a range of settings. The simulations also demonstrate that the proposed approach does not suffer from known limitations of the likelihood cross-validation method which breaks down with commonly used kernels when the continuous variables are drawn from fat-tailed distributions. An empirical application demonstrates that the proposed method can yield superior predictions relative to commonly used parametric models.     

Empirical likelihood estimation of discretely sampled processes of OU type

This paper presents an empirical likelihood estimation procedure for parameters of the discretely sampled process of Ornstein-Uhlenbeck type. The proposed procedure is based on the condi- tional characteristic function, and the maximum empirical likelihood estimator is proved to be consistent and asymptotically normal. Moreover, this estimator is shown to be asymptotically efficient under some mild conditions. When the background driving Lévy process is of type A or B, we show that the intensity parameter c...     

指数分布抽样基本定理及在三参数一般指数分布参数估计中的应用

讨论三参数一般指数分布的参数估计,首先讨论了三参数一般指数分布参数的最大似然估计的求解问题,当其中参数α=1时,应用指数分布抽样基本定理,得到了三参数一般指数分布其它参数的一致最小方差无偏估计;并且由此给出求解三参数一般指数分布参数最大似然估计的迭代方法,得到了三参数一般指数分布参数最大似然估计的近似值,给出了模拟结果以说明迭代方法的收敛性;并以相关文献的观察数据作为样本,得到了三参数一般指数分布的参数估计,从而说明了迭代方法的有效性.     

Estimation of failure probability using semi-definite logit model

We will propose a new and practical method for estimating the failure probability of a large number of small to medium scale companies using their balance sheet data. We will use the maximum likelihood method to estimate the best parameters of the logit function, where the failure intensity function in its exponent is represented as a convex quadratic function instead of a commonly used linear function. The reasons for using this type of function are : (i) it can better represent the observed nonlinear dependence of failure probability on financial attributes, (ii) the resulting likelihood function can be maximized using a cutting plane algorithm developed for nonlinear semi-definite programming problems.We will show that we can achieve better prediction performance than the standard logit model, using thousands of sample companies.Revised: December 2002,     

Direct Likelihood Evaluation for the Renewal Hawkes Process

An interesting extension of the widely applied Hawkes self-exiting point process, the renewal Hawkes (RHawkes) process, was recently proposed by Wheatley, Filimonov, and Sornette, which has the potential to significantly widen the application domains of the self-exciting point processes. However, they claimed that computation of the likelihood of the RHawkes process requires exponential time and therefore is practically impossible. They proposed two expectation–maximization (EM) type algorithms to compute the maximum likelihood estimator (MLE) of the model parameters. Because of the fundamental role of likelihood in statistical inference, a practically feasible method for likelihood evaluation is highly desirable. In this article, we provide an algorithm that evaluates the likelihood of the RHawkes process in quadratic time, a drastic improvement from the exponential time claimed by Wheatley, Filimonov, and Sornette. We demonstrate the superior performance of the resulting MLEs of the model relative to the EM estimators through simulations. We also present a computationally efficient procedure to calculate the Rosenblatt residuals of the process for goodness-of-fit assessment, and a simple yet efficient procedure for future event prediction. The proposed methodologies were applied on real data from seismology and finance. An R package implementing the proposed methodologies is included in the supplementary materials.     

An optimal modification of the LIML estimation for many instruments and persistent heteroscedasticity

We consider the estimation of coefficients of a structural equation with many instrumental variables in a simultaneous equation system. It is mathematically equivalent to the estimating equations estimation or a reduced rank regression in the statistical multivariate linear models when the number of restrictions or the dimension of estimating equations increases with the sample size. As a semi-parametric method, we propose a class of modifications of the limited information maximum likelihood (LIML) estimator to improve its asymptotic properties as well as the small sample properties for many instruments and persistent heteroscedasticity. We show that an asymptotically optimal modification of the LIML estimator, which is called AOM-LIML, improves the LIML estimator and other estimation methods. We give a set of sufficient conditions for an asymptotic optimality when the number of instruments or the dimension of the estimating equations is large with persistent heteroscedasticity including a case of many weak instruments.     

One-step estimation of spatial dependence parameters: Properties and extensions of the APLE statistic

We consider one-step estimation of parameters that represent the strength of spatial dependence in a geostatistical or lattice spatial model. While the maximum likelihood estimators (MLE) of spatial dependence parameters are known to have various desirable properties, they do not have closed-form expressions. Therefore, we consider a one-step alternative to maximum likelihood estimation based on solving an approximate (i.e., one-step) profile likelihood estimating equation. The resulting approximate profile likelihood estimator (APLE) has a closed-form representation, making it a suitable alternative to the widely used Moran’s I statistic. Since the finite-sample and asymptotic properties of one-step estimators of covariance-function parameters have not been studied rigorously, we explore these properties for the APLE of the spatial dependence parameter in the simultaneous autoregressive (SAR) model. Motivated by the APLE statistic’s closed from, we develop exploratory spatial data analysis tools that capture regions of local clustering or the extent to which the strength of spatial dependence varies across space. We illustrate these exploratory tools using both simulated data and observed crime rates in Columbus, OH.     

Efficiency of profile likelihood in semi-parametric models

Profile likelihood is a popular method of estimation in the presence of an infinite-dimensional nuisance parameter, as the method reduces the infinite-dimensional estimation problem to a finite-dimensional one. In this paper we investigate the efficiency of a semi-parametric maximum likelihood estimator based on the profile likelihood. By introducing a new parametrization, we improve on the seminal work of Murphy and van der Vaart (J Am Stat Assoc, 95: 449–485, 2000): our improvement establishes the efficiency of the estimator through the direct quadratic expansion of the profile likelihood, which requires fewer assumptions. To illustrate the method an application to two-phase outcome-dependent sampling design is given.     

Component risk in multiparameter estimation

Summary For estimating the mean of ap-variate normal distribution under a quadratic loss, a class of estimators, known as Stein's estimators, is known to dominate the maximum likelihood estimator (MLE) forp≧3. But, whereas the risk of the MLE has the same value, equal to a constant, for each component, the maximum component risk of Stein's estimator is large for large values ofp. Certain modification of Stein's rule has been proposed in the literature for reducing the maximum component risk. In this paper, a new rule is given for reducing the maximum component risk. The new rule yields larger reduction in the maximum component risk, compared to its competitor.     

Estimation of parameters in latent class models using fuzzy clustering algorithms

A mixture approach to clustering is an important technique in cluster analysis. A mixture of multivariate multinomial distributions is usually used to analyze categorical data with latent class model. The parameter estimation is an important step for a mixture distribution. Described here are four approaches to estimating the parameters of a mixture of multivariate multinomial distributions. The first approach is an extended maximum likelihood (ML) method. The second approach is based on the well-known expectation maximization (EM) algorithm. The third approach is the classification maximum likelihood (CML) algorithm. In this paper, we propose a new approach using the so-called fuzzy class model and then create the fuzzy classification maximum likelihood (FCML) approach for categorical data. The accuracy, robustness and effectiveness of these four types of algorithms for estimating the parameters of multivariate binomial mixtures are compared using real empirical data and samples drawn from the multivariate binomial mixtures of two classes. The results show that the proposed FCML algorithm presents better accuracy, robustness and effectiveness. Overall, the FCML algorithm has the superiority over the ML, EM and CML algorithms. Thus, we recommend FCML as another good tool for estimating the parameters of mixture multivariate multinomial models.     

On the maximum likelihood estimator for the Generalized Extreme-Value distribution

The vanilla method in univariate extreme-value theory consists of fitting the three-parameter Generalized Extreme-Value (GEV) distribution to a sample of block maxima. Despite claims to the contrary, the asymptotic normality of the maximum likelihood estimator has never been established. In this paper, a formal proof is given using a general result on the maximum likelihood estimator for parametric families that are differentiable in quadratic mean but whose supports depend on the parameter. An interesting side result concerns the (lack of) differentiability in quadratic mean of the GEV family.     

Penalized empirical likelihood estimation of semiparametric models

We propose an empirical likelihood-based estimation method for conditional estimating equations containing unknown functions, which can be applied for various semiparametric models. The proposed method is based on the methods of conditional empirical likelihood and penalization. Thus, our estimator is called the penalized empirical likelihood (PEL) estimator. For the whole parameter including infinite-dimensional unknown functions, we derive the consistency and a convergence rate of the PEL estimator. Furthermore, for the finite-dimensional parametric component, we show the asymptotic normality and efficiency of the PEL estimator. We illustrate the theory by three examples. Simulation results show reasonable finite sample properties of our estimator.     

Generalizing the Pareto to the log-Pareto model and statistical inference

In this article we introduce a full-fledged statistical model of log-Pareto distribution functions (dfs) parametrized by two shape parameters and a scale parameter. Pareto dfs can be regained in the limit by varying parameters of log-Pareto dfs, whence the log-Pareto model can be regarded as an extension of the Pareto model. Log-Pareto dfs are first of all obtained by means of exponential transformations of Pareto dfs. We also indicate an iterated application of such a procedure. A class of generalized log-Pareto dfs is considered as well. In addition, power-pot (p-pot) stable dfs – related to p-max stable dfs – are introduced and log-Pareto dfs are identified as special cases. A modification of a quick (systematic) estimator is proposed as an initial estimator for the numerical computation of the maximum likelihood estimator (MLE) in the 3-parameter model.      

Inference for multivariate normal mixtures

Multivariate normal mixtures provide a flexible model for high-dimensional data. They are widely used in statistical genetics, statistical finance, and other disciplines. Due to the unboundedness of the likelihood function, classical likelihood-based methods, which may have nice practical properties, are inconsistent. In this paper, we recommend a penalized likelihood method for estimating the mixing distribution. We show that the maximum penalized likelihood estimator is strongly consistent when the number of components has a known upper bound. We also explore a convenient EM-algorithm for computing the maximum penalized likelihood estimator. Extensive simulations are conducted to explore the effectiveness and the practical limitations of both the new method and the ratified maximum likelihood estimators. Guidelines are provided based on the simulation results.     

Statistical inference for coherent systems with Weibull distributed component lifetimes under complete and incomplete information

Point estimators for the parameters of the component lifetime distribution in coherent systems are evolved assuming to be independently and identically Weibull distributed component lifetimes. We study both complete and incomplete information under continuous monitoring of the essential component lifetimes. First, we prove that the maximum likelihood estimator (MLE) under complete information based on progressively Type‐II censored system lifetimes uniquely exists and we present two approaches to compute the estimates. Furthermore, we consider an ad hoc estimator, a max‐probability plan estimator and the MLE for the parameters under incomplete information. In order to compute the MLEs, we consider a direct maximization of the likelihood and an EM‐algorithm–type approach, respectively. In all cases, we illustrate the results by simulations of the five‐component bridge system and the 10‐component parallel system, respectively.     

On maximum likelihood estimation of the drift matrix of a degenerated O–U process

In this work, we consider a 2n-dimension Ornstein–Uhlenbeck (O–U) process with a singular diffusion matrix. This process represents a currently used model for mechanical systems subject to random vibrations. We study the problem of estimating the drift parameters of the stochastic differential equation that governs the O–U process. The maximum likelihood estimator proposed and explored in Koncz (J Anal Math 13(1):75–91, 1987) is revisited and applied to our model. We prove the local asymptotic normality property and the convergence of moments of the estimator. Simulation studies based on representative examples taken from the literature illustrate the obtained theoretical results.     

Properties of estimators of baseline hazard functions in a semiparametric cure model

We consider a semiparametric cure model combining the Cox model with the logistic model. There are the two distinct methods for estimating the nonparametric baseline hazard function of the model; one is based on a pseudo partial likelihood and the other is to use an EM algorithm. In this paper, we discuss the consistency and the asymptotic normality of the estimators from the two methods. Then, we show that the estimator from the pseudo partial likelihood can be characterized by the (forward) Volterra integral equation, and the estimator from the EM algorithm by the Fredholm integral equation. These characterizations reveal differences in the properties between the estimators from the two methods. In addition, a simulation study is performed to numerically confirm the results in several finite samples.     

几何分布截尾样本场合步进应力加速寿命试验TFR模型下的统计分析

讨论了几何分布产品在步进应力加速试验TFR模型下寿命分布.给出了其寿命分布函数步进形式,在截尾样本场合利用极大似然估计方法和拟矩估计方法求出了未知参数的点估计,最后利用计算机模拟考察了说明本文方法的可行性.