Empirical Likelihood Confidence Bands in Density Estimation

Abstract

Empirical likelihood methods are developed for constructing confidence bands in problems of nonparametric density estimation. These techniques have an advantage over more conventional methods in that the shape of the bands is determined solely by the data. We show how to construct an empirical likelihood functional, rather than a function, and contour it to produce the confidence bands. Analogs of Wilks's theorem are established in this infinite-parameter setting and may be used to select the appropriate contour. An alternative calibration, based on the bootstrap, is also suggested. Large-sample theory is developed to show that the bands have asymptotically correct coverage, and a numerical example is presented to demonstrate the technique. Comparisons are made with the use of bootstrap replications to choose both the shape and size of the bands.     

基于最大惩罚似然的高斯混合模型无监督分类研究

本文提出了一个基于高斯混合模型的无监督分类算法. 考虑到利用EM算法求解高斯混合模型的参数参数估计问题容易陷入局部最优解, 我们引入逆Wishart分布来代替传统的Jeffery先验. 几个实验数据的结果表明, 采用该方法估计无监督分类的成分数, 无论是估计的正确率, 还是运算速度, 都有较大提高.     

Sharp Error Estimate for Maximum Likelihood Estimator of Nonstationary Diffusion Processes

We consider the maximum likelihood estimator of the unknown parameter in a class of nonstationary diffusion processes. We give further a precise estimate for the error of the estimator.     

An EM-Like Algorithm for Semi- and Nonparametric Estimation in Multivariate Mixtures

We propose an algorithm for nonparametric estimation for finite mixtures of multivariate random vectors that strongly resembles a true EM algorithm. The vectors are assumed to have independent coordinates conditional upon knowing from which mixture component they come, but otherwise their density functions are completely unspecified. Sometimes, the density functions may be partially specified by Euclidean parameters, a case we call semiparametric. Our algorithm is much more flexible and easily applicable than existing algorithms in the literature; it can be extended to any number of mixture components and any number of vector coordinates of the multivariate observations. Thus it may be applied even in situations where the model is not identifiable, so care is called for when using it in situations for which identifiability is difficult to establish conclusively. Our algorithm yields much smaller mean integrated squared errors than an alternative algorithm in a simulation study. In another example using a real dataset, it provides new insights that extend previous analyses. Finally, we present two different variations of our algorithm, one stochastic and one deterministic, and find anecdotal evidence that there is not a great deal of difference between the performance of these two variants. The computer code and data used in this article are available online.     

A Functional EM Algorithm for Mixing Density Estimation via Nonparametric Penalized Likelihood Maximization

When the true mixing density is known to be continuous, the maximum likelihood estimate of the mixing density does not provide a satisfying answer due to its degeneracy. Estimation of mixing densities is a well-known ill-posed indirect problem. In this article, we propose to estimate the mixing density by maximizing a penalized likelihood and call the resulting estimate the nonparametric maximum penalized likelihood estimate (NPMPLE). Using theory and methods from the calculus of variations and differential equations, a new functional EM algorithm is derived for computing the NPMPLE of the mixing density. In the algorithm, maximizers in M-steps are found by solving an ordinary differential equation with boundary conditions numerically. Simulation studies show the algorithm outperforms other existing methods such as the popular EMS algorithm. Some theoretical properties of the NPMPLE and the algorithm are also discussed. Computer code used in this article is available online.     

Maximum likelihood estimation in the multi-path change-point problem

Maximum likelihood estimators of the parameters of the distributions before and after the change and the distribution of the time to change in the multi-path change-point problem are derived and shown to be consistent. The maximization of the likelihood can be carried out by using either the EM algorithm or results from mixture distributions. In fact, these two approaches give equivalent algorithms. Simulations to evaluate the performance of the maximum likelihood estimators under practical conditions, and two examples using data on highway fatalities in the United States, and on the health effects of urea formaldehyde foam insulation, are also provided.This work was supported in part by the Natural Science and Engineering Council of Canada, and the Fonds pour la Formation de chercheurs et l'aide à la Recherche Gouvernment du Québec.Lawrence Joseph is also a member of the Department of Epidemiology and Biostatistics of McGill University.     

用蒙特卡罗方法求取广义线性混合模型之最大似然估计:应用于求取乳癌死亡率之小区域估计

本文使用蒙特卡罗方法, 求得广义线性混合模型之最大似然估计, 并提供用来评估统计参数之收敛和精确度之实用方法\bd 仿真研究显示无偏之固定效应参数估计, 而方差分量估计之误差则相近于前人结果\bd 应用举例为使用泊松分布求取乳癌死亡率之小区域估计.     

Stable Computation of Maximum Likelihood Estimates in Identity Link Poisson Regression

Identity link Poisson regression is useful when the mean of a count variable depends additively on a collection of predictor variables. It is particularly important in epidemiology, for modeling absolute differences in disease incidence rates as a function of covariates. A complication of such models is that standard computational methods for maximum likelihood estimation can be numerically unstable due to the nonnegativity constraints on the Poisson means. Here we present a straightforward and flexible method that provides stable maximization of the likelihood function over the constrained parameter space. This is achieved by conducting a sequence of maximizations within subsets of the parameter space, after which the global maximum is identified from among the subset maxima. The method adapts and extends EM algorithms that are useful in specialized applications involving Poisson deconvolution, but which do not apply in more general regression contexts. As well as allowing categorical and continuous covariates, the method has the flexibility to accommodate covariates with an unspecified isotonic form. Its computational reliability makes it particularly useful in bootstrap analyses, which may require stable convergence for thousands of implementations. Computations are illustrated using epidemiological data on occupational mortality, and biological data on crab population counts. This article has supplementary material online.     

凸约束优化的非单调信赖域算法的收敛性

本文对凸约束优化问题提出一类新的非单调信赖域算法,在二次模型Hesse矩阵{Bk}一致有界条件下,证明了算法具有强收敛性;在{Bk}线性增长的条件下,证明了算法具有弱收敛性;这推广了现有约束或凸约束优化问题的各种信赖域算法,改进了收敛性结果。     

A Cautionary Note on Likelihood Ratio Tests in Mixture Models

We show that iterative methods for maximizing the likelihood in a mixture of exponentials model depend strongly on their particular implementation. Different starting strategies and stopping rules yield completely different estimators of the parameters. This is demonstrated for the likelihood ratio test of homogeneity against two-component exponential mixtures, when the test statistic is calculated by the EM algorithm.     

Bayesian and Maximum Likelihood Estimation for Gaussian Processes on an Incomplete Lattice

This article proposes a new approach for Bayesian and maximum likelihood parameter estimation for stationary Gaussian processes observed on a large lattice with missing values. We propose a Markov chain Monte Carlo approach for Bayesian inference, and a Monte Carlo expectation-maximization algorithm for maximum likelihood inference. Our approach uses data augmentation and circulant embedding of the covariance matrix, and provides likelihood-based inference for the parameters and the missing data. Using simulated data and an application to satellite sea surface temperatures in the Pacific Ocean, we show that our method provides accurate inference on lattices of sizes up to 512 × 512, and is competitive with two popular methods: composite likelihood and spectral approximations.     

比例风险模型下参数估计的两种MM算法的一些应用

对于带有删失机制的生存数据的研究,比例风险模型是应用最为广泛的统计模型之一。实际中,为得到其参数的极大似然估计需要采用数值方法计算得分方程的解。MinorizationMaximization算法(以下简称"MM算法")将求解复杂的目标函数的极值问题转化为求解简单的代理函数的极值问题。本文主要探讨,在比例风险模型下通过两种不同的思想为偏似然函数构造代理函数,从而得到的两种MM算法。通过数值模拟和实际数据分析实现这两种MM算法在比例风险模型下的一些应用。     

Finding the Number of Normal Groups in Model-Based Clustering via Constrained Likelihoods

Deciding the number of clusters k is one of the most difficult problems in cluster analysis. For this purpose, complexity-penalized likelihood approaches have been introduced in model-based clustering, such as the well-known Bayesian information criterion and integrated complete likelihood criteria. However, the classification/mixture likelihoods considered in these approaches are unbounded without any constraint on the cluster scatter matrices. Constraints also prevent traditional EM and CEM algorithms from being trapped in (spurious) local maxima. Controlling the maximal ratio between the eigenvalues of the scatter matrices to be smaller than a fixed constant c ? 1 is a sensible idea for setting such constraints. A new penalized likelihood criterion which takes into account the higher model complexity that a higher value of c entails is proposed. Based on this criterion, a novel and fully automated procedure, leading to a small ranked list of optimal (k, c) couples is provided. A new plot called “car-bike,” which provides a concise summary of the solutions, is introduced. The performance of the procedure is assessed both in empirical examples and through a simulation study as a function of cluster overlap. Supplementary materials for the article are available online.     

线性模型中自相关的经验似然比检验

研究线性回归模型中的自相关检验问题,用经验似然的方法构造检验统计量,得到了零假设下检验统计量的渐近分布,我们的检验方法不但可以检验一阶自相关,也可以检验高阶自相关,数值模拟表明检验方法具有良好的检验功效.     

半序约束下多维正态总体均值和协方差阵的最大似然估计

对给定K个P维正态总体,未知均值和协方差阵分别为θi和Λi,i=1,2,…,k,本文考虑均值和协方差阵之间都在一个简单半序约束θ1≤θ2≤…≤θk,Λ1≥Λ2≥…≥Λk>0条件下的估计问题.讨论θi和Λi的最大似然估计的性质,并给出一个求解的迭代方法.     

正态总体均值与标准差比在简单半序约束下的最大似然估计

本文考虑两个正态总体均值和方差是未知参数,均值与标准差的比在简单半序约束下,且样本个数不同时,计算均值和标准差的最大似然估计的问题.给出了计算均值和标准差的最大似然估计的方法.同时还给出了三个总体均值与标准差的比在简单半序约束下求均值和标准差的最大似然估计的计算方法,并给出了迭代算法的模拟结果.     

Weak convergence of empirical and quantile processes in sup-norm metrics via kmt-constructions

There has been much interest recently in the specially constructed empirical processes of Komlós, Major and Tusnády [2]; as one would guess, much of the application has come from the Hungarian school.In this note we contribute to the unifying effect this profound work has had by showing how the major theorem of O'Reilly [4] follows in rather elementary fashion from this powerful construction. We also take this opportunity to restate O'Reilly's criterion in an elementary form that is far more intelligible.     

结构经济序列分析的状态空间方法

本文讨论结构经济时间序列用状态空间模型进行分解处理的方法．在§１中综述结构时间序列的状态空间描述．§２中着重论述了将处理不完全数据的ＥＭ－算法应用于状态空间模型参数的极大似然估计．在§３中给出采用本文所述方法对一些我国宏观经济序列的计算实例．     

A new smoothing-regularization approach for a maximum-likelihood estimation problem

We consider the problem min _i=1 ^m (aⁱ,x–b_iloga ⁱ, z) subject tox 0 which occurs as a maximum-likelihood estimation problem in several areas, and particularly in positron emission tomography. After noticing that this problem is equivalent to mind(b, Ax) subject tox 0, whered is the Kullback-Leibler information divergence andA, b are the matrix and vector with rows and entriesa ⁱ,b _i, respectively, we suggest a regularized problem mind(b, Ax) + d(v, Sx), where is the regularization parameter,S is a smoothing matrix, andv is a fixed vector. We present a computationally attractive algorithm for the regularized problem, establish its convergence, and show that the regularized solutions, as goes to 0, converge to the solution of the original problem which minimizes a convex function related tod(v, Sx). We give convergence-rate results both for the regularized solutions and for their functional values.The research of A. N. Iusem was partially supported by CNPq Grant No. 301280/86-MA.     

Translated Brownian Motions and Associated Wick Products

Abstract

The concept of Wick product is strongly related to the underlying Brownian motion we have fixed on the probability space. Via the Girsanov's theorem we construct a family of new Brownian motions, obtained as translations of the original one, and to each of them we associate a Wick product. This produces a family of Wick products, named γ-Wick products, parameterized by the performed translations. We aim to describe this family of products. We also define a new family of stochastic integrals, which are related in a natural way to the γ-Wick products.