Density estimation by the penalized combinatorial method

Let f be an unknown multivariate density belonging to a prespecified parametric class of densities, , where k is unknown, but for all k and each has finite Vapnik-Chervonenkis dimension. Given an i.i.d. sample of size n drawn from f, we show that it is possible to select automatically, and without extra restrictions on f, an estimate with the property that . Our method is inspired by the combinatorial tools developed in Devroye and Lugosi (Combinatorial Methods in Density Estimation, Springer, New York, 2001) and it includes a wide range of density models, such as mixture models or exponential families.     

Complexity penalized support estimation

We consider the estimation of the support of a probability density function with iid observations. The estimator to be considered is a minimizer of a complexity penalized excess mass criterion. We present a fast algorithm for the construction of the estimator. The estimator is able to estimate supports which consists of disconnected regions. We will prove that the estimator achieves minimax rates of convergence up to a logarithmic factor simultaneously over a scale of Hölder smoothness classes for the boundary of the support. The proof assumes a sharp boundary for the support.     

Consistency of the penalized MLE for two-parameter gamma mixture models

Two-parameter gamma distributions are widely used in liability theory, lifetime data analysis, financial statistics, and other areas. Finite mixtures of gamma distributions are their natural extensions, and they are particularly useful when the population is suspected of heterogeneity. These distributions are successfully employed in various applications, but many researchers falsely believe that the maximum likelihood estimator of the mixing distribution is consistent. Similarly to finite mixtures of normal distributions, the likelihood function under finite gamma mixtures is unbounded. Because of this, each observed value leads to a global maximum that is irrelevant to the true distribution. We apply a seemingly negligible penalty to the likelihood according to the shape parameters in the fitted model. We show that this penalty restores the consistency of the likelihoodbased estimator of the mixing distribution under finite gamma mixture models. We present simulation results to validate the consistency conclusion, and we give an example to illustrate the key points.     

Optimal tuning parameter estimation in maximum penalized likelihood method

In maximum penalized or regularized methods, it is important to select a tuning parameter appropriately. This paper proposes a direct plug-in method for tuning parameter selection. The tuning parameters selected using a generalized information criterion (Konishi and Kitagawa, Biometrika, 83, 875–890, 1996) and cross-validation (Stone, Journal of the Royal Statistical Society, Series B, 58, 267–288, 1974) are shown to be asymptotically equivalent to those selected using the proposed method, from the perspective of estimation of an optimal tuning parameter. Because of its directness, the proposed method is superior to the two selection methods mentioned above in terms of computational cost. Some numerical examples which contain the penalized spline generalized linear model regressions are provided.     

Portfolio selection under model uncertainty: a penalized moment-based optimization approach

We present a new approach that enables investors to seek a reasonably robust policy for portfolio selection in the presence of rare but high-impact realization of moment uncertainty. In practice, portfolio managers face difficulty in seeking a balance between relying on their knowledge of a reference financial model and taking into account possible ambiguity of the model. Based on the concept of Distributionally Robust Optimization (DRO), we introduce a new penalty framework that provides investors flexibility to define prior reference models using the distributional information of the first two moments and accounts for model ambiguity in terms of extreme moment uncertainty. We show that in our approach a globally-optimal portfolio can in general be obtained in a computationally tractable manner. We also show that for a wide range of specifications our proposed model can be recast as semidefinite programs. Computational experiments show that our penalized moment-based approach outperforms classical DRO approaches in terms of both average and downside-risk performance using historical data.     

On maximum likelihood estimation of a Pareto mixture

In this paper we deal with maximum likelihood estimation (MLE) of the parameters of a Pareto mixture. Standard MLE procedures are difficult to apply in this setup, because the distributions of the observations do not have common support. We study the properties of the estimators under different hypotheses; in particular, we show that, when all the parameters are unknown, the estimators can be found maximizing the profile likelihood function. Then we turn to the computational aspects of the problem, and develop three alternative procedures: an EM-type algorithm, a Simulated Annealing and an algorithm based on Cross-Entropy minimization. The work is motivated by an application in the operational risk measurement field: we fit a Pareto mixture to operational losses recorded by a bank in two different business lines. Under the assumption that each population follows a Pareto distribution, the appropriate model is a mixture of Pareto distributions where all the parameters have to be estimated.     

Statistical estimation of a mixture of Gaussian distributions

The paper is devoted to the problem of statistical estimation of a multivariate distribution density, which is a discrete mixture of Gaussian distributions. A heuristic approach is considered, based on the use of the EM algorithm and nonparametric density estimation with a sequential increase in the number of components of the mixture. Criteria for testing of model adequacy are discussed.     

Bayes estimation in a mixture inverse Gaussian model

In this paper a mixture model involving the inverse Gaussian distribution and its length biased version is studied from a Bayesian view-point. Using proper priors, the Bayes estimates of the parameters of the model are derived and the results are applied on the aircraft data of Proschan (1963,Technometrics,5, 375–383). The posterior distributions of the parameters are expressed in terms of the confluent-hypergeometric function and the modified Bessel function of the third kind. The integral involved in the expression of the estimate of the mean is evaluated by numerical techniques.     

A new approach for parameter estimation of finite Weibull mixture distributions for reliability modeling

The aim of this paper is to model lifetime data for systems that have failure modes by using the finite mixture of Weibull distributions. It involves estimating of the unknown parameters which is an important task in statistics, especially in life testing and reliability analysis. The proposed approach depends on different methods that will be used to develop the estimates such as MLE through the EM algorithm. In addition, Bayesian estimations will be investigated and some other extensions such as Graphic, Non-Linear Median Rank Regression and Monte Carlo simulation methods can be used to model the system under consideration. A numerical application will be used through the proposed approach. This paper also presents a comparison of the fitted probability density functions, reliability functions and hazard functions of the 3-parameter Weibull and Weibull mixture distributions using the proposed approach and other conventional methods which characterize the distribution of failure times for the system components. GOF is used to determine the best distribution for modeling lifetime data, the priority will be for the proposed approach which has more accurate parameter estimates.     

Density estimation for a class of stationary nonlinear processes

Let {X _t ;t∈ℤ be a strictly stationary nonlinear process of the formX _t =ε _t +∑ _r=1 ^∞ W _rt , whereW _rt can be written as a functiong _r (ε _t−1,...ε _t-r-q ), {ε _t ;t∈ℤ is a sequence of independent and identically distributed (i.i.d.) random variables withE|ε₁| ^g < ∞ for some γ>0 andq≥0 is fixed integer. Under certain mild regularity conditions ofg _r and {ε _t } we then show thatX ₁ has a density functionf and that the standard kernel type estimator baded on a realization {X ₁,...,X _n } from {X _t } is, asymptotically, normal and converges a.s. tof(x) asn→∞. The research of this author was partially carried out while he was a research scholar, on a sabbatical leave, at the Department of Statistics and Probability, Michigan State University.     

Density estimation for linear processes

Summary LetX _t , ...,X _n be random variables forming a realization from a linear process where {Z _t } is a sequence of independent and identically distributed random variables with E|Z _t |<∞ for some ε>0, andg _r →0 asr→∞ at some specified rate. LetX ₁ have a probability density functionf. It is then established that for every realx, the standard kernel type estimator based onX _t (1≦t≦n) is, under some general regularity conditions, asymptotically normal and converges a.s. tof(x) asn→∞. Research was supported in part by the Air Force Office of Scientific Research Grant No. AFOSR-81-0058.     

Density estimation using game theory

In this note we show that the mathematical tools of cooperative game theory allow a successful approach to the statistical problem of estimating a density function. Specifically, any random sample of an absolutely continuous random variable determines a transferable utility game, the Shapley value of which proves to be an estimator of the density function of binned kernel and WARPing types, with good computational and statistical properties.Authors acknowledge the financial support of Spanish Ministry for Science and Technology and FEDER through projects BFM2002-03213 and BEC2002-04102-C02-02 and of Xunta de Galicia through projects PGIDT00PXI20104PR and PGIDT03PXIC20701PN. They also thank the comments of two anonymous referees.     

Bayesian approach for mixture models with grouped data

Finite mixture modeling approach is widely used for the analysis of bimodal or multimodal data that are individually observed in many situations. However, in some applications, the analysis becomes substantially challenging as the available data are grouped into categories. In this work, we assume that the observed data are grouped into distinct non-overlapping intervals and follow a finite mixture of normal distributions. For the inference of the model parameters, we propose a parametric approach that accounts for the categorical features of the data. The main idea of our method is to impute the missing information of the original data through the Bayesian framework using the Gibbs sampling techniques. The proposed method was compared with the maximum likelihood approach, which uses the Expectation-Maximization algorithm for the estimation of the model parameters. It was also illustrated with an application to the Old Faithful geyser data.     

A unified approach to estimation of noncentrality parameters,the multiple correlation coefficient,and mixture models

We consider a class of mixture models for positive continuous data and the estimation of an underlying parameter θ of the mixing distribution. With a unified approach, we obtain classes of dominating estimators under squared error loss of an unbiased estimator, which include smooth estimators. Applications include estimating noncentrality parameters of chi-square and F-distributions, as well as ρ ²/(1 ? ρ ²), where ρ is amultivariate correlation coefficient in a multivariate normal set-up. Finally, the findings are extended to situations, where there exists a lower bound constraint on θ.     

Local influence analysis for penalized Gaussian likelihood estimation in partially linear single-index models

Single-index model is a potentially tool for multivariate nonparametric regression, generalizes both the generalized linear models(GLM) and the missing-link function problem in GLM. In this paper, we extend Cook’s local influence analysis to the penalized Gaussian likelihood estimator based on P-spline for the partially linear single-index model. Some influence measures, based on the minor perturbation of the model, are derived for the penalized least squares estimation. An illustrative example is also presented.     

偏t正态数据下混合线性联合位置与尺度模型的参数估计

偏t正态分布是分析尖峰,厚尾数据的重要统计工具之一.研究提出了偏t正态数据下混合线性联合位置与尺度模型,通过EM算法和Newton-Raphson方法研究了该模型参数的极大似然估计.并通过随机模拟试验验证了所提出方法的有效性.最后,结合实际数据验证了该模型和方法具有实用性和可行性.     

Stochastic comparison algorithm for continuous optimization with estimation

The problem of stochastic optimization for arbitrary objective functions presents a dual challenge. First, one needs to repeatedly estimate the objective function; when no closed-form expression is available, this is only possible through simulation. Second, one has to face the possibility of determining local, rather than global, optima. In this paper, we show how the stochastic comparison approach recently proposed in Ref. 1 for discrete optimization can be used in continuous optimization. We prove that the continuous stochastic comparison algorithm converges to an -neighborhood of the global optimum for any >0. Several applications of this approach to problems with different features are provided and compared to simulated annealing and gradient descent algorithms.This work was supported in part by the National Science Foundation under Grants EID-92-12122 and ECS-88-01912, and by a Grant from United Technologies/Otis Elevator Company.     

Adaptive penalized M-estimation with current status data

Current status data arises when a continuous response is reduced to an indicator of whether the response is greater or less than a random threshold value. In this article we consider adaptive penalized M-estimators (including the penalized least squares estimators and the penalized maximum likelihood estimators) for nonparametric and semiparametric models with current status data, under the assumption that the unknown nonparametric parameters belong to unknown Sobolev spaces. The Cox model is used as a representative of the semiparametric models. It is shown that the modified penalized M-estimators of the nonparametric parameters can achieve adaptive convergence rates, even when the degrees of smoothing are not known in advance. consistency, asymptotic normality and inference based on the weighted bootstrap for the estimators of the regression parameter in the Cox model are also established. A simulation study is conducted for the Cox model to evaluate the finite sample efficacy of the proposed approach and to compare it with the ordinary maximum likelihood estimator. It is demonstrated that the proposed method is computationally superior.We apply the proposed approach to the California Partner Study analysis.