On the mean square error of maximum likelihood estimates of the distribution density of sufficient statistics of the multivariate normal distribution

The extensive use of maximum likelihood estimates underscores the importance of the problem of statistical estimation of their errors. These estimates are of utmost importance in cases where the family of normal distributions and the families related to the normal distributions are considered [1, 2, 4]. The mean square errors of the maximum likelihood estimates of the normal density were investigated in the author's paper [3]. The mean square errors of statistical estimates of some families of densities related to the normal distributions were considered in the papers [4–6]. In the present paper, we obtain an asymptotic expansion of the mean square error of the maximum likelihood estimates of the densities of the joint distribution of sufficient statistics of the family of multivariate normal distributions. The results obtained allow us to construct the mean square errors of the maximum likelihood estimates for the chi-square density and Wishart's density. Translated fromStatisticheskie Metody Otsenivaniya i Proverki Gipotez, pp. 4–11, Perm. 1990.     

Certain estimation problems for multivariate hypergeometric models

Certain estimation problems associated with the multivariate hypergeometric models: the property of completeness, maximum likelihood estimates of the parameters of multivariate negative hypergeometric, multivariate negative inverse hypergeometric, Bayesian estimation of the parameters of multivariate hypergeometric and multivariate inverse hypergeometrics are discussed in this paper. A two stage approach for generating the prior distribution, first by setting up a parametric super population and then choosing a prior distribution is followed. Posterior expectations and variances of certain functions of the parameters of the finite population are provided in cases of direct and inverse sampling procedures. It is shown that under extreme diffuseness of prior knowledge the posterior distribution of the finite population mean has an approximate mean and variance (N-n)S ²/Nn, providing a Bayesian interpretation for the classical unbiased estimates in traditional sample survey theory.     

Maximum likelihood estimates and likelihood ratio criteria for location and scale parameters of the multivariatel 1-norm symmetric distributions

In this paper we introduce three families of multivariate and matrixl ₁-norm symmetric distributions with location and scale parameters and discuss their maximum likelihood estimates and likelihood ratio criteria. It is shown that under certain condition sthey have the same form as those for independent exponential variates.Projects supported by the science Fund of the Chinese Academy of Sciences.     

A Flexible Model for Generalized Linear Regression with Measurement Error

This paper focuses on the question of specification of measurement error distribution and the distribution of true predictors in generalized linear models when the predictors are subject to measurement errors. The standard measurement error model typically assumes that the measurement error distribution and the distribution of covariates unobservable in the main study are normal. To make the model flexible enough we, instead, assume that the measurement error distribution is multivariate t and the distribution of true covariates is a finite mixture of normal densities. Likelihood–based method is developed to estimate the regression parameters. However, direct maximization of the marginal likelihood is numerically difficult. Thus as an alternative to it we apply the EM algorithm. This makes the computation of likelihood estimates feasible. The performance of the proposed model is investigated by simulation study.     

Estimating Frailty Models via Poisson Hierarchical Generalized Linear Models

Frailty models extend proportional hazards models to multivariate survival data. Hierarchical-likelihood provides a simple unified framework for various random effect models such as hierarchical generalized linear models, frailty models, and mixed linear models with censoring. Wereview the hierarchical-likelihood estimation methods for frailty models. Hierarchical-likelihood for frailty models can be expressed as that for Poisson hierarchical generalized linear models. Frailty models can thus be fitted using Poisson hierarchical generalized linear models. Properties of the new methodology are demonstrated by simulation. The new method reduces the bias of maximum likelihood and penalized likelihood estimates.     

Nonlinear logistic discrimination via regularized radial basis functions for classifying high-dimensional data

A flexible nonparametric method is proposed for classifying high- dimensional data with a complex structure. The proposed method can be regarded as an extended version of linear logistic discriminant procedures, in which the linear predictor is replaced by a radial-basis-expansion predictor. Radial basis functions with a hyperparameter are used to take the information on covariates and class labels into account; this was nearly impossible within the previously proposed hybrid learning framework. The penalized maximum likelihood estimation procedure is employed to obtain stable parameter estimates. A crucial issue in the model-construction process is the choice of a suitable model from candidates. This issue is examined from information-theoretic and Bayesian viewpoints and we employed Ando et al. (Japanese Journal of Applied Statistics, 31, 123–139, 2002)’s model evaluation criteria. The proposed method is available not only for the high-dimensional data but also for the variable selection problem. Real data analysis and Monte Carlo experiments show that our proposed method performs well in classifying future observations in practical situations. The simulation results also show that the use of the hyperparameter in the basis functions improves the prediction performance.     

Logspline Density Estimation for Censored Data

Abstract

Logspline density estimation is developed for data that may be right censored, left censored, or interval censored. A fully automatic method, which involves the maximum likelihood method and may involve stepwise knot deletion and either the Akaike information criterion (AIC) or Bayesian information criterion (BIC), is used to determine the estimate. In solving the maximum likelihood equations, the Newton–Raphson method is augmented by occasional searches in the direction of steepest ascent. Also, a user interface based on S is described for obtaining estimates of the density function, distribution function, and quantile function and for generating a random sample from the fitted distribution.     

Generalized least squares and maximum likelihood estimations of multivariate polychoric correlations

This paper investigates the generalized least squares estimation and the maximum likelihood estimation of the parameters in a multivariate polychoric correlations model, based on data from a multidimensional contingency table. Asymptotic properties of the estimators are discussed. An iterative procedure based on the Gauss-Newton algorithm is implemented to produce the generalized least squares estimates and the standard errors estimates. It is shown that via an iteratively reweighted method, the algorithm produces the maximum likelihood estimates as well. Numerical results on the finite sample behaviors of the methods are reported.     

An effective selection of regression variables when the error distribution is incorrectly specified

Summary An asymptotically efficient selection of regression variables is considered in the situation where the statistician estimates regression parameters by the maximum likelihood method but fails to choose a likelihood function matching the true error distribution. The proposed procedure is useful when a robust regression technique is applied but the data in fact do not require that treatment. Examples and a Monte Carlo study are presented and relationships to other selectors such as Mallows'C _p are investigated. Research supported by Deutsche Forschungsgemeinschaft, Sonderforschungsbereich 123 “Stochastische Mathematische Modelle” and AFOSR Contract No. F49620 82 C 0009.     

ML Estimation of the MultivariatetDistribution and the EM Algorithm

Maximum likelihood estimation of the multivariatetdistribution, especially with unknown degrees of freedom, has been an interesting topic in the development of the EM algorithm. After a brief review of the EM algorithm and its application to finding the maximum likelihood estimates of the parameters of thetdistribution, this paper provides new versions of the ECME algorithm for maximum likelihood estimation of the multivariatetdistribution from data with possibly missing values. The results show that the new versions of the ECME algorithm converge faster than the previous procedures. Most important, the idea of this new implementation is quite general and useful for the development of the EM algorithm. Comparisons of different methods based on two datasets are presented.     

An Inversion-Free Estimating Equations Approach for Gaussian Process Models

One of the scalability bottlenecks for the large-scale usage of Gaussian processes is the computation of the maximum likelihood estimates of the parameters of the covariance matrix. The classical approach requires a Cholesky factorization of the dense covariance matrix for each optimization iteration. In this work, we present an estimating equations approach for the parameters of zero-mean Gaussian processes. The distinguishing feature of this approach is that no linear system needs to be solved with the covariance matrix. Our approach requires solving an optimization problem for which the main computational expense for the calculation of its objective and gradient is the evaluation of traces of products of the covariance matrix with itself and with its derivatives. For many problems, this is an O(nlog?n) effort, and it is always no larger than O(n²). We prove that when the covariance matrix has a bounded condition number, our approach has the same convergence rate as does maximum likelihood in that the Godambe information matrix of the resulting estimator is at least as large as a fixed fraction of the Fisher information matrix. We demonstrate the effectiveness of the proposed approach on two synthetic examples, one of which involves more than 1 million data points.     

Estimation of parameters in the discrete distributions of order k

This paper considers estimating parameters in the discrete distributions of order k such as the binomial, the geometric, the Poisson and the logarithmic series distributions of order k. It is discussed how to calculate maximum likelihood estimates of parameters of the distributions based on independent observations. Further, asymptotic properties of estimators by the method of moments are investigated. In some cases, it is found that the values of asymptotic efficiency of the moment estimators are surprisingly close to one.     

Multivariate approximations to portfolio return distribution

This article proposes a three-step procedure to estimate portfolio return distributions under the multivariate Gram–Charlier (MGC) distribution. The method combines quasi maximum likelihood (QML) estimation for conditional means and variances and the method of moments (MM) estimation for the rest of the density parameters, including the correlation coefficients. The procedure involves consistent estimates even under density misspecification and solves the so-called ‘curse of dimensionality’ of multivariate modelling. Furthermore, the use of a MGC distribution represents a flexible and general approximation to the true distribution of portfolio returns and accounts for all its empirical regularities. An application of such procedure is performed for a portfolio composed of three European indices as an illustration. The MM estimation of the MGC (MGC-MM) is compared with the traditional maximum likelihood of both the MGC and multivariate Student’s t (benchmark) densities. A simulation on Value-at-Risk (VaR) performance for an equally weighted portfolio at 1 and 5 % confidence indicates that the MGC-MM method provides reasonable approximations to the true empirical VaR. Therefore, the procedure seems to be a useful tool for risk managers and practitioners.     

Contractivity and Analyticity in l p of Some Approximation of the Heat Equation

We consider the lumped mass method with piecewise linear finite elements in two dimensions. When the triangulation is of Delaunay type it is known that the discrete scheme satisfies a maximum principle. In this work we pursue the analysis and prove that the discrete semi-group is l ^p contractive in a sector, which implies smoothing effects and provide some resolvent estimates.     

Relative efficiencies of estimates using patterned covariances or correlations in the multivariate normal estimation problem

Summary The relative efficiency of maximum likelihood estimates is studied when taking advantage of underlying linear patterns in the covariances or correlations when estimating covariance matrices. We compare the variances of estimates of the covariance matrix obtained under two nested patterns with the assumption that the more restricted pattern is the true state. Formulas for the asymptotic variances are given which are exact for linear covariance patterns when explicit maximum likelihood estimates exist. Several specific examples are given using complete symmetry, circular symmetry and general covariance patterns as well as an example involving a covariance matrix with a linear pattern in the correlations.     

Maximum likelihood estimation for multivariate skew normal mixture models

This paper provides a flexible mixture modeling framework using the multivariate skew normal distribution. A feasible EM algorithm is developed for finding the maximum likelihood estimates of parameters in this context. A general information-based method for obtaining the asymptotic covariance matrix of the maximum likelihood estimators is also presented. The proposed methodology is illustrated with a real example and results are also compared with those obtained from fitting normal mixtures.     

Selection of smoothing parameters in<Emphasis Type="Italic">B</Emphasis>-spline nonparametric regression models using information criteria

We consider the use ofB-spline nonparametric regression models estimated by the maximum penalized likelihood method for extracting information from data with complex nonlinear structure. Crucial points inB-spline smoothing are the choices of a smoothing parameter and the number of basis functions, for which several selectors have been proposed based on cross-validation and Akaike information criterion known as AIC. It might be however noticed that AIC is a criterion for evaluating models estimated by the maximum likelihood method, and it was derived under the assumption that the ture distribution belongs to the specified parametric model. In this paper we derive information criteria for evaluatingB-spline nonparametric regression models estimated by the maximum penalized likelihood method in the context of generalized linear models under model misspecification. We use Monte Carlo experiments and real data examples to examine the properties of our criteria including various selectors proposed previously.     

Tests for nonparametric parts on partially linear single index models

Tests for nonparametric parts on partially linear single index models are considered in this paper. Based on the estimates obtained by the local linear method, the generalized likelihood ratio tests for the models are established. Under the null hypotheses the normalized tests follow asymptotically the χ2-distribution with the scale constants and the degrees of freedom being independent of the nuisance parameters, which is called the Wilks phenomenon. A simulated example is used to evaluate the performances of the testing procedures empirically.     

An optimal modification of the LIML estimation for many instruments and persistent heteroscedasticity

We consider the estimation of coefficients of a structural equation with many instrumental variables in a simultaneous equation system. It is mathematically equivalent to the estimating equations estimation or a reduced rank regression in the statistical multivariate linear models when the number of restrictions or the dimension of estimating equations increases with the sample size. As a semi-parametric method, we propose a class of modifications of the limited information maximum likelihood (LIML) estimator to improve its asymptotic properties as well as the small sample properties for many instruments and persistent heteroscedasticity. We show that an asymptotically optimal modification of the LIML estimator, which is called AOM-LIML, improves the LIML estimator and other estimation methods. We give a set of sufficient conditions for an asymptotic optimality when the number of instruments or the dimension of the estimating equations is large with persistent heteroscedasticity including a case of many weak instruments.     

Smoothing complements and randomized score functions

This paper establishes connections between two derivative estimation techniques:infinitesimal perturbation analysis (IPA) and thelikelihood ratio orscore function method. We introduce a systematic way of expanding the domain of the former to include that of the latter, and show that many likelihood ratio derivative estimators are IPA estimators obtained in a consistent manner through a special construction. Our extension of IPA is based onmultiplicative smoothing. A function with discontinuities is multiplied by asmoothing complement, a continuous function that takes the value zero at a jump of the first function. The product of these functions is continuous and provides an indirect derivative estimator after an appropriate normalization. We show that, in substantial generality, the derivative of a smoothing complement is a randomized score function: its conditional expectation is a derivative of a likelihood ratio. If no conditional expectation is applied, derivative estimates based on multiplicative smoothing have higher variance than corresponding estimates based on likelihood ratios.