Heteroscedastic normal-exponential mixture models: Bayesian and classical approaches

In this paper, we introduce a Bayesian analysis for mixture of distributions belonging to the exponential family. As a special case we consider a mixture of normal exponential distributions including joint modeling of the mean and variance. We also consider joint modeling of the mean and variance heterogeneity. Markov Chain Monte Carlo (MCMC) methods are used to obtain the posterior summaries of interest. We also introduce and apply an EM algorithm, where the maximization is obtained applying the Fisher scoring algorithm. Finally, we also include analysis of real data sets to illustrate the proposed methodology.     

Heteroscedastic nonlinear regression models based on scale mixtures of skew-normal distributions

An extension of some standard likelihood based procedures to heteroscedastic nonlinear regression models under scale mixtures of skew-normal (SMSN) distributions is developed. We derive a simple EM-type algorithm for iteratively computing maximum likelihood (ML) estimates and the observed information matrix is derived analytically. Simulation studies demonstrate the robustness of this flexible class against outlying and influential observations, as well as nice asymptotic properties of the proposed EM-type ML estimates. Finally, the methodology is illustrated using an ultrasonic calibration data.     

Robust variable selection for finite mixture regression models

Finite mixture regression (FMR) models are frequently used in statistical modeling, often with many covariates with low significance. Variable selection techniques can be employed to identify the covariates with little influence on the response. The problem of variable selection in FMR models is studied here. Penalized likelihood-based approaches are sensitive to data contamination, and their efficiency may be significantly reduced when the model is slightly misspecified. We propose a new robust variable selection procedure for FMR models. The proposed method is based on minimum-distance techniques, which seem to have some automatic robustness to model misspecification. We show that the proposed estimator has the variable selection consistency and oracle property. The finite-sample breakdown point of the estimator is established to demonstrate its robustness. We examine small-sample and robustness properties of the estimator using a Monte Carlo study. We also analyze a real data set.     

Partially adaptive robust estimation of regression models and applications

This paper provides an accessible exposition of recently developed partially adaptive estimation methods and their application. These methods are robust to thick-tailed or asymmetric error distributions and should be of interest to researchers and practitioners in data mining, agent learning, and mathematical modeling in a wide range of disciplines. In particular, partially adaptive estimation methods can serve as robust alternatives to ordinary regression analysis, as well as machine learning methods developed by the artificial intelligence and computing communities.Results from analysis of three problem domains demonstrate application of the theory.     

A class of mixture of experts models for general insurance: Theoretical developments

In the Property and Casualty (P&C) ratemaking process, it is critical to understand the effect of policyholders’ risk profile to the number and amount of claims, the dependence among various business lines and the claim distributions. To include all the above features, it is essential to develop a regression model which is flexible and theoretically justified. Motivated by the issues above, we propose a class of logit-weighted reduced mixture of experts (LRMoE) models for multivariate claim frequencies or severities distributions. LRMoE is interpretable, as it has two components: Gating functions, which classify policyholders into various latent sub-classes; and Expert functions, which govern the distributional properties of the claims. Also, upon the development of denseness theory in regression setting, we can heuristically interpret the LRMoE as a “fully flexible” model to capture any distributional, dependence and regression structures subject to a denseness condition. Further, the mathematical tractability of the LRMoE is guaranteed since it satisfies various marginalization and moment properties. Finally, we discuss some special choices of expert functions that make the corresponding LRMoE “fully flexible”. In the subsequent paper (Fung et al., 2019b), we will focus on the estimation and application aspects of the LRMoE.     

Heteroscedastic mixture transition distribution (HMTD) model

The paper further studies the heteroscedastic mixture transition distribution (HMTD) model introduced by Berchtold. Both the expectation and the standard deviation of each component are written as functions of the past of the process. The stationarity conditions are derived. An expectation conditional maximization (ECM) algorithm is used and shown to work well for estimation, the model selection problem is addressed, and the formulaes for computing the observed information matrix are derived. The shape changing feature of conditional distributions makes the model capable of modelling time series with asymmetric or multimodal distribution. The model is applied to several simulated and real datasets with satisfactory results.     

Laplace Variational Approximation for Semiparametric Regression in the Presence of Heteroscedastic Errors

Variational approximations provide fast, deterministic alternatives to Markov chain Monte Carlo for Bayesian inference on the parameters of complex, hierarchical models. Variational approximations are often limited in practicality in the absence of conjugate posterior distributions. Recent work has focused on the application of variational methods to models with only partial conjugacy, such as in semiparametric regression with heteroscedastic errors. Here, both the mean and log variance functions are modeled as smooth functions of covariates. For this problem, we derive a mean field variational approximation with an embedded Laplace approximation to account for the nonconjugate structure. Empirical results with simulated and real data show that our approximate method has significant computational advantages over traditional Markov chain Monte Carlo; in this case, a delayed rejection adaptive Metropolis algorithm. The variational approximation is much faster and eliminates the need for tuning parameter selection, achieves good fits for both the mean and log variance functions, and reasonably reflects the posterior uncertainty. We apply the methods to log-intensity data from a small angle X-ray scattering experiment, in which properly accounting for the smooth heteroscedasticity leads to significant improvements in posterior inference for key physical characteristics of an organic molecule.     

Clustering electricity consumers using high‐dimensional regression mixture models

A massive amount of data about individual electrical consumptions are now provided with new metering technologies and smart grids. These new data are especially useful for load profiling and load modeling at different scales of the electrical network. A new methodology based on mixture of high‐dimensional regression models is used to perform clustering of individual customers. It leads to uncovering clusters corresponding to different regression models. Temporal information is incorporated in order to prepare the next step, the fit of a forecasting model in each cluster. Only the electrical signal is involved, slicing the electrical signal into consecutive curves to consider it as a discrete time series of curves. Interpretation of the models is given on a real smart meter dataset of Irish customers.     

Theorems and applications of two-dimensional Laplace transforms

The object of this paper is to establish two new formulas for calculating Laplace tranform pairs of two dimensions from known one-dimensional Laplace transforms. Two boundary-value problems are also solved by using double Laplace transforms.Published in Ukrainskii Matematicheskii Zhurnal, Vol. 39, No. 3, pp. 374–377, May–June, 1987.     

Robust errors-in-variables linear regression via Laplace distribution

Robust estimation procedures for linear and mixture linear errors-in-variables regression models are proposed based on the relationship between the least absolute deviation criterion and maximum likelihood estimation in a Laplace distribution. The finite sample performance of the proposed procedures is evaluated by simulation studies.     

Heteroscedastic replicated measurement error models under asymmetric heavy-tailed distributions

We propose a heteroscedastic replicated measurement error model based on the class of scale mixtures of skew-normal distributions, which allows the variances of measurement errors to vary across subjects. We develop EM algorithms to calculate maximum likelihood estimates for the model with or without equation error. An empirical Bayes approach is applied to estimate the true covariate and predict the response. Simulation studies show that the proposed models can provide reliable results and the inference is not unduly affected by outliers and distribution misspecification. The method has also been used to analyze a real data of plant root decomposition.     

Finite mixture of regression models for censored data based on scale mixtures of normal distributions

Advances in Data Analysis and Classification - In statistical analysis, particularly in econometrics, the finite mixture of regression models based on the normality assumption is routinely used to...     

A new class of Laplace inverses and their applications

In this work, a new class of inverse Laplace transforms of exponential functions involving nested square roots are determined. Using these new inverses and other techniques from Laplace transform theory, a new class of three-parameter definite integrals, that yield to exact evaluation, is generated. It is shown that these integrals evaluate to simple closed-form expressions. These results are then verified using independent analytical techniques. Special and limiting cases of the parameters are investigated, some of which yield well-known expressions from classical analysis. Asymptotic results for these integrals and inverses are also given. In addition, a representation of the complementary error function as a limit is presented. Last, some aspects concerning the numerical implementation of these inverses are discussed and several applications in continuum mechanics are noted.     

Combined regression models

The goal of this paper is two-fold. First, new regression models obtained by combinations of the least squares (LS), minimax (MM), and the least sum of absolute deviations (LSAD) are proposed. Second, measures for assessing the influence of observations on the fitted models are suggested. The paper is interdisciplinary because the theory behind the proposed method draws from results in the operations research area. The methods are illustrated by their application to some examples and graphical illustrations are given.     

Extending local mixture models

Local mixture models have proved useful in many statistical applications. This paper looks at ways in which the local assumption, which is used in an asymptotic approximation, can be relaxed in order to generate a much larger class of models which still have the very attractive geometric and inferential properties of local mixture models. The tool used to develop this large class of models is the Karhunen–Loève decomposition. Computational issues associated with working with these models are also briefly considered.     

Copula analysis of mixture models

Contemporary computers collect databases that can be too large for classical methods to handle. The present work takes data whose observations are distribution functions (rather than the single numerical point value of classical data) and presents a computational statistical approach of a new methodology to group the distributions into classes. The clustering method links the searched partition to the decomposition of mixture densities, through the notions of a function of distributions and of multi-dimensional copulas. The new clustering technique is illustrated by ascertaining distinct temperature and humidity regions for a global climate dataset and shows that the results compare favorably with those obtained from the standard EM algorithm method.