Mixtures of truncated basis functions

In this paper we propose a framework, called mixtures of truncated basis functions (MoTBFs), for representing general hybrid Bayesian networks. The proposed framework generalizes both the mixture of truncated exponentials (MTEs) framework and the Mixture of Polynomials (MoPs) framework. Similar to MTEs and MoPs, MoTBFs are defined so that the potentials are closed under combination and marginalization, which ensures that inference in MoTBF networks can be performed efficiently using the Shafer-Shenoy architecture.Based on a generalized Fourier series approximation, we devise a method for efficiently approximating an arbitrary density function using the MoTBF framework. The translation method is more flexible than existing MTE or MoP-based methods, and it supports an online/anytime tradeoff between the accuracy and the complexity of the approximation. Experimental results show that the approximations obtained are either comparable or significantly better than the approximations obtained using existing methods.     

Fuzzy projection pursuit density estimation by eigenvalue method

This paper focuses on the use of kernel method and projection pursuit regression for non-parametric probability density estimation. Direct application of the kernel method is not able to pick up characteristic features of multidimensional density function. We propose a fuzzy projection pursuit density estimation based on the membership function and the eigenvector of the covariance matrix. Marginal densities along the subspace spanned by the projection vector are estimated. The proposed projection pursuit is one of the methods which are able to bypass the ‘curse of dimensionality’ in multidimensional density estimation. An application to experimental design for machining accuracy of end milling with the tool in small diameter is presented to demonstrate its usefulness.     

Classification Modulo Invariance,With Application to Face Recognition

This article presents techniques for constructing classifiers that combine statistical information from training data with tangent approximations to known transformations; it demonstrates the techniques by applying them to a face recognition task. Our approach is to build Bayes classifiers with approximate class-conditional probability densities for measured data. The high dimension of the measurements in modern classification problems such as speech or image recognition makes inferring probability densities from feasibly sized training datasets difficult. We address the difficulty by imposing severely simplifying assumptions and exploiting a priori information about transformations to which classification should be invariant. For the face recognition task, we used a five-parameter group of such transformations consisting of rotation, shifts, and scalings. On the face recognition task, a classifier based on our techniques has an error rate that is 20% lower than that of the best algorithm in a reference software distribution.     

Decision functions for chain classifiers based on Bayesian networks for multi-label classification

Multi-label classification problems require each instance to be assigned a subset of a defined set of labels. This problem is equivalent to finding a multi-valued decision function that predicts a vector of binary classes. In this paper we study the decision boundaries of two widely used approaches for building multi-label classifiers, when Bayesian network-augmented naive Bayes classifiers are used as base models: Binary relevance method and chain classifiers. In particular extending previous single-label results to multi-label chain classifiers, we find polynomial expressions for the multi-valued decision functions associated with these methods. We prove upper boundings on the expressive power of both methods and we prove that chain classifiers provide a more expressive model than the binary relevance method.     

Empirical Bayes inference for the Weibull model

In this study, the theory of statistical kernel density estimation has been applied for deriving non-parametric kernel prior to the empirical Bayes which frees the Bayesian inference from subjectivity that has worried some statisticians. For comparing the empirical Bayes based on the kernel prior with the fully Bayes based on the informative prior, the mean square error and the mean percentage error for the Weibull model parameters are studied based on these approaches under both symmetric and asymmetric loss functions, via Monte Carlo simulations. The results are quite favorable to the empirical Bayes that provides better estimates and outperforms the fully Bayes for different sample sizes and several values of the true parameters. Finally, a numerical example is given to demonstrate the efficiency of the empirical Bayes.     

Empirical Bayes predictive densities for high-dimensional normal models

This paper addresses the problem of estimating the density of a future outcome from a multivariate normal model. We propose a class of empirical Bayes predictive densities and evaluate their performances under the Kullback–Leibler (KL) divergence. We show that these empirical Bayes predictive densities dominate the Bayesian predictive density under the uniform prior and thus are minimax under some general conditions. We also establish the asymptotic optimality of these empirical Bayes predictive densities in infinite-dimensional parameter spaces through an oracle inequality.     

Two issues in using mixtures of polynomials for inference in hybrid Bayesian networks

We discuss two issues in using mixtures of polynomials (MOPs) for inference in hybrid Bayesian networks. MOPs were proposed by Shenoy and West for mitigating the problem of integration in inference in hybrid Bayesian networks. First, in defining MOP for multi-dimensional functions, one requirement is that the pieces where the polynomials are defined are hypercubes. In this paper, we discuss relaxing this condition so that each piece is defined on regions called hyper-rhombuses. This relaxation means that MOPs are closed under transformations required for multi-dimensional linear deterministic conditionals, such as Z = X + Y, etc. Also, this relaxation allows us to construct MOP approximations of the probability density functions (PDFs) of the multi-dimensional conditional linear Gaussian distributions using a MOP approximation of the PDF of the univariate standard normal distribution. Second, Shenoy and West suggest using the Taylor series expansion of differentiable functions for finding MOP approximations of PDFs. In this paper, we describe a new method for finding MOP approximations based on Lagrange interpolating polynomials (LIP) with Chebyshev points. We describe how the LIP method can be used to find efficient MOP approximations of PDFs. We illustrate our methods using conditional linear Gaussian PDFs in one, two, and three dimensions, and conditional log-normal PDFs in one and two dimensions. We compare the efficiencies of the hyper-rhombus condition with the hypercube condition. Also, we compare the LIP method with the Taylor series method.     

Variational Bayes approximations for clustering via mixtures of normal inverse Gaussian distributions

Parameter estimation for model-based clustering using a finite mixture of normal inverse Gaussian (NIG) distributions is achieved through variational Bayes approximations. Univariate NIG mixtures and multivariate NIG mixtures are considered. The use of variational Bayes approximations here is a substantial departure from the traditional EM approach and alleviates some of the associated computational complexities and uncertainties. Our variational algorithm is applied to simulated and real data. The paper concludes with discussion and suggestions for future work.     

Mixtures of Gamma Distributions With Applications

This article proposes a Bayesian density estimation method based upon mixtures of gamma distributions. It considers both the cases of known mixture size, using a Gibbs sampling scheme with a Metropolis step, and unknown mixture size, using a reversible jump technique that allows us to move from one mixture size to another. We illustrate our methods using a number of simulated datasets, generated from distributions covering a wide range of cases: single distributions, mixtures of distributions with equal means and different variances, mixtures of distributions with different means and small variances and, finally, a distribution contaminated by low-weighted distributions with different means and equal, small variances. An application to estimation of some quantities for a M/G/1 queue is given, using real E-mail data from CNR-IAMI.     

Adaptive Bayes type estimators of ergodic diffusion processes from discrete observations

We consider adaptive Bayesian estimation of both drift and diffusion coefficient parameters for ergodic multidimensional diffusion processes based on sampled data. Under a general condition on the discretization step of the sampled data, three kinds of adaptive Bayes type estimators are proposed by applying adaptive maximum likelihood type methods of Uchida and Yoshida (Stoch Process Appl 122:2885–2924, 2012) to Bayesian procedures. We show asymptotic normality and convergence of moments for the adaptive Bayes type estimators by means of the Ibragimov–Has’minskii–Kutoyants program together with the polynomial type large deviation inequality for the statistical random field.     

Malliavin calculus for difference approximations of multidimensional diffusions: Truncated local limit theorem

The truncated local limit theorem is proved for difference approximations of multidimensional diffusions. Under very mild conditions on the distributions of difference terms, this theorem states that the transition probabilities of these approximations, after truncation of some asymptotically negligible terms, possess densities uniformly convergent to the transition probability density for the limiting diffusion and satisfy certain uniform diffusion-type estimates. The proof is based on a new version of the Malliavin calculus for the product of a finite family of measures that may contain nontrivial singular components. Applications to the uniform estimation of mixing and convergence rates for difference approximations of stochastic differential equations and to the convergence of difference approximations of local times for multidimensional diffusions are presented. Published in Ukrains’kyi Matematychnyi Zhurnal, Vol. 60, No. 3, pp. 340–381, March, 2008.     

Efficient MCMC estimation of some elliptical copula regression models through scale mixtures of normals

This paper proposes an efficient estimation method for some elliptical copula regression models by expressing both copula density and marginal density functions as scale mixtures of normals (SMN). Implementing these models using the SMN is novel and allows efficient estimation via Bayesian methods. An innovative algorithm for the case of complex semicontinuous margins is also presented. We utilize the facts that copulas are invariant to the location and scale of the margins; all elliptical distributions have the same correlation structure; and some densities can be represented by the SMN. Two simulation studies, one on continuous margins and the other on semicontinuous margins, highlight the favorable performance of the proposed methods. Two empirical studies, one on the US excess returns and one on the Thai wage earnings, further illustrate the applicability of the proposals.     

Partition Weighted Approach For Estimating the Marginal Posterior Density With Applications

The computation of marginal posterior density in Bayesian analysis is essential in that it can provide complete information about parameters of interest. Furthermore, the marginal posterior density can be used for computing Bayes factors, posterior model probabilities, and diagnostic measures. The conditional marginal density estimator (CMDE) is theoretically the best for marginal density estimation but requires the closed-form expression of the conditional posterior density, which is often not available in many applications. We develop the partition weighted marginal density estimator (PWMDE) to realize the CMDE. This unbiased estimator requires only a single Markov chain Monte Carlo output from the joint posterior distribution and the known unnormalized posterior density. The theoretical properties and various applications of the PWMDE are examined in detail. The PWMDE method is also extended to the estimation of conditional posterior densities. We carry out simulation studies to investigate the empirical performance of the PWMDE and further demonstrate the desirable features of the proposed method with two real data sets from a study of dissociative identity disorder patients and a prostate cancer study, respectively. Supplementary materials for this article are available online.     

A Bayesian framework for the combination of classifier outputs

We explore a Bayesian framework for constructing combinations of classifier outputs, as a means to improving overall classification results. We propose a sequential Bayesian framework to estimate the posterior probability of being in a certain class given multiple classifiers. This framework, which employs meta-Gaussian modelling but makes no assumptions about the distribution of classifier outputs, allows us to capture nonlinear dependencies between the combined classifiers and individuals. An important property of our method is that it produces a combined classifier that dominates the individuals upon which it is based in terms of Bayes risk, error rate, and receiver operating characteristic (ROC) curve. To illustrate the method, we show empirical results from the combination of credit scores generated from four different scoring models.     

帕累托索赔分布中风险参数的经验贝叶斯估计

本文建立了贝叶斯模型,讨论了帕累托索赔额分布中参数的估计问题,得到了风险参数的极大似然估计、贝叶斯估计和信度估计,并证明了这些估计的强相合性.在均方误差的意义下比较了这些估计的好坏,并通过数值模拟对均方误差进行了验证,结果表明,贝叶斯估计比其他估计具有较小的均方误差.最后,给出了结构参数的估计并证明了经验贝叶斯估计和经验贝叶斯信度估计的渐近最优性.     

A Marginal Sampler for σ-Stable Poisson–Kingman Mixture Models

We investigate the class of σ-stable Poisson–Kingman random probability measures (RPMs) in the context of Bayesian nonparametric mixture modeling. This is a large class of discrete RPMs, which encompasses most of the popular discrete RPMs used in Bayesian nonparametrics, such as the Dirichlet process, Pitman–Yor process, the normalized inverse Gaussian process, and the normalized generalized Gamma process. We show how certain sampling properties and marginal characterizations of σ-stable Poisson–Kingman RPMs can be usefully exploited for devising a Markov chain Monte Carlo (MCMC) algorithm for performing posterior inference with a Bayesian nonparametric mixture model. Specifically, we introduce a novel and efficient MCMC sampling scheme in an augmented space that has a small number of auxiliary variables per iteration. We apply our sampling scheme to a density estimation and clustering tasks with unidimensional and multidimensional datasets, and compare it against competing MCMC sampling schemes. Supplementary materials for this article are available online.     

Inference in hybrid Bayesian networks using mixtures of polynomials

The main goal of this paper is to describe inference in hybrid Bayesian networks (BNs) using mixture of polynomials (MOP) approximations of probability density functions (PDFs). Hybrid BNs contain a mix of discrete, continuous, and conditionally deterministic random variables. The conditionals for continuous variables are typically described by conditional PDFs. A major hurdle in making inference in hybrid BNs is marginalization of continuous variables, which involves integrating combinations of conditional PDFs. In this paper, we suggest the use of MOP approximations of PDFs, which are similar in spirit to using mixtures of truncated exponentials (MTEs) approximations. MOP functions can be easily integrated, and are closed under combination and marginalization. This enables us to propagate MOP potentials in the extended Shenoy-Shafer architecture for inference in hybrid BNs that can include deterministic variables. MOP approximations have several advantages over MTE approximations of PDFs. They are easier to find, even for multi-dimensional conditional PDFs, and are applicable for a larger class of deterministic functions in hybrid BNs.     

��I�ͻ�Ͻ�β��ָ��——�������ֲ�����ʧЧģ�͵�ͳ�Ʒ���

In this paper, we investigate a competing risks model based on exponentiated Weibull distribution under Type-I progressively hybrid censoring scheme. To estimate the unknown parameters and reliability function, the maximum likelihood estimators and asymptotic confidence intervals are derived. Since Bayesian posterior density functions cannot be given in closed forms, we adopt Markov chain Monte Carlo method to calculate approximate Bayes estimators and highest posterior density credible intervals. To illustrate the estimation methods, a simulation study is carried out with numerical results. It is concluded that the maximum likelihood estimation and Bayesian estimation can be used for statistical inference in competing risks model under Type-I progressively hybrid censoring scheme.     

MARS: selecting basis functions and knots with an empirical Bayes method

An empirical Bayes method to select basis functions and knots in multivariate adaptive regression spline (MARS) is proposed, which takes both advantages of frequentist model selection approaches and Bayesian approaches. A penalized likelihood is maximized to estimate regression coefficients for selected basis functions, and an approximated marginal likelihood is maximized to select knots and variables involved in basis functions. Moreover, the Akaike Bayes information criterion (ABIC) is used to determine the number of basis functions. It is shown that the proposed method gives estimation of regression structure that is relatively parsimonious and more stable for some example data sets.     

Learning Bayesian network classifiers by risk minimization

Bayesian networks (BNs) provide a powerful graphical model for encoding the probabilistic relationships among a set of variables, and hence can naturally be used for classification. However, Bayesian network classifiers (BNCs) learned in the common way using likelihood scores usually tend to achieve only mediocre classification accuracy because these scores are less specific to classification, but rather suit a general inference problem. We propose risk minimization by cross validation (RMCV) using the 0/1 loss function, which is a classification-oriented score for unrestricted BNCs. RMCV is an extension of classification-oriented scores commonly used in learning restricted BNCs and non-BN classifiers. Using small real and synthetic problems, allowing for learning all possible graphs, we empirically demonstrate RMCV superiority to marginal and class-conditional likelihood-based scores with respect to classification accuracy. Experiments using twenty-two real-world datasets show that BNCs learned using an RMCV-based algorithm significantly outperform the naive Bayesian classifier (NBC), tree augmented NBC (TAN), and other BNCs learned using marginal or conditional likelihood scores and are on par with non-BN state of the art classifiers, such as support vector machine, neural network, and classification tree. These experiments also show that an optimized version of RMCV is faster than all unrestricted BNCs and comparable with the neural network with respect to run-time. The main conclusion from our experiments is that unrestricted BNCs, when learned properly, can be a good alternative to restricted BNCs and traditional machine-learning classifiers with respect to both accuracy and efficiency.