Comments on “Imprecise probability models for learning multinomial distributions from data. Applications to learning credal networks” by Andrés R. Masegosa and Serafín Moral

We briefly overview the problem of learning probabilities from data using imprecise probability models that express very weak prior beliefs. Then we comment on the new contributions to this question given in the paper by Masegosa and Moral and provide some insights about the performance of their models in data mining experiments of classification.     

Learning imprecise probability models: Conceptual and practical challenges

The paper by Masegosa and Moral, on “Imprecise probability models for learning multinomial distributions from data”, considers the combination of observed data and minimal prior assumptions so as to produce possibly interval-valued parameter estimates. We offer an evaluation of Masegosa and Moral's proposals.     

An ensemble method using credal decision trees

Supervised classification learning can be considered as an important tool for decision support. In this paper, we present a method for supervised classification learning, which ensembles decision trees obtained via convex sets of probability distributions (also called credal sets) and uncertainty measures. Our method forces the use of different decision trees and it has mainly the following characteristics: it obtains a good percentage of correct classifications and an improvement in time of processing compared with known classification methods; it not needs to fix the number of decision trees to be used; and it can be parallelized to apply it on very large data sets.     

Bayesian networks with a logistic regression model for the conditional probabilities

Logistic regression techniques can be used to restrict the conditional probabilities of a Bayesian network for discrete variables. More specifically, each variable of the network can be modeled through a logistic regression model, in which the parents of the variable define the covariates. When all main effects and interactions between the parent variables are incorporated as covariates, the conditional probabilities are estimated without restrictions, as in a traditional Bayesian network. By incorporating interaction terms up to a specific order only, the number of parameters can be drastically reduced. Furthermore, ordered logistic regression can be used when the categories of a variable are ordered, resulting in even more parsimonious models. Parameters are estimated by a modified junction tree algorithm. The approach is illustrated with the Alarm network.     

Generalized loopy 2U: A new algorithm for approximate inference in credal networks

Credal networks generalize Bayesian networks by relaxing the requirement of precision of probabilities. Credal networks are considerably more expressive than Bayesian networks, but this makes belief updating NP-hard even on polytrees. We develop a new efficient algorithm for approximate belief updating in credal networks. The algorithm is based on an important representation result we prove for general credal networks: that any credal network can be equivalently reformulated as a credal network with binary variables; moreover, the transformation, which is considerably more complex than in the Bayesian case, can be implemented in polynomial time. The equivalent binary credal network is then updated by L2U, a loopy approximate algorithm for binary credal networks. Overall, we generalize L2U to non-binary credal networks, obtaining a scalable algorithm for the general case, which is approximate only because of its loopy nature. The accuracy of the inferences with respect to other state-of-the-art algorithms is evaluated by extensive numerical tests.     

Nonparametric adaptive detection in fading channels based on sequential Monte Carlo and Bayesian model averaging

Recently, a Bayesian receiver for blind detection in fading channels has been proposed by Chen, Wang and Liu (200,IEEE Trans. Inform. Theory,46, 2079–2094), based on the sequential Monte Carlo methodology. That work is built on a parametric modelling of the fading process in the form of a state-space model, and assumes the knowledge of the second-order statistics of the fading channel. In this paper, we develop a nonparametric approach to the problem of blind detection in fading channels, without assuming any knowledge of the channel statistics. The basic idea is to decompose the fading process using a wavelet basis, and to use the sequential Monte Carlo technique to track both the wavelet coefficients and the transmitted symbols. Moreover, the algorithm is adaptive to time varying speed/smoothness in the fading process and the uncertainty on the number of wavelet coefficients (shrinkage order) needed. Simulation results are provided to demonstrate the excellent performance of the proposed blind adaptive receivers. This work was supported in part by the U.S. National Science Foundation (NSF) under grants CCR-9875314, CCR-9980599, DMS-9982846, DMS-0073651 and DMS-0073601.     

Interval estimates for posterior probabilities in a multivariate normal classification model

This paper is devoted to the asymptotic distribution of estimators for the posterior probability that a p-dimensional observation vector originates from one of k normal distributions with identical covariance matrices. The estimators are based on training samples for the k distributions involved. Observation vector and prior probabilities are regarded as given constants. The validity of various estimators and approximate confidence intervals is investigated by simulation experiments.     

A prior near-ignorance Gaussian process model for nonparametric regression

This paper proposes a prior near-ignorance model for regression based on a set of Gaussian Processes (GP). GPs are natural prior distributions for Bayesian regression. They offer a great modeling flexibility and have found widespread application in many regression problems. However, a GP requires the prior elicitation of its mean function, which represents our prior belief about the shape of the regression function, and of the covariance between any two function values.In the absence of prior information, it may be difficult to fully specify these infinite dimensional parameters. In this work, by modeling the prior mean of the GP as a linear combination of a set of basis functions and assuming as prior for the combination coefficients a set of conjugate distributions obtained as limits of truncate exponential priors, we have been able to model prior ignorance about the mean of the GP. The resulting model satisfies translation invariance, learning and, under some constraints, convergence, which are desirable properties for a prior near-ignorance model. Moreover, it is shown in this paper how this model can be extended to allow for a weaker specification of the GP covariance between function values, by letting each basis function to vary in a set of functions.Application to hypothesis testing has shown how the use of this model induces the capability of automatically detecting when a reliable decision cannot be made based on the available data.     

Merging experts’ opinions: A Bayesian hierarchical model with mixture of prior distributions

In this paper, a general approach is proposed to address a full Bayesian analysis for the class of quadratic natural exponential families in the presence of several expert sources of prior information. By expressing the opinion of each expert as a conjugate prior distribution, a mixture model is used by the decision maker to arrive at a consensus of the sources. A hyperprior distribution on the mixing parameters is considered and a procedure based on the expected Kullback–Leibler divergence is proposed to analytically calculate the hyperparameter values. Next, the experts’ prior beliefs are calibrated with respect to the combined posterior belief over the quantity of interest by using expected Kullback–Leibler divergences, which are estimated with a computationally low-cost method. Finally, it is remarkable that the proposed approach can be easily applied in practice, as it is shown with an application.     

A discrete particle swarm optimization method for feature selection in binary classification problems

This paper investigates the feature subset selection problem for the binary classification problem using logistic regression model. We developed a modified discrete particle swarm optimization (PSO) algorithm for the feature subset selection problem. This approach embodies an adaptive feature selection procedure which dynamically accounts for the relevance and dependence of the features included the feature subset. We compare the proposed methodology with the tabu search and scatter search algorithms using publicly available datasets. The results show that the proposed discrete PSO algorithm is competitive in terms of both classification accuracy and computational performance.     

Spatio-temporal averaging for a class of hybrid systems and application to conductance-based neuron models

We obtain a limit theorem endowed with quantitative estimates for a general class of infinite dimensional hybrid processes with intrinsically two different time scales and including a population. As an application, we consider a large class of conductance-based neuron models describing the nerve impulse propagation along a neural cell at the scales of ion channels.     

Independence for full conditional probabilities: Structure,factorization, non-uniqueness,and Bayesian networks

This paper examines concepts of independence for full conditional probabilities; that is, for set-functions that encode conditional probabilities as primary objects, and that allow conditioning on events of probability zero. Full conditional probabilities have been used in economics, in philosophy, in statistics, in artificial intelligence. This paper characterizes the structure of full conditional probabilities under various concepts of independence; limitations of existing concepts are examined with respect to the theory of Bayesian networks. The concept of layer independence (factorization across layers) is introduced; this seems to be the first concept of independence for full conditional probabilities that satisfies the graphoid properties of Symmetry, Redundancy, Decomposition, Weak Union, and Contraction. A theory of Bayesian networks is proposed where full conditional probabilities are encoded using infinitesimals, with a brief discussion of hyperreal full conditional probabilities.     

A higher order Markov model for analyzing covariate dependence

During the recent past, there has been a renewed interest in Markov chain for its attractive properties for analyzing real life data emerging from time series or longitudinal data in various fields. The models were proposed for fitting first or higher order Markov chains. However, there is a serious lack of realistic methods for linking covariate dependence with transition probabilities in order to analyze the factors associated with such transitions especially for higher order Markov chains. L.R. Muenz and L.V. Rubinstein [Markov models for covariate dependence of binary sequences, Biometrics 41 (1985) 91–101] employed logistic regression models to analyze the transition probabilities for a first order Markov model. The methodology is still far from generalization in terms of formulating a model for higher order Markov chains. In this study, it is aimed to provide a comprehensive covariate-dependent Markov model for higher order. The proposed model generalizes the estimation procedure for Markov models for any order. The proposed models and inference procedures are simple and the covariate dependence of the transition probabilities of any order can be examined without making the underlying model complex. An example from rainfall data is illustrated in this paper that shows the utility of the proposed model for analyzing complex real life problems. The application of the proposed method indicates that the higher order covariate dependent Markov models can be conveniently employed in a very useful manner and the results can provide in-depth insights to both the researchers and policymakers to resolve complex problems of underlying factors attributing to different types of transitions, reverse transitions and repeated transitions. The estimation and test procedures can be employed for any order of Markov model without making the theory and interpretation difficult for the common users.     

Technology scoring model for reflecting evaluator’s perception within confidence limits

The main purpose of this study is to propose a new technology scoring model for reflecting the total perception scoring phenomenon which happens often in many evaluation settings. A base model used is a logistic regression for non-default prediction of a firm. The point estimator used to predict the probability for non-default based on this model does not consider the risk involved in the estimation error. We propose to update the point estimator within its confidence interval using the evaluator’s perception. The proposed approach takes into account not only the risk involved in the estimation error of the point estimator but also the total perception scoring phenomenon. Empirical evidence of a better prediction ability of the proposed model is displayed in terms of the area under the ROC curves. Additionally, we showed that the proposed model can take advantage when it is applied to smaller data size. It is expected that the proposed approach can be applied to various technology related decision-makings such as R&D investment, alliance, transfer, and loan.     

Recursive partition and amalgamation with the exponential family: Theory and applications

The theory of tree-growing (RECPAM approach) is developed for outcome variables which are distributed as the canonical exponential family. The general RECPAM approach (consisting of three steps: recursive partition, pruning and amalgamation), is reviewed. This is seen as constructing a partition with maximal information content about a parameter to be predicted, followed by simplification by the elimination of ‘negligible’ information. The measure of information is defined for an exponential family outcome as a deviance difference, and appropriate modifications of pruning and amalgamation rules are discussed. It is further shown how the proposed approach makes it possible to develop tree-growing for situations usually treated by generalized linear models (GLIM). In particular, Poisson and logistic regression can be tree-structured. Moreover, censored survival data can be treated, as in GLIM, by observing a formal equivalence of the likelihood under random censoring and an appropriate Poisson model. Three examples are given of application to Poisson, binary and censored survival data.     

Building a set of additive value functions representing a reference preorder and intensities of preference: GRIP method

We present a method called Generalized Regression with Intensities of Preference (GRIP) for ranking a finite set of actions evaluated on multiple criteria. GRIP builds a set of additive value functions compatible with preference information composed of a partial preorder and required intensities of preference on a subset of actions, called reference actions. It constructs not only the preference relation in the considered set of actions, but it also gives information about intensities of preference for pairs of actions from this set for a given decision maker (DM). Distinguishing necessary and possible consequences of preference information on the considered set of actions, GRIP answers questions of robustness analysis. The proposed methodology can be seen as an extension of the UTA method based on ordinal regression. GRIP can also be compared to the AHP method, which requires pairwise comparison of all actions and criteria, and yields a priority ranking of actions. As for the preference information being used, GRIP can be compared, moreover, to the MACBETH method which also takes into account a preference order of actions and intensity of preference for pairs of actions. The preference information used in GRIP does not need, however, to be complete: the DM is asked to provide comparisons of only those pairs of reference actions on particular criteria for which his/her judgment is sufficiently certain. This is an important advantage comparing to methods which, instead, require comparison of all possible pairs of actions on all the considered criteria. Moreover, GRIP works with a set of general additive value functions compatible with the preference information, while other methods use a single and less general value function, such as the weighted-sum.     

Score and Wald tests for the multivariate growth curve model with missing data

We present the score and Wald test analogues to Srivastava's (1985, Comm. Statist. A—Theory Methods, 14, 775–792) likelihood ratio tests for the multivariate growth curve model with missing data, and illustrate their use with data from an immunotherapy experiment (Fukushima et al. (1982, Int. J. Cancer, 29, 107–112, 113–117)).     

Security protection design for deception and real system regimes: A model and analysis

In this paper, we model a possible deception system with the explicit purpose of enticing unauthorized users and restricting their access to the real system. The proposed model represents a system designer’s defensive actions against intruders in a way that maximizes the difference between the intruders’ cost and the system designer’s cost of system protection. Under the assumption of a dual entity system, the proposed model shows that intruders differ in behavior depending on the system’s vulnerability at the time of intrusion as well as depending on their own economic incentives. The optimal results of the proposed model provide the system designer with insights on how to configure the level of protection for the two systems.