Independence for full conditional probabilities: Structure,factorization, non-uniqueness,and Bayesian networks

This paper examines concepts of independence for full conditional probabilities; that is, for set-functions that encode conditional probabilities as primary objects, and that allow conditioning on events of probability zero. Full conditional probabilities have been used in economics, in philosophy, in statistics, in artificial intelligence. This paper characterizes the structure of full conditional probabilities under various concepts of independence; limitations of existing concepts are examined with respect to the theory of Bayesian networks. The concept of layer independence (factorization across layers) is introduced; this seems to be the first concept of independence for full conditional probabilities that satisfies the graphoid properties of Symmetry, Redundancy, Decomposition, Weak Union, and Contraction. A theory of Bayesian networks is proposed where full conditional probabilities are encoded using infinitesimals, with a brief discussion of hyperreal full conditional probabilities.     

Bias analysis for misclassification in a multicategorical exposure in a logistic regression model

We derive the expressions of asymptotic biases when ignoring the misclassification in a multicategory exposure. For a model with a misclassified exposure variable only, we provide a general conclusion on the direction of the biases under nondifferential misclassification assumption. To better understand the bias formulas, we use a numerical example.     

Gated Bayesian networks for algorithmic trading

This paper introduces a new probabilistic graphical model called gated Bayesian network (GBN). This model evolved from the need to represent processes that include several distinct phases. In essence, a GBN is a model that combines several Bayesian networks (BNs) in such a manner that they may be active or inactive during queries to the model. We use objects called gates to combine BNs, and to activate and deactivate them when predefined logical statements are satisfied. In this paper we also present an algorithm for semi-automatic learning of GBNs. We use the algorithm to learn GBNs that output buy and sell decisions for use in algorithmic trading systems. We show how the learnt GBNs can substantially lower risk towards invested capital, while they at the same time generate similar or better rewards, compared to the benchmark investment strategy buy-and-hold. We also explore some differences and similarities between GBNs and other related formalisms.     

Hierarchical Bayesian models applied to air surveillance radars

Evaluation tests for air surveillance radars are often formulated in terms of the probability to detect a target at a specified range. Statistical methods applied in these tests do not explore all data in a full probabilistic model, which is crucial when dealing with small samples. The collected data are arranged longitudinally, in different levels (altitude), indexed both in time and distance. In this context we propose the application of dynamic Bayesian hierarchical models as an efficient way to incorporate the complete data set. Markov Chain Monte Carlo methods (MCMC) are used to make inference and to evaluate the proposed models.     

Discounted Bayesian search problems with unknown detection probabilities

In this paper we consider three discrete-time discounted Bayesian search problems with an unknown number of objects and uncertainty about the distribution of the objects among the boxes. Moreover, we admit uncertainty about the detection probabilities. The goal is to determine a policy which finds (dependent on the search problem) at least one object or all objects with minimal expected total cost. We give sufficient conditions for the optimality of the greedy policy which has been introduced in Liebig/Rieder (1996). For some examples in which the greedy policy is not optimal we derive a bound for the error.     

The breakdown behavior of the maximum likelihood estimator in the logistic regression model

In this note we discuss the breakdown behavior of the maximum likelihood (ML) estimator in the logistic regression model. We formally prove that the ML-estimator never explodes to infinity, but rather breaks down to zero when adding severe outliers to a data set. An example confirms this behavior.     

An optimization-based approach for the design of Bayesian networks

Bayesian networks model conditional dependencies among the domain variables, and provide a way to deduce their interrelationships as well as a method for the classification of new instances. One of the most challenging problems in using Bayesian networks, in the absence of a domain expert who can dictate the model, is inducing the structure of the network from a large, multivariate data set. We propose a new methodology for the design of the structure of a Bayesian network based on concepts of graph theory and nonlinear integer optimization techniques.     

A preferential attachment model with Poisson growth for scale-free networks

We propose a scale-free network model with a tunable power-law exponent. The Poisson growth model, as we call it, is an offshoot of the celebrated model of Barabási and Albert where a network is generated iteratively from a small seed network; at each step a node is added together with a number of incident edges preferentially attached to nodes already in the network. A key feature of our model is that the number of edges added at each step is a random variable with Poisson distribution, and, unlike the Barabási–Albert model where this quantity is fixed, it can generate any network. Our model is motivated by an application in Bayesian inference implemented as Markov chain Monte Carlo to estimate a network; for this purpose, we also give a formula for the probability of a network under our model.     

Bayesian priors based on a parameter transformation using the distribution function

One of the tasks of the Bayesian consulting statistician is to elicit prior information from his client who may be unfamiliar with parametric statistical models. In some cases it may be more illuminating to base a prior distribution for parameter on the transformed version F(/), where F is the data distribution function and v is a designated reference value, rather than on directly. This approach is outlined and explored in various directions to assess its implications. Some applications are given, including general linear regression and transformed linear models.     

Hierarchical linear regression models for conditional quantiles

The quantile regression has several useful features and therefore is gradually developing into a comprehensive approach to the statistical analysis of linear and nonlinear response models, but it cannot deal effectively with the data with a hierarchical structure. In practice, the existence of such data hierarchies is neither accidental nor ignorable, it is a common phenomenon. To ignore this hierarchical data structure risks overlooking the importance of group effects, and may also render many of the traditional statistical analysis techniques used for studying data relationships invalid. On the other hand, the hierarchical models take a hierarchical data structure into account and have also many applications in statistics, ranging from overdispersion to constructing min-max estimators. However, the hierarchical models are virtually the mean regression, therefore, they cannot be used to characterize the entire conditional distribution of a dependent variable given high-dimensional covariates. Furthermore, the estimated coefficient vector (marginal effects) is sensitive to an outlier observation on the dependent variable. In this article, a new approach, which is based on the Gauss-Seidel iteration and taking a full advantage of the quantile regression and hierarchical models, is developed. On the theoretical front, we also consider the asymptotic properties of the new method, obtaining the simple conditions for an n1/2-convergence and an asymptotic normality. We also illustrate the use of the technique with the real educational data which is hierarchical and how the results can be explained.     

Credal model averaging for classification: representing prior ignorance and expert opinions

Bayesian model averaging (BMA) is the state of the art approach for overcoming model uncertainty. Yet, especially on small data sets, the results yielded by BMA might be sensitive to the prior over the models. Credal model averaging (CMA) addresses this problem by substituting the single prior over the models by a set of priors (credal set). Such approach solves the problem of how to choose the prior over the models and automates sensitivity analysis. We discuss various CMA algorithms for building an ensemble of logistic regressors characterized by different sets of covariates. We show how CMA can be appropriately tuned to the case in which one is prior-ignorant and to the case in which instead domain knowledge is available. CMA detects prior-dependent instances, namely instances in which a different class is more probable depending on the prior over the models. On such instances CMA suspends the judgment, returning multiple classes. We thoroughly compare different BMA and CMA variants on a real case study, predicting presence of Alpine marmot burrows in an Alpine valley. We find that BMA is almost a random guesser on the instances recognized as prior-dependent by CMA.     

Learning from incomplete data in Bayesian networks with qualitative influences

Domain experts can often quite reliably specify the sign of influences between variables in a Bayesian network. If we exploit this prior knowledge in estimating the probabilities of the network, it is more likely to be accepted by its users and may in fact be better calibrated with reality. We present two algorithms that exploit prior knowledge of qualitative influences in learning the parameters of a Bayesian network from incomplete data. The isotonic regression EM, or irEM, algorithm adds an isotonic regression step to standard EM in each iteration, to obtain parameter estimates that satisfy the given qualitative influences. In an attempt to reduce the computational burden involved, we further define the qirEM algorithm that enforces the constraints imposed by the qualitative influences only once, after convergence of standard EM. We evaluate the performance of both algorithms through experiments. Our results demonstrate that exploitation of the qualitative influences improves the parameter estimates over standard EM, and more so if the proportion of missing data is relatively large. The results also show that the qirEM algorithm performs just as well as its computationally more expensive counterpart irEM.     

A variable selection method based on Tabu search for logistic regression models

A Tabu search method is proposed and analysed for selecting variables that are subsequently used in Logistic Regression Models. The aim is to find from among a set of m variables a smaller subset which enables the efficient classification of cases. Reducing dimensionality has some very well-known advantages that are summarized in literature. The specific problem consists in finding, for a small integer value of p, a subset of size p of the original set of variables that yields the greatest percentage of hits in Logistic Regression. The proposed Tabu search method performs a deep search in the solution space that alternates between a basic phase (that uses simple moves) and a diversification phase (to explore regions not previously visited). Testing shows that it obtains significantly better results than the Stepwise, Backward or Forward methods used by classic statistical packages. Some results of applying these methods are presented.     

约束条件下线性模型协方差阵扰动的影响分析

该文研究了协方差阵扰动和数据删除对最佳线性无偏估计(BLUE)的影响问题, 给出了在约束条件下一般线性模型与在约束条件下Gauss-Markov模型及在约束条件下数据删除模型中回归参数β的BLUE之间的关系式. 作者还定义了度量影响大小的广义Cook距离D_V并给出了D_V的两个计算公式.     

Simulation metamodeling with dynamic Bayesian networks

This paper presents a novel approach to simulation metamodeling using dynamic Bayesian networks (DBNs) in the context of discrete event simulation. A DBN is a probabilistic model that represents the joint distribution of a sequence of random variables and enables the efficient calculation of their marginal and conditional distributions. In this paper, the construction of a DBN based on simulation data and its utilization in simulation analyses are presented. The DBN metamodel allows the study of the time evolution of simulation by tracking the probability distribution of the simulation state over the duration of the simulation. This feature is unprecedented among existing simulation metamodels. The DBN metamodel also enables effective what-if analysis which reveals the conditional evolution of the simulation. In such an analysis, the simulation state at a given time is fixed and the probability distributions representing the state at other time instants are updated. Simulation parameters can be included in the DBN metamodel as external random variables. Then, the DBN offers a way to study the effects of parameter values and their uncertainty on the evolution of the simulation. The accuracy of the analyses allowed by DBNs is studied by constructing appropriate confidence intervals. These analyses could be conducted based on raw simulation data but the use of DBNs reduces the duration of repetitive analyses and is expedited by available Bayesian network software. The construction and analysis capabilities of DBN metamodels are illustrated with two example simulation studies.     

Bivariate distributions via a Pareto conditional distribution and a regression function

Uniqueness of specification of a bivariate distribution by a Pareto conditional and a consistent regression function is investigated. New characterizations of the Mardia bivariate Pareto distribution and the bivariate Pareto conditionals distribution are obtained.     

潜伏期变化的随机流行病模型及其贝叶斯推断

本文讨论了潜伏期和传染期均服从威布尔分布、易感性随机变化的一类随机流行病模型,并利用M CM C算法对潜伏期、传染期的参数和易感性的超参数作了贝叶期推断.这种分析方法比以往各种方法更适用于各类疾病.     

Adaptive Markov Chain Monte Carlo for Bayesian Variable Selection

We describe adaptive Markov chain Monte Carlo (MCMC) methods for sampling posterior distributions arising from Bayesian variable selection problems. Point-mass mixture priors are commonly used in Bayesian variable selection problems in regression. However, for generalized linear and nonlinear models where the conditional densities cannot be obtained directly, the resulting mixture posterior may be difficult to sample using standard MCMC methods due to multimodality. We introduce an adaptive MCMC scheme that automatically tunes the parameters of a family of mixture proposal distributions during simulation. The resulting chain adapts to sample efficiently from multimodal target distributions. For variable selection problems point-mass components are included in the mixture, and the associated weights adapt to approximate marginal posterior variable inclusion probabilities, while the remaining components approximate the posterior over nonzero values. The resulting sampler transitions efficiently between models, performing parameter estimation and variable selection simultaneously. Ergodicity and convergence are guaranteed by limiting the adaptation based on recent theoretical results. The algorithm is demonstrated on a logistic regression model, a sparse kernel regression, and a random field model from statistical biophysics; in each case the adaptive algorithm dramatically outperforms traditional MH algorithms. Supplementary materials for this article are available online.     

On regression model selection for the data with correlated errors

A class of regression model selection criteria for the data with correlated errors is proposed. The proposed class of selection criteria is an estimator of weighted prediction risk. In addition, the proposed selection criteria are the generalizations of several commonly used criteria in statistical analysis. The theoretical and asymptotic properties for the class of criteria are established. Further, in the medium-sample case, the results based on a simulation study are quite consistent with the theoretical ones. The proposed criteria perform well in the simulations. Several applications are also given for a variety of statistical models.     

Local influence in multilevel regression for growth curves

Influence analysis is important in modelling and identification of special patterns in the data. It is well established in ordinary regression. However, analogous diagnostics are generally not available for the multilevel regression model, in which estimation involves a complex iterative algorithm. This paper studies the local influence of small perturbations on the parameter estimates in the multilevel regression model with application to growth curves. The estimation is based on the iterative generalized least-squares (IGLS) method suggested by Goldstein (Biometrika 73 (1986) 43). The generalized influence function and generalized Cook statistic (Biometrika 84(1) (1997) 175) of IGLS of unknown parameters under some specific simultaneous perturbations are derived to study the joint influence of subject units on parameter estimators. The perturbation scheme is introduced through a variance–covariance matrix of error variables. A one-step approximation formula is suggested for simplifying the computations. The method is examined on growth-curve data.