Bayesian Variable and Transformation Selection in Linear Regression

This article suggests a method for variable and transformation selection based on posterior probabilities. Our approach allows for consideration of all possible combinations of untransformed and transformed predictors along with transformed and untransformed versions of the response. To transform the predictors in the model, we use a change-point model, or “change-point transformation,” which can yield more interpretable models and transformations than the standard Box–Tidwell approach. We also address the problem of model uncertainty in the selection of models. By averaging over models, we account for the uncertainty inherent in inference based on a single model chosen from the set of models under consideration. We use a Markov chain Monte Carlo model composition (MC³) method which allows us to average over linear regression models when the space of models under consideration is very large. This considers the selection of variables and transformations at the same time. In an example, we show that model averaging improves predictive performance as compared with any single model that might reasonably be selected, both in terms of overall predictive score and of the coverage of prediction intervals. Software to apply the proposed methodology is available via StatLib.     

广义变系数模型的Bayesian B样条估计

提出了广义变系数模型函数系数的一种新的估计方法．我们用B样条函数逼近函数系数,不具体选择节点的个数,而是节点个数取均匀的无信息先验,样条函数系数取正态先验,用Bayesian模型平均的方法估计各个函数系数．这种估计方法一个主要特点是允许各个函数系数所需节点个数的后验分布不同,因此允许不同函数系数使用不同的光滑参数．另外,本文还给出了Bayesian B样条估计的计算方法,并通过模拟例子,说明广义变系数模型的函数系数可以由Bayesian B样条估计方法得到很好的估计．     

基于最小二乘近似的模型平均方法

本文提出基于最小二乘近似的模型平均方法.该方法可用于线性模型、广义线性模型和分位数回归等各种常用模型.特别地,经典的Mallows模型平均方法是该方法的特例.现存的模型平均文献中,渐近分布的证明一般需要局部误设定假设,所得的极限分布的形式也比较复杂.本文将在不使用局部误设定假设的情形下证明该方法的渐近正态性.另外,本文...     

The non-Markovian nature of nested logit choice

Discrete choice models are widely used for understanding how customers choose between a variety of substitutable goods. We investigate the relationship between two well studied choice models, the Nested Logit (NL) model and the Markov choice model. Both models generalize the classic Multinomial Logit model and admit tractable algorithms for assortment optimization. Previous evidence indicates that the NL model may be well approximated by, or be a special case of, the Markov model. We establish that the Nested Logit model, in general, cannot be represented by a Markov model. Further, we show that there exists a family of instances of the NL model where the choice probabilities cannot be approximated to within a constant error by any Markov choice model.     

Two-parameter spectral averaging and localization for non-monotonic random Schrödinger operators

We prove exponential localization at all energies for two types of one-dimensional random Schrödinger operators: the Poisson model and the random displacement model. As opposed to Anderson-type models, these operators are not monotonic in the random parameters. Therefore the classical one-parameter version of spectral averaging, as used in localization proofs for Anderson models, breaks down. We use the new method of two-parameter spectral averaging and apply it to the Poisson as well as the displacement case. In addition, we apply results from inverse spectral theory, which show that two-parameter spectral averaging works for sufficiently many energies (all but a discrete set) to conclude localization at all energies.

     

Integer programming models for the q-mode problem

The q-mode problem is a combinatorial optimization problem that requires partitioning of objects into clusters. We discuss theoretical properties of an existing mixed integer programming (MIP) model for this problem and offer alternative models and enhancements. Through a comprehensive experiment we investigate computational properties of these MIP models. This experiment reveals that, in practice, the MIP approach is more effective for instances containing strong natural clusters and it is not as effective for instances containing weak natural clusters. The experiment also reveals that one of the MIP models that we propose is more effective than the other models for solving larger instances of the problem.     

An effective recursive partitioning approach for the packing of identical rectangles in a rectangle

In this work, we deal with the problem of packing (orthogonally and without overlapping) identical rectangles in a rectangle. This problem appears in different logistics settings, such as the loading of boxes on pallets, the arrangements of pallets in trucks and the stowing of cargo in ships. We present a recursive partitioning approach combining improved versions of a recursive five-block heuristic and an L-approach for packing rectangles into larger rectangles and L-shaped pieces. The combined approach is able to rapidly find the optimal solutions of all instances of the pallet loading problem sets Cover I and II (more than 50?000 instances). It is also effective for solving the instances of problem set Cover III (almost 100?000 instances) and practical examples of a woodpulp stowage problem, if compared to other methods from the literature. Some theoretical results are also discussed and, based on them, efficient computer implementations are introduced. The computer implementation and the data sets are available for benchmarking purposes.     

Imprecise probability models for learning multinomial distributions from data. Applications to learning credal networks

This paper considers the problem of learning multinomial distributions from a sample of independent observations. The Bayesian approach usually assumes a prior Dirichlet distribution about the probabilities of the different possible values. However, there is no consensus on the parameters of this Dirichlet distribution. Here, it will be shown that this is not a simple problem, providing examples in which different selection criteria are reasonable. To solve it the Imprecise Dirichlet Model (IDM) was introduced. But this model has important drawbacks, as the problems associated to learning from indirect observations. As an alternative approach, the Imprecise Sample Size Dirichlet Model (ISSDM) is introduced and its properties are studied. The prior distribution over the parameters of a multinomial distribution is the basis to learn Bayesian networks using Bayesian scores. Here, we will show that the ISSDM can be used to learn imprecise Bayesian networks, also called credal networks when all the distributions share a common graphical structure. Some experiments are reported on the use of the ISSDM to learn the structure of a graphical model and to build supervised classifiers.     

Robust Model Averaging Method Based on LOF Algorithm

Model averaging is a good alternative to model selection, which can deal with the uncertainty from model selection process and make full use of the information from various candidate models. However, most of the existing model averaging criteria do not consider the influence of outliers on the estimation procedures. The purpose of this paper is to develop a robust model averaging approach based on the local outlier factor (LOF) algorithm which can downweight the outliers in the covariates. Asymptotic optimality of the proposed robust model averaging estimator is derived under some regularity conditions. Further, we prove the consistency of the LOF-based weight estimator tending to the theoretically optimal weight vector. Numerical studies including Monte Carlo simulations and a real data example are provided to illustrate our proposed methodology.     

Comments on “Imprecise probability models for learning multinomial distributions from data. Applications to learning credal networks” by Andrés R. Masegosa and Serafín Moral

We briefly overview the problem of learning probabilities from data using imprecise probability models that express very weak prior beliefs. Then we comment on the new contributions to this question given in the paper by Masegosa and Moral and provide some insights about the performance of their models in data mining experiments of classification.     

“新常态”下的中国天然气消费分析及预测

作为一种优质、高效的绿色能源,天然气在中国能源结构中所占比重逐渐增加。但可再生能源的崛起使得天然气成为过渡能源的选择,天然气消费量的增长趋势不明晰,因此相关企业及部门需要合理、准确的天然气需求预测模型为未来的工作提供指导性信息。基于此,本文首先从经济水平、产业结构、能源结构、天然气价格等方面选取影响天然气消费的12个因素。其次,运用贝叶斯模型平均(BMA)法构建了一个包含相关文献中常用的6个影响因素的基准模型,针对该模型,围绕影响天然气消费量的各种因素,以逐个添加的方式建立对比模型,从备选模型中选出预测精度最高的对未来天然气消费量进行预测。最后,将BMA模型与ARIMA模型、ETS模型、BVAR模型、逐步回归模型以及等权重加权平均模型的预测精度进行对比。结果表明,最优的BMA模型包含了涉及经济水平、产业结构、能源结构、人口因素、天然气价格、天然气供给六个方面9个影响因素,其预测精度优于对比预测模型,且该模型预测 2022年天然气消费量将达到3254.153亿立方米,年均增长率为8%。     

Formulations for a problem of petroleum transportation

Oil tankers play a fundamental role in every offshore petroleum supply chain and due to its high price, it is essential to optimize its use. Since this optimization requires handling detailed operational aspects, complete optimization models are typically intractable. Thus, a usual approach is to solve a tactical level model prior to optimize the operational details. In this case, it is desirable that tactical models are as precise as possible to avoid too severe adjustments in the next optimization level. In this paper, we study tactical models for a crude oil transportation problem by tankers. We did our work on the top of a previous paper found in the literature. The previous model considers inventory capacities and discrete lot sizes to be transported, aiming to meet given demands over a finite time horizon. We compare several formulations for this model using 50 instances from the literature and proposing 25 new harder ones. A column generation-based heuristic is also proposed to find good feasible solutions with less computational burden than the heuristics of the commercial solver used.     

响应变量删失情况下线性模型的FIC模型选择和模型平均

本文给出了响应变量随机右删失情况下线性模型的FIC (focused information criterion) 模型选择方法和光滑FIC 模型平均估计方法, 证明了兴趣参数的FIC 模型选择估计和光滑FIC 模型平均估计的渐近正态性, 通过随机模拟研究了估计的有限样本性质, 模拟结果显示, 从均方误差和一定置信水平置信区间的经验覆盖概率看, 兴趣参数的光滑FIC 模型平均估计均优于FIC, AIC (Akaikeinformation criterion) 和BIC (Bayesian information citerion) 等模型选择估计; 而FIC 模型选择估计与AIC 和BIC 等模型选择估计相比, 也表现出了一定的优越性. 通过分析原发性胆汁性肝硬化数据集, 说明了本文方法在实际问题中的应用.     

Epistemic irrelevance in credal nets: The case of imprecise Markov trees

We focus on credal nets, which are graphical models that generalise Bayesian nets to imprecise probability. We replace the notion of strong independence commonly used in credal nets with the weaker notion of epistemic irrelevance, which is arguably more suited for a behavioural theory of probability. Focusing on directed trees, we show how to combine the given local uncertainty models in the nodes of the graph into a global model, and we use this to construct and justify an exact message-passing algorithm that computes updated beliefs for a variable in the tree. The algorithm, which is linear in the number of nodes, is formulated entirely in terms of coherent lower previsions, and is shown to satisfy a number of rationality requirements. We supply examples of the algorithm’s operation, and report an application to on-line character recognition that illustrates the advantages of our approach for prediction. We comment on the perspectives, opened by the availability, for the first time, of a truly efficient algorithm based on epistemic irrelevance.     

Lagrangian relaxation guided problem space search heuristics for generalized assignment problems

We develop and test a heuristic based on Lagrangian relaxation and problem space search to solve the generalized assignment problem (GAP). The heuristic combines the iterative search capability of subgradient optimization used to solve the Lagrangian relaxation of the GAP formulation and the perturbation scheme of problem space search to obtain high-quality solutions to the GAP. We test the heuristic using different upper bound generation routines developed within the overall mechanism. Using the existing problem data sets of various levels of difficulty and sizes, including the challenging largest instances, we observe that the heuristic with a specific version of the upper bound routine works well on most of the benchmark instances known and provides high-quality solutions quickly. An advantage of the approach is its generic nature, simplicity, and implementation flexibility.     

Inference of an oscillating model for the yeast cell cycle

High-throughput techniques allow measurement of hundreds of cell components simultaneously. The inference of interactions between cell components from these experimental data facilitates the understanding of complex regulatory processes. Differential equations have been established to model the dynamic behavior of these regulatory networks quantitatively. Usually traditional regression methods for estimating model parameters fail in this setting, since they overfit the data. This is even the case, if the focus is on modeling subnetworks of, at most, a few tens of components. In a Bayesian learning approach, this problem is avoided by a restriction of the search space with prior probability distributions over model parameters.This paper combines both differential equation models and a Bayesian approach. We model the periodic behavior of proteins involved in the cell cycle of the budding yeast Saccharomyces cerevisiae, with differential equations, which are based on chemical reaction kinetics. One property of these systems is that they usually converge to a steady state, and lots of efforts have been made to explain the observed periodic behavior. We introduce an approach to infer an oscillating network from experimental data. First, an oscillating core network is learned. This is extended by further components by using a Bayesian approach in a second step. A specifically designed hierarchical prior distribution over interaction strengths prevents overfitting, and drives the solutions to sparse networks with only a few significant interactions.We apply our method to a simulated and a real world dataset and reveal main regulatory interactions. Moreover, we are able to reconstruct the dynamic behavior of the network.     

Generalized mirror averaging and <Emphasis Type="Italic">D</Emphasis>-convex aggregation

We study the problem of aggregation of estimators. Given a collection of M different estimators, we construct a new estimator, called aggregate, which is nearly as good as the best linear combination over an l ₁-ball of ℝ^M of the initial estimators. The aggregate is obtained by a particular version of the mirror averaging algorithm. We show that our aggregation procedure statisfies sharp oracle inequalities under general assumptions. Then we apply these results to a new aggregation problem: D-convex aggregation. Finally we implement our procedure in a Gaussian regression model with random design and we prove its optimality in a minimax sense up to a logarithmic factor.      

Frequentist model averaging for linear mixed-effects models

Linear mixed-effects models are a powerful tool for the analysis of longitudinal data. The aim of this paper is to study model averaging for linear mixed-effects models. The asymptotic distribution of the frequentist model average estimator is derived, and a confidence interval procedure with an actual coverage probability that tends to the nominal level in large samples is developed. The two confidence intervals based on the model averaging and based on the full model are shown to be asymptotically equivalent. A simulation study shows good finite sample performance of the model average estimators.     

The table placement problem: a research challenge at the EWI 2007

The table placement problem consists in deciding how to seat the participants attending a social lunch or dinner so that the total social benefit of the event is maximum. Four different approaches are presented: a linear model, a bin-packing-based-approach, a quadratic assignment problem, and a greedy heuristic. The different formulations are computationally compared over a set of artificial instances and on the real data for the EURO Winter Institute 2007 Gala dinner.     

A generalization of the set averaging theorem

We consider the possibility of generalizing the averaging theorem from the case of sets from n-dimensional Euclidean space to the case of sets from Banach spaces. The result is a cornerstone for constructing the theory of the Riemann integral for non-convex-valued multivalued mappings and for proving the convexity of this multivalued integral. We obtain a generalization of the averaging theorem to the case of sets from uniformly smooth Banach spaces as well as some corollaries.