High-Dimensional Mixed Graphical Models

While graphical models for continuous data (Gaussian graphical models) and discrete data (Ising models) have been extensively studied, there is little work on graphical models for datasets with both continuous and discrete variables (mixed data), which are common in many scientific applications. We propose a novel graphical model for mixed data, which is simple enough to be suitable for high-dimensional data, yet flexible enough to represent all possible graph structures. We develop a computationally efficient regression-based algorithm for fitting the model by focusing on the conditional log-likelihood of each variable given the rest. The parameters have a natural group structure, and sparsity in the fitted graph is attained by incorporating a group lasso penalty, approximated by a weighted lasso penalty for computational efficiency. We demonstrate the effectiveness of our method through an extensive simulation study and apply it to a music annotation dataset (CAL500), obtaining a sparse and interpretable graphical model relating the continuous features of the audio signal to binary variables such as genre, emotions, and usage associated with particular songs. While we focus on binary discrete variables for the main presentation, we also show that the proposed methodology can be easily extended to general discrete variables.     

Graphical Models for Ordinal Data

This article considers a graphical model for ordinal variables, where it is assumed that the data are generated by discretizing the marginal distributions of a latent multivariate Gaussian distribution. The relationships between these ordinal variables are then described by the underlying Gaussian graphical model and can be inferred by estimating the corresponding concentration matrix. Direct estimation of the model is computationally expensive, but an approximate EM-like algorithm is developed to provide an accurate estimate of the parameters at a fraction of the computational cost. Numerical evidence based on simulation studies shows the strong performance of the algorithm, which is also illustrated on datasets on movie ratings and an educational survey.     

Feature-Inclusion Stochastic Search for Gaussian Graphical Models

We describe a serial algorithm called feature-inclusion stochastic search, or FINCS, that uses online estimates of edge-inclusion probabilities to guide Bayesian model determination in Gaussian graphical models. FINCS is compared to MCMC, to Metropolis-based search methods, and to the popular lasso; it is found to be superior along a variety of dimensions, leading to better sets of discovered models, greater speed and stability, and reasonable estimates of edge-inclusion probabilities. We illustrate FINCS on an example involving mutual-fund data, where we compare the model-averaged predictive performance of models discovered with FINCS to those discovered by competing methods.     

ConvexLAR: An Extension of Least Angle Regression

The least angle regression (LAR) was proposed by Efron, Hastie, Johnstone and Tibshirani in the year 2004 for continuous model selection in linear regression. It is motivated by a geometric argument and tracks a path along which the predictors enter successively and the active predictors always maintain the same absolute correlation (angle) with the residual vector. Although it gains popularity quickly, its extensions seem rare compared to the penalty methods. In this expository article, we show that the powerful geometric idea of LAR can be generalized in a fruitful way. We propose a ConvexLAR algorithm that works for any convex loss function and naturally extends to group selection and data adaptive variable selection. After simple modification, it also yields new exact path algorithms for certain penalty methods such as a convex loss function with lasso or group lasso penalty. Variable selection in recurrent event and panel count data analysis, Ada-Boost, and Gaussian graphical model is reconsidered from the ConvexLAR angle. Supplementary materials for this article are available online.     

Selection of the Regularization Parameter in Graphical Models Using Network Characteristics

Gaussian graphical models represent the underlying graph structure of conditional dependence between random variables, which can be determined using their partial correlation or precision matrix. In a high-dimensional setting, the precision matrix is estimated using penalized likelihood by adding a penalization term, which controls the amount of sparsity in the precision matrix and totally characterizes the complexity and structure of the graph. The most commonly used penalization term is the L1 norm of the precision matrix scaled by the regularization parameter, which determines the trade-off between sparsity of the graph and fit to the data. In this article, we propose several procedures to select the regularization parameter in the estimation of graphical models that focus on recovering reliably the appropriate network structure of the graph. We conduct an extensive simulation study to show that the proposed methods produce useful results for different network topologies. The approaches are also applied in a high-dimensional case study of gene expression data with the aim to discover the genes relevant to colon cancer. Using these data, we find graph structures, which are verified to display significant biological gene associations. Supplementary material is available online.     

Sparse Partially Linear Additive Models

The generalized partially linear additive model (GPLAM) is a flexible and interpretable approach to building predictive models. It combines features in an additive manner, allowing each to have either a linear or nonlinear effect on the response. However, the choice of which features to treat as linear or nonlinear is typically assumed known. Thus, to make a GPLAM a viable approach in situations in which little is known a priori about the features, one must overcome two primary model selection challenges: deciding which features to include in the model and determining which of these features to treat nonlinearly. We introduce the sparse partially linear additive model (SPLAM), which combines model fitting and both of these model selection challenges into a single convex optimization problem. SPLAM provides a bridge between the lasso and sparse additive models. Through a statistical oracle inequality and thorough simulation, we demonstrate that SPLAM can outperform other methods across a broad spectrum of statistical regimes, including the high-dimensional (p ? N) setting. We develop efficient algorithms that are applied to real datasets with half a million samples and over 45,000 features with excellent predictive performance. Supplementary materials for this article are available online.     

Penalized Orthogonal Iteration for Sparse Estimation of Generalized Eigenvalue Problem

We propose a new algorithm for sparse estimation of eigenvectors in generalized eigenvalue problems (GEPs). The GEP arises in a number of modern data-analytic situations and statistical methods, including principal component analysis (PCA), multiclass linear discriminant analysis (LDA), canonical correlation analysis (CCA), sufficient dimension reduction (SDR), and invariant co-ordinate selection. We propose to modify the standard generalized orthogonal iteration with a sparsity-inducing penalty for the eigenvectors. To achieve this goal, we generalize the equation-solving step of orthogonal iteration to a penalized convex optimization problem. The resulting algorithm, called penalized orthogonal iteration, provides accurate estimation of the true eigenspace, when it is sparse. Also proposed is a computationally more efficient alternative, which works well for PCA and LDA problems. Numerical studies reveal that the proposed algorithms are competitive, and that our tuning procedure works well. We demonstrate applications of the proposed algorithm to obtain sparse estimates for PCA, multiclass LDA, CCA, and SDR. Supplementary materials for this article are available online.     

Structure Learning of mbox{PM}_{2.5} Distribution Using Sparse Graphical Models

We consider the structure learning problem of the mbox{PM}_{2.5} pollution data over 31 provincial capitals in China. Specifically, we make use of the graphical model tools to study the hubs and the community structures of the mbox{PM}_{2.5} pollution networks. The results show that the hubs in the mbox{PM}_{2.5}pollution networks are always seriously polluted cities, and the mbox{PM}_{2.5} pollution networks have significant community structures which consist of cities which in some sense can be regarded as blocks with similar cause of pollution. In view of the results, we suggest that the government should strengthen theeffort to treat the seriously polluted areas and western China areas. Moreover, the management of the mbox{PM}_{2.5} pollution should be region-dependent.     

Some theoretical results on the Grouped Variables Lasso

We consider the linear regression model with Gaussian error. We estimate the unknown parameters by a procedure inspired by the Group Lasso estimator introduced in [22]. We show that this estimator satisfies a sparsity inequality, i.e., a bound in terms of the number of non-zero components of the oracle regression vector. We prove that this bound is better, in some cases, than the one achieved by the Lasso and the Dantzig selector.      

����ϡ��ͼģ�͵�\mbox{PM}_{2.5}�ֲ�������ṹѧϰ

??We consider the structure learning problem of the \mbox{PM}_{2.5} pollution data over 31 provincial capitals in China. Specifically, we make use of the graphical model tools to study the hubs and the community structures of the \mbox{PM}_{2.5} pollution networks. The results show that the hubs in the \mbox{PM}_{2.5}pollution networks are always seriously polluted cities, and the \mbox{PM}_{2.5} pollution networks have significant community structures which consist of cities which in some sense can be regarded as blocks with similar cause of pollution. In view of the results, we suggest that the government should strengthen the effort to treat the seriously polluted areas and western China areas. Moreover, the management of the \mbox{PM}_{2.5} pollution should be region-dependent.     

Structure theorems over polynomial rings

Given a polynomial ring R over a field k and a finite group G, we consider a finitely generated graded RG-module S. We regard S as a kG-module and show that various conditions on S are equivalent, such as only containing finitely many isomorphism classes of indecomposable summands or satisfying a structure theorem in the sense of [D. Karagueuzian, P. Symonds, The module structure of a group action on a polynomial ring: A finiteness theorem, preprint, http://www.ma.umist.ac.uk/pas/preprints].     

Normal Linear Regression Models With Recursive Graphical Markov Structure

A multivariate normal statistical model defined by the Markov properties determined by an acyclic digraph admits a recursive factorization of its likelihood function (LF) into the product of conditional LFs, each factor having the form of a classical multivariate linear regression model (≡WMANOVA model). Here these models are extended in a natural way to normal linear regression models whose LFs continue to admit such recursive factorizations, from which maximum likelihood estimators and likelihood ratio (LR) test statistics can be derived by classical linear methods. The central distribution of the LR test statistic for testing one such multivariate normal linear regression model against another is derived, and the relation of these regression models to block-recursive normal linear systems is established. It is shown how a collection of nonnested dependent normal linear regression models (≡Wseemingly unrelated regressions) can be combined into a single multivariate normal linear regression model by imposing a parsimonious set of graphical Markov (≡Wconditional independence) restrictions.     

Modified Cross-Validation for Penalized High-Dimensional Linear Regression Models

In this article, for Lasso penalized linear regression models in high-dimensional settings, we propose a modified cross-validation (CV) method for selecting the penalty parameter. The methodology is extended to other penalties, such as Elastic Net. We conduct extensive simulation studies and real data analysis to compare the performance of the modified CV method with other methods. It is shown that the popular K-fold CV method includes many noise variables in the selected model, while the modified CV works well in a wide range of coefficient and correlation settings. Supplementary materials containing the computer code are available online.     

Learning failure-free PRISM programs

PRISM is a probabilistic logic programming formalism which allows defining a probability distribution over possible worlds. This paper investigates learning a class of generative PRISM programs known as failure-free. The aim is to learn recursive PRISM programs which can be used to model stochastic processes. These programs generalise dynamic Bayesian networks by defining a halting distribution over the generative process. Dynamic Bayesian networks model infinite stochastic processes. Sampling from infinite process can only be done by specifying the length of sequences that the process generates. In this case, only observations of a fixed length of sequences can be obtained. On the other hand, the recursive PRISM programs considered in this paper are self-terminating upon some halting conditions. Thus, they generate observations of different lengths of sequences. The direction taken by this paper is to combine ideas from inductive logic programming and learning Bayesian networks to learn PRISM programs. It builds upon the inductive logic programming approach of learning from entailment.     

基于时序图模型结构估计的股票联动研究

我国股市个股价格同时上涨或同时下跌的联动现象极为普遍,传统上使用向量自回归、协整、有向非循环图等方法主要用于少量股票或市场之间的联动性研究,不适于直接对大规模个股之间的联动关系进行研究。文章关注大规模时序图模型结构建立及估计方法,通过将ADL方法引入SPACE算法,提出了可以估计高维低样时序图模型的ADL-SPACE算法;设计模拟实验考察了算法中惩罚参数λ值的设置对于节点自回归相关性捕获的有效性;在实证研究中,文章使用了ADL-SPACE算法对个股联动研究了三方面的内容:1.基于个股联动的代表性行业之间的联动性;2.设计了我国A股市场中行业联动强度,对行业内外联动性进行综合评价和分析;3.采用一阶滞后个股基于时序图模型结果构造了投资组合,模拟显示收益预期表现良好。以上研究均表明时序SPACE图模型方法在大规模股票的联动探测中有较好的应用前景。     

机器学习中的最优化模型与对偶

数学最优化是以数学的方式来刻画和找出问题最优解的一门学科.机器学习利用数据构造预测方法,并对这些方法进行研究.介绍了机器学习中与支持向量机和稀疏重构相关的最优化模型.在此基础上,给出了三个典型最优化模型的对偶问题,并详细地讨论了对偶在求解这些问题中的应用.     

基于超球结构的支持向量机增量学习算法

本文首先分析了增量学习过程中支持向量与非支持向量的相互转化问题,而后在此基础上提出了基于超球结构的支持向量机增量学习算法。该算法主要利用超球结构,完成对增量学习中训练样本的选取,进而完成分类器的重构。实验表明,该算法比传统支持向量机增量学习算法具有更高的分类精度。     

Learning Bayesian network classifiers by risk minimization

Bayesian networks (BNs) provide a powerful graphical model for encoding the probabilistic relationships among a set of variables, and hence can naturally be used for classification. However, Bayesian network classifiers (BNCs) learned in the common way using likelihood scores usually tend to achieve only mediocre classification accuracy because these scores are less specific to classification, but rather suit a general inference problem. We propose risk minimization by cross validation (RMCV) using the 0/1 loss function, which is a classification-oriented score for unrestricted BNCs. RMCV is an extension of classification-oriented scores commonly used in learning restricted BNCs and non-BN classifiers. Using small real and synthetic problems, allowing for learning all possible graphs, we empirically demonstrate RMCV superiority to marginal and class-conditional likelihood-based scores with respect to classification accuracy. Experiments using twenty-two real-world datasets show that BNCs learned using an RMCV-based algorithm significantly outperform the naive Bayesian classifier (NBC), tree augmented NBC (TAN), and other BNCs learned using marginal or conditional likelihood scores and are on par with non-BN state of the art classifiers, such as support vector machine, neural network, and classification tree. These experiments also show that an optimized version of RMCV is faster than all unrestricted BNCs and comparable with the neural network with respect to run-time. The main conclusion from our experiments is that unrestricted BNCs, when learned properly, can be a good alternative to restricted BNCs and traditional machine-learning classifiers with respect to both accuracy and efficiency.     

基于学习效应的质量经济性动态最优化模型

随着市场竞争的日益加剧,企业越来越关注质量作为竞争武器所带来的经济效益.然而目前关于质量经济性的研究多集中于质量成本的概念和理论.为了克服质量成本模型的局限性,将产品制造过程中的学习效应引入质量经济性模型中,通过分析产品质量的提高所带来的学习效应,研究了产品质量与利润现值之间的定量关系,得出最优产品质量水平及相应的最优产量和最优价格随时间的变化轨迹,说明了在竞争条件下企业逐步改进质量降低价格的合理性.     

线性回归模型中变量选择方法综述

变量选择是统计分析与推断中的重要内容,也是当今研究的热点课题。本文系统介绍了线性回归模型中变量选择的研究概况和最新进展,并指出了有待进一步研究的问题。