Efficient strategies for deriving the subset VAR models

Algorithms for computing the subset Vector Autoregressive (VAR) models are proposed. These algorithms can be used to choose a subset of the most statistically-significant variables of a VAR model. In such cases, the selection criteria are based on the residual sum of squares or the estimated residual covariance matrix. The VAR model with zero coefficient restrictions is formulated as a Seemingly Unrelated Regressions (SUR) model. Furthermore, the SUR model is transformed into one of smaller size, where the exogenous matrices comprise columns of a triangular matrix. Efficient algorithms which exploit the common columns of the exogenous matrices, sparse structure of the variance-covariance of the disturbances and special properties of the SUR models are investigated. The main computational tool of the selection strategies is the generalized QR decomposition and its modification. This work is in part supported by the Swiss National Foundation for Research Grants 101312-100757/1, 200020-10016/1 and 101412-105978. Correspondence to: Cristian Gatu, Institut d'informatique, Université de Neuchatel, Emile-Argand 11, Case Postale 2, CH-2007 Neuchatel, Switzerland. e-mail: erricos@ucy.ac.cy     

A hybrid algorithm to optimize RBF network architecture and parameters for nonlinear time series prediction

This paper proposes a novel hybrid algorithm for automatic selection of the proper input variables, the number of hidden nodes of the radial basis function (RBF) network, and optimizing network parameters (weights, centers and widths) simultaneously. In the proposed algorithm, the inputs and the number of hidden nodes of the RBF network are represented by binary-coded strings and evolved by a genetic algorithm (GA). Simultaneously, for each chromosome with fixed inputs and number of hidden nodes, the corresponding parameters of the network are real-coded and optimized by a gradient-based fast-converging parameter estimation method. Performance of the presented hybrid approach is evaluated by several benchmark time series modeling and prediction problems. Experimental results show that the proposed approach produces parsimonious RBF networks, and obtains better modeling accuracy than some other algorithms.     

Universal Alignment Probabilities and Subset Selection for Ordinal Optimization

We examine in this paper the subset selection procedure in the context of ordinal optimization introduced in Ref. 1. Major concepts including goal softening, selection subset, alignment probability, and ordered performance curve are formally introduced. A two-parameter model is devised to calculate alignment probabilities for a wide range of cases using two different selection rules: blind pick and horse race. Our major result includes the suggestion of quantifiable subset selection sizes which are universally applicable to many simulation and modeling problems, as demonstrated by the examples in this paper.     

A hybrid spares demand forecasting method dedicated to mining industry

The paper addresses the problem of lumpy demand forecasting which is typical for spare parts. Several prediction methods are presented in the paper - traditional techniques based on time series and advanced methods which use artificial neural networks. The paper presents a new hybrid spares demand forecasting method dedicated to mining companies. The method combines information criteria, regression modeling and artificial neural networks. The paper also discusses simulation research related to efficiency assessment of the chosen variable selection methods and its application in the newly developed forecasting method. The assessment of this method is conducted by a comparison with traditional methods and is based on selected forecast errors.     

Fuzzy criteria for feature selection

The presence of less relevant or highly correlated features often decrease classification accuracy. Feature selection in which most informative variables are selected for model generation is an important step in data-driven modeling. In feature selection, one often tries to satisfy multiple criteria such as feature discriminating power, model performance or subset cardinality. Therefore, a multi-objective formulation of the feature selection problem is more appropriate. In this paper, we propose to use fuzzy criteria in feature selection by using a fuzzy decision making framework. This formulation allows for a more flexible definition of the goals in feature selection, and avoids the problem of weighting different goals is classical multi-objective optimization. The optimization problem is solved using an ant colony optimization algorithm proposed in our previous work. We illustrate the added value of the approach by applying our proposed fuzzy feature selection algorithm to eight benchmark problems.     

Sparse Vector Autoregressive Modeling

The vector autoregressive (VAR) model has been widely used for modeling temporal dependence in a multivariate time series. For large (and even moderate) dimensions, the number of the AR coefficients can be prohibitively large, resulting in noisy estimates, unstable predictions, and difficult-to-interpret temporal dependence. To overcome such drawbacks, we propose a two-stage approach for fitting sparse VAR (sVAR) models in which many of the AR coefficients are zero. The first stage selects nonzero AR coefficients based on an estimate of the partial spectral coherence (PSC) together with the use of BIC. The PSC is useful for quantifying the conditional relationship between marginal series in a multivariate process. A refinement second stage is then applied to further reduce the number of parameters. The performance of this two-stage approach is illustrated with simulation and real data examples. Supplementary materials for this article are available online.     

New constructs for the description of combinatorial optimization problems in algebraic modeling languages

Algebraic languages are at the heart of many successful optimization modeling systems, yet they have been used with only limited success for combinatorial (or discrete) optimization. We show in this paper, through a series of examples, how an algebraic modeling language might be extended to help with a greater variety of combinatorial optimization problems. We consider specifically those problems that are readily expressed as the choice of a subset from a certain set of objects, rather than as the assignment of numerical values to variables. Since there is no practicable universal algorithm for problems of this kind, we explore a hybrid approach that employs a general-purpose subset enumeration scheme together with problem-specific directives to guide an efficient search.     

Modeling and analysis using hybrid Petri nets

This paper is devoted to the use of hybrid Petri nets (PNs) for modeling and control of hybrid dynamic systems (HDS). Modeling, analysis and control of HDS attract ever more of researchers’ attention and several works have been devoted to these topics. We consider in this paper the extensions of the PN formalism (initially conceived for modeling and analysis of discrete event systems) in the direction of hybrid modeling. We present, first, the continuous PN models. These models are obtained from discrete PNs by the fluidification of the markings. They constitute the first steps in the extension of PNs toward hybrid modeling. Then, we present two hybrid PN models, which differ in the class of HDS they can deal with. The first one is used for deterministic HDS modeling, whereas the second one can deal with HDS with nondeterministic behavior.     

基于引力搜索和粒子群混合优化算法的证券投资组合问题研究

本文研究考虑交易成本的投资组合模型,分别以风险价值(VAR)和夏普比率(SR)作为投资组合的风险评价指标和效益评价指标。为有效求解此模型,本文在引力搜索和粒子群算法的基础上提出了一种混合优化算法(IN-GSA-PSO),将粒子群算法的群体最佳位置和个体最佳位置与引力搜索算法的加速度算子有机结合,使混合优化算法充分发挥单一算法的开采能力和探索能力。通过对算法相关参数的合理设置,算法能够达到全局搜索和局部搜索的平衡,快速收敛到模型的最优解。本文选取上证50股2014年下半年126个交易日的数据,运用Matlab软件进行仿真实验,实验结果显示,考虑交易成本的投资组合模型可使投资者得到更高的收益率。研究同时表明,基于PSO和GSA的混合算法在求解投资组合模型时比单一算法具有更好的性能,能够得到满意的优化结果。     

Using simulated annealing to optimize the feature selection problem in marketing applications

The feature selection (also, specification) problem is concerned with finding the most influential subset of predictors in predictive modeling from a much larger set of potential predictors that can contain hundreds of predictors. The problem belongs to the realm of combinatorial optimization where the objective is to find the subset of variables that optimize the value of some goodness of fit function. Due to the dimensionality of the problem, the feature selection problem belongs to the group of NP-hard problems. Most of the available predictors are noisy or redundant and add very little, if any, to the prediction power of the model. Using all the predictors in the model often results in strong over-fitting and very poor predictions. Constructing a prediction model by checking out all possible subsets is impractical due to computational volume. Looking on the contribution of each predictor separately is not accurate because it ignores the inter-correlations between predictors. As a result, no analytic solution is available for the feature selection problem, requiring that one resorts to heuristics. In this paper we employ the simulated annealing (SA) approach, which is one of the leading stochastic search methods, for specifying a large-scale linear regression model. The SA results are compared to the results of the more common stepwise regression (SWR) approach for model specification. The models are applied on realistic data sets in database marketing. We also use simulated data sets to investigate what data characteristics make the SWR approach equivalent to the supposedly more superior SA approach.     

Identification of fuzzy relation models using hierarchical fair competition-based parallel genetic algorithms and information granulation

The paper is concerned with a hybrid optimization of fuzzy inference systems based on hierarchical fair competition-based parallel genetic algorithms (HFCGA) and information granulation. The process of information granulation is realized with the aid of the C-Means clustering. HFCGA being a multi-population based parallel genetic algorithms (PGA) is exploited here to realize structure optimization and carry out parameter estimation of the fuzzy models. The HFCGA becomes helpful in the context of fuzzy models as it restricts a premature convergence encountered quite often in optimization problems. It concerns a set of parameters of the model including among others the number of input variables to be used, a specific subset of input variables, and the number of membership functions. In the hybrid optimization process, two general optimization mechanisms are explored. The structural development of the fuzzy model is realized via the HFCGA optimization and C-Means, whereas to deal with the parametric optimization we proceed with a standard least square method and the use of the HFCGA technique. A suite of comparative studies demonstrates that the proposed algorithm leads to the models whose performance is superior in comparison with some other constructs commonly used in fuzzy modeling.     

基于VAR风险指标的投资组合模糊优化

在二目标有价证券选择基础上 ,引入目前流行的风险指标 VAR,以收益率与风险损失为目标 ,将模糊概念运用于有价证券组合选择 ,按投资者给定的期望目标及容差 ,讨论了 S型隶属函数模型 .通过 VAR的给定 ,将投资者所能承受的最大损失锁定 ,更好地反映出投资者对目标值的取值意图 .依据深圳股票市场9只股票收益率数据 ,采用进化规划进行优化计算 ,并验证模型的有效性 .     

Parallel Variational Bayes for Large Datasets With an Application to Generalized Linear Mixed Models

The article develops a hybrid variational Bayes (VB) algorithm that combines the mean-field and stochastic linear regression fixed-form VB methods. The new estimation algorithm can be used to approximate any posterior without relying on conjugate priors. We propose a divide and recombine strategy for the analysis of large datasets, which partitions a large dataset into smaller subsets and then combines the variational distributions that have been learned in parallel on each separate subset using the hybrid VB algorithm. We also describe an efficient model selection strategy using cross-validation, which is straightforward to implement as a by-product of the parallel run. The proposed method is applied to fitting generalized linear mixed models. The computational efficiency of the parallel and hybrid VB algorithm is demonstrated on several simulated and real datasets. Supplementary material for this article is available online.     

Portfolio selection based on fuzzy cross-entropy

In this paper, the Kapur cross-entropy minimization model for portfolio selection problem is discussed under fuzzy environment, which minimizes the divergence of the fuzzy investment return from a priori one. First, three mathematical models are proposed by defining divergence as cross-entropy, average return as expected value and risk as variance, semivariance and chance of bad outcome, respectively. In order to solve these models under fuzzy environment, a hybrid intelligent algorithm is designed by integrating numerical integration, fuzzy simulation and genetic algorithm. Finally, several numerical examples are given to illustrate the modeling idea and the effectiveness of the proposed algorithm.     

Bayesian variable selection with sparse and correlation priors for high-dimensional data analysis

The main challenge in working with gene expression microarrays is that the sample size is small compared to the large number of variables (genes). In many studies, the main focus is on finding a small subset of the genes, which are the most important ones for differentiating between different types of cancer, for simpler and cheaper diagnostic arrays. In this paper, a sparse Bayesian variable selection method in probit model is proposed for gene selection and classification. We assign a sparse prior for regression parameters and perform variable selection by indexing the covariates of the model with a binary vector. The correlation prior for the binary vector assigned in this paper is able to distinguish models with the same size. The performance of the proposed method is demonstrated with one simulated data and two well known real data sets, and the results show that our method is comparable with other existing methods in variable selection and classification.     

A method for selecting a subset of alternatives for future decision making

In this paper we present an algorithm for finding a subset from a large number of alternatives. The criterion for selecting this subset is based on the assumption that ultimately one alternative will be chosen and implemented from this subset. Some areas of application for the subset selection techniques are presented. Extensions of this research are suggested.     

Potential inventory cost reductions using advanced time series forecasting techniques

This paper compares demand forecasts computed using the time series forecasting techniques of vector autoregression (VAR) and Bayesian VAR (BVAR) with forecasts computed using exponential smoothing and seasonal decomposition. These forecasts for three demand data series were used to determine three inventory management policies for each time series. The inventory costs associated with each of these policies were used as a further basis for comparison of the forecasting techniques. The results show that the BVAR technique, which uses mixed estimation, is particularly useful in reducing inventory costs in cases where the limited historical data offer little useful information for forecasting. The BVAR technique was effective in improving forecast accuracy and reducing inventory costs in two of the three cases tested. In the third case, unrestricted VAR and exponential smoothing produced the lowest experimental forecast errors and computed inventory costs. Furthermore, this research illustrates that improvements in demand forecasting can provide better cost reductions than relying on stochastic inventory models to provide cost reductions.     

Estimation de la matrice de variance asymptotique des estimateurs du QMV de modèles ARMA faibles multivariés

In this Note, we consider the problems of estimating the asymptotic variance of the quasi-maximum likelihood estimator (QMLE) of vector autoregressive moving-average (VARMA) models under the assumption that the errors are uncorrelated but not necessarily independent (i.e. weak VARMA). We first give expressions for the derivatives of the VARMA residuals in terms of the parameters of the models. Secondly we give an explicit expression of the asymptotic variance of the QMLE, in terms of the VAR and MA polynomials, and of the second- and fourth-order structure of the noise. We deduce a consistent estimator of the asymptotic variance of the QMLE.     

分位数变系数模型基于核光滑的变量选择

分位数变系数模型是一种稳健的非参数建模方法.使用变系数模型分析数据时,一个自然的问题是如何同时选择重要变量和从重要变量中识别常数效应变量.本文基于分位数方法研究具有稳健和有效性的估计和变量选择程序.利用局部光滑和自适应组变量选择方法,并对分位数损失函数施加双惩罚,我们获得了惩罚估计.通过BIC准则合适地选择调节参数,提出的变量选择方法具有oracle理论性质,并通过模拟研究和脂肪实例数据分析来说明新方法的有用性.数值结果表明,在不需要知道关于变量和误差分布的任何信息前提下,本文提出的方法能够识别不重要变量同时能区分出常数效应变量.     

Parameter estimation and model selection for mixtures of truncated exponentials

Bayesian networks with mixtures of truncated exponentials (MTEs) support efficient inference algorithms and provide a flexible way of modeling hybrid domains (domains containing both discrete and continuous variables). On the other hand, estimating an MTE from data has turned out to be a difficult task, and most prevalent learning methods treat parameter estimation as a regression problem. The drawback of this approach is that by not directly attempting to find the parameter estimates that maximize the likelihood, there is no principled way of performing subsequent model selection using those parameter estimates. In this paper we describe an estimation method that directly aims at learning the parameters of an MTE potential following a maximum likelihood approach. Empirical results demonstrate that the proposed method yields significantly better likelihood results than existing regression-based methods. We also show how model selection, which in the case of univariate MTEs amounts to partitioning the domain and selecting the number of exponential terms, can be performed using the BIC score.