Gradient learning in a classification setting by gradient descent

Learning gradients is one approach for variable selection and feature covariation estimation when dealing with large data of many variables or coordinates. In a classification setting involving a convex loss function, a possible algorithm for gradient learning is implemented by solving convex quadratic programming optimization problems induced by regularization schemes in reproducing kernel Hilbert spaces. The complexity for such an algorithm might be very high when the number of variables or samples is huge. We introduce a gradient descent algorithm for gradient learning in classification. The implementation of this algorithm is simple and its convergence is elegantly studied. Explicit learning rates are presented in terms of the regularization parameter and the step size. Deep analysis for approximation by reproducing kernel Hilbert spaces under some mild conditions on the probability measure for sampling allows us to deal with a general class of convex loss functions.     

Multiobjective expected value model for portfolio selection in fuzzy environment

In this paper, we propose a credibilistic framework for portfolio selection problem using an expected value multiobjective model with fuzzy parameters. We consider short term return, long term return, risk and liquidity as key financial criteria. A solution procedure comprising fuzzy goal programming and fuzzy simulation based real-coded genetic algorithm is developed to solve the model. The proposed solution approach is considered advantageous particularly for the cases where the fuzzy parameters of the problem may assume any general functional form. An empirical study is included to illustrate the usefulness of the proposed model and solution approach in real-world applications of portfolio selection.     

A mixed R&D projects and securities portfolio selection model

The business environment is full of uncertainty. Allocating the wealth among various asset classes may lower the risk of overall portfolio and increase the potential for more benefit over the long term. In this paper, we propose a mixed single-stage R&D projects and multi-stage securities portfolio selection model. Specifically, we present a bi-objective mixed-integer stochastic programming model. Moreover, we use semi-absolute deviation risk functions to measure the risk of mixed asset portfolio. Based on the idea of moments approximation method via linear programming, we propose a scenario generation approach for the mixed single-stage R&D projects and multi-stage securities portfolio selection problem. The bi-objective mixed-integer stochastic programming problem can be solved by transforming it into a single objective mixed-integer stochastic programming problem. A numerical example is given to illustrate the behavior of the proposed mixed single stage R&D projects and multi-stage securities portfolio selection model.     

On the inventory model with variable lead time and price–quantity discount

In this paper, we propose a mixed integer optimization approach for solving the inventory problem with variable lead time, crashing cost, and price–quantity discount. A linear programming relaxation based on piecewise linearization techniques is derived for the problem. It first converts non-linear terms into the sum of absolute terms, which are then linearized by goal programming techniques and linearization approaches. The proposed method can eliminate the complicated multiple-step solution process used in the traditional inventory models. In addition, the proposed model allows constraints to be added by the inventory decision-maker as deemed appropriate in real-world situations.     

A new model and a hyper-heuristic approach for two-dimensional shelf space allocation

In this paper, we propose a two-dimensional shelf space allocation model. The second dimension stems from the height of the shelf. This results in an integer nonlinear programming model with a complex form of objective function. We propose a multiple neighborhood approach which is a hybridization of a simulated annealing algorithm with a hyper-heuristic learning mechanism. Experiments based on empirical data from both real-world and artificial instances show that the shelf space utilization and the resulting sales can be greatly improved when compared with a gradient method. Sensitivity analysis on the input parameters and the shelf space show the benefits of the proposed algorithm both in sales and in robustness.     

Infinite Dirichlet mixture models learning via expectation propagation

In this article, we propose a novel Bayesian nonparametric clustering algorithm based on a Dirichlet process mixture of Dirichlet distributions which have been shown to be very flexible for modeling proportional data. The idea is to let the number of mixture components increases as new data to cluster arrive in such a manner that the model selection problem (i.e. determination of the number of clusters) can be answered without recourse to classic selection criteria. Thus, the proposed model can be considered as an infinite Dirichlet mixture model. An expectation propagation inference framework is developed to learn this model by obtaining a full posterior distribution on its parameters. Within this learning framework, the model complexity and all the involved parameters are evaluated simultaneously. To show the practical relevance and efficiency of our model, we perform a detailed analysis using extensive simulations based on both synthetic and real data. In particular, real data are generated from three challenging applications namely images categorization, anomaly intrusion detection and videos summarization.     

SICOpt: Solution Approach for Nonlinear Integer Stochastic Programming Problems

We propose a novel solution approach for the class of two-stage nonlinear integer stochastic programming models. These problems are characterized by large scale dimensions, as the number of constraints and variables depend on the number of realizations (scenarios) used to capture the underlying distributions of the random data. In addition, the integrality constraints on the decision variables make the solution process even much more difficult preventing the application of general purpose solvers. The proposed solution approach integrates the branch-and-bound framework with the interior point method. The main advantage of this choice is the effective exploitation of the specific structure exhibited by the different subproblems at each node of the search tree. A specifically designed warm start procedure and an early branching technique improve the overall efficiency. Our contribution is well founded from a theoretical point of view and is characterized by good computational efficiency, without any loss in terms of effectiveness. Some preliminary numerical results, obtained by solving a challenging real-life problem, prove the robustness and the efficiency of the proposed approach.     

An efficient second-order cone programming approach for optimal selection in tree breeding

It is common in forest tree breeding that selection of populations must consider conservation of genetic diversity, while at the same time attempting to maximize response to selection. To optimize selection in these situations, the constraint on genetic diversity can be mathematically described with the numerator relationship matrix as a quadratic constraint. Pong-Wong and Woolliams formulated the optimal selection problem using semidefinite programming (SDP). Their SDP approach gave an accurate optimal value, but required rather long computation time. In this paper, we propose an second-order cone programming (SOCP) approach to reduce the heavy computation cost. First, we demonstrate that a simple SOCP formulation achieves the same numerical solution as the SDP approach. A simple SOCP formulation is, however, not much more efficient compared to the SDP approach, so we focused on the sparsity structure of the numerator relationship matrix, and we develop a more efficient SOCP formulation using Henderson’s algorithm. Numerical results show that the proposed formulation, which we call a compact SOCP, greatly reduced computation time. In a case study, an optimal selection problem that demanded 39,200 s under the SDP approach was solved in less than 2 s by the compact SOCP formulation. The proposed approach is now available as a part of the software package OPSEL.     

Discovering model structure for partially linear models

Partially linear models (PLMs) have been widely used in statistical modeling, where prior knowledge is often required on which variables have linear or nonlinear effects in the PLMs. In this paper, we propose a model-free structure selection method for the PLMs, which aims to discover the model structure in the PLMs through automatically identifying variables that have linear or nonlinear effects on the response. The proposed method is formulated in a framework of gradient learning, equipped with a flexible reproducing kernel Hilbert space. The resultant optimization task is solved by an efficient proximal gradient descent algorithm. More importantly, the asymptotic estimation and selection consistencies of the proposed method are established without specifying any explicit model assumption, which assure that the true model structure in the PLMs can be correctly identified with high probability. The effectiveness of the proposed method is also supported by a variety of simulated and real-life examples.

     

Methods of critical value reduction for type-2 fuzzy variables and their applications

A type-2 fuzzy variable is a map from a fuzzy possibility space to the real number space; it is an appropriate tool for describing type-2 fuzziness. This paper first presents three kinds of critical values (CVs) for a regular fuzzy variable (RFV), and proposes three novel methods of reduction for a type-2 fuzzy variable. Secondly, this paper applies the reduction methods to data envelopment analysis (DEA) models with type-2 fuzzy inputs and outputs, and develops a new class of generalized credibility DEA models. According to the properties of generalized credibility, when the inputs and outputs are mutually independent type-2 triangular fuzzy variables, we can turn the proposed fuzzy DEA model into its equivalent parametric programming problem, in which the parameters can be used to characterize the degree of uncertainty about type-2 fuzziness. For any given parameters, the parametric programming model becomes a linear programming one that can be solved using standard optimization solvers. Finally, one numerical example is provided to illustrate the modeling idea and the efficiency of the proposed DEA model.     

Using metaheuristic compromise programming for the solution of multiple-objective scheduling problems

In this paper, we propose a generic approach to find compromise solutions for multiple-objective scheduling problems using metaheuristics. As an illustration, we present a new hybrid tabu search/variable neighbourhood search application of this approach for the solution of a bi-objective scheduling problem. Through numerical experiments we demonstrate its efficiency and effectiveness. We have confirmed that compromise programming with the tabu-VNS metaheuristic generates solutions that approach those of the known reference sets.     

A policy gradient method for semi-Markov decision processes with application to call admission control

Solving a semi-Markov decision process (SMDP) using value or policy iteration requires precise knowledge of the probabilistic model and suffers from the curse of dimensionality. To overcome these limitations, we present a reinforcement learning approach where one optimizes the SMDP performance criterion with respect to a family of parameterised policies. We propose an online algorithm that simultaneously estimates the gradient of the performance criterion and optimises it using stochastic approximation. We apply our algorithm to call admission control. Our proposed policy gradient SMDP algorithm and its application to admission control is novel.     

A cutting plane algorithm for MV portfolio selection model

This paper deals with a portfolio selection problem with fuzzy return rates. A possibilistic mean variance (FMVC) portfolio selection model was proposed. The possibilistic programming problem can be transformed into a linear optimal problem with an additional quadratic constraint by possibilistic theory. For such problems there are no special standard algorithms. We propose a cutting plane algorithm to solve (FMVC). The nonlinear programming problem can be solved by sequence linear programming problem. A numerical example is given to illustrate the behavior of the proposed model and algorithm.     

Nonparametric variable selection and its application to additive models

Variable selection for multivariate nonparametric regression models usually involves parameterized approximation for nonparametric functions in the objective function. However, this parameterized approximation often increases the number of parameters significantly, leading to the “curse of dimensionality” and inaccurate estimation. In this paper, we propose a novel and easily implemented approach to do variable selection in nonparametric models without parameterized approximation, enabling selection consistency to be achieved. The proposed method is applied to do variable selection for additive models. A two-stage procedure with selection and adaptive estimation is proposed, and the properties of this method are investigated. This two-stage algorithm is adaptive to the smoothness of the underlying components, and the estimation consistency can reach a parametric rate if the underlying model is really parametric. Simulation studies are conducted to examine the performance of the proposed method. Furthermore, a real data example is analyzed for illustration.

     

部分线性单指标模型的复合分位数回归及变量选择

本文提出复合最小化平均分位数损失估计方法 (composite minimizing average check loss estimation,CMACLE)用于实现部分线性单指标模型(partial linear single-index models,PLSIM)的复合分位数回归(composite quantile regression,CQR).首先基于高维核函数构造参数部分的复合分位数回归意义下的相合估计,在此相合估计的基础上,通过采用指标核函数进一步得到参数和非参数函数的可达最优收敛速度的估计,并建立所得估计的渐近正态性,比较PLSIM的CQR估计和最小平均方差估计(MAVE)的相对渐近效率.进一步地,本文提出CQR框架下PLSIM的变量选择方法,证明所提变量选择方法的oracle性质.随机模拟和实例分析验证了所提方法在有限样本时的表现,证实了所提方法的优良性.     

Robust scheduling of parallel machines with sequence-dependent set-up costs

In this paper we propose a robust approach for solving the scheduling problem of parallel machines with sequence-dependent set-up costs. In the literature, several mathematical models and solution methods have been proposed to solve such scheduling problems, but most of which are based on the strong assumption that input data are known in a deterministic way. In this paper, a fuzzy mathematical programming model is formulated by taking into account the uncertainty in processing times to provide the optimal solution as a trade-off between total set-up cost and robustness in demand satisfaction. The proposed approach requires the solution of a non-linear mixed integer programming (NLMIP), that can be formulated as an equivalent mixed integer linear programming (MILP) model. The resulting MILP model in real applications could be intractable due to its NP-hardness. Therefore, we propose a solution method technique, based on the solution of an approximated model, whose dimension is remarkably reduced with respect to the original counterpart. Numerical experiments conducted on the basis of data taken from a real application show that the average deviation of the reduced model solution over the optimum is less than 1.5%.     

BDD-based optimization for the quadratic stable set problem

The quadratic stable set problem (QSSP) is a natural extension of the well-known maximum stable set problem. The QSSP is NP-hard and can be formulated as a binary quadratic program, which makes it an interesting case study to be tackled from different optimization paradigms. In this paper, we propose a novel representation for the QSSP through binary decision diagrams (BDDs) and adapt a hybrid optimization approach which integrates BDDs and mixed-integer programming (MIP) for solving the QSSP. The exact framework highlights the modeling flexibility offered through decision diagrams to handle nonlinear problems. In addition, the hybrid approach leverages two different representations by exploring, in a complementary way, the solution space with BDD and MIP technologies. Machine learning then becomes a valuable component within the method to guide the search mechanisms. In the numerical experiments, the hybrid approach shows to be superior, by at least one order of magnitude, than two leading commercial MIP solvers with quadratic programming capabilities and a semidefinite-based branch-and-bound solver.     

An Inverse Reliability-based Approach for Designing under Uncertainty with Application to Robust Piston Design

In this work, we propose an optimization framework for designing under uncertainty that considers both robustness and reliability issues. This approach is generic enough to be applicable to engineering design problems involving nonconvex objective and constraint functions defined in terms of random variables that follow any distribution. The problem formulation employs an Inverse Reliability Strategy that uses percentile performance to address both robustness objectives and reliability constraints. Robustness is achieved through a design objective that evaluates performance variation as a percentile difference between the right and left trails of the specified goals. Reliability requirements are formulated as Inverse Reliability constraints that are based on equivalent percentile performance levels. The general proposed approach first approximates the formulated problem via a Gaussian Kriging model. This is then used to evaluate the percentile performance characteristics of the different measures inherent in the problem formulation for various design variable settings via a Most Probable Point of Inverse Reliability search algorithm. By using these percentile evaluations in concert with the response surface methodology, a polynomial programming approximation is generated. The resulting problem formulation is finally solved to global optimality using the Reformulation–Linearization Technique (RLT) approach. We demonstrate this overall proposed approach by applying it to solve the problem of reducing piston slap, an undesirable engine noise due to the secondary motion of a piston within a cylinder.     

PSO-based and SA-based metaheuristics for bilinear programming problems: an application to the pooling problem

Bilinear programming problems (BLP) are subsets of nonconvex quadratic programs and can be classified as strongly NP-Hard. The exact methods to solve the BLPs are inefficient for large instances and only a few heuristic methods exist. In this study, we propose two metaheuristic methods, one is based on particle swarm optimization (PSO) and the other is based on simulated annealing (SA). Both of the proposed approaches take advantage of the bilinear structure of the problem. For the PSO-based method, a search variable, which is selected among the variable sets causing bilinearity, is subjected to particle swarm optimization. The SA-based procedure incorporates a variable neighborhood scheme. The pooling problem, which has several application areas in chemical industry and formulated as a BLP, is selected as a test bed to analyze the performances. Extensive experiments are conducted and they indicate the success of the proposed solution methods.