Parametric vector optimization

In this paper we consider a sequence of vector optimization problems. We aim to generalize a vector condition that relates the parametric function and the limit function. In particular, we recover our condition given in the scalar case. Our stability approach is such that the limit of the sequence of solutions that correspond to vector optimization problems to be a solution of a limit vector optimization problem. Therefore, one can view our statement as an existence result. This general framework has been used in several previous works. In our main theorem, we use the notion of strong lower cone-semi-continuity. An example is given to illustrate why only cone-lower semi-continuity for the limit function is not sufficient for our result.     

Necessary global optimality conditions for nonlinear programming problems with polynomial constraints

In this paper, we develop necessary conditions for global optimality that apply to non-linear programming problems with polynomial constraints which cover a broad range of optimization problems that arise in applications of continuous as well as discrete optimization. In particular, we show that our optimality conditions readily apply to problems where the objective function is the difference of polynomial and convex functions over polynomial constraints, and to classes of fractional programming problems. Our necessary conditions become also sufficient for global optimality for polynomial programming problems. Our approach makes use of polynomial over-estimators and, a polynomial version of a theorem of the alternative which is a variant of the Positivstellensatz in semi-algebraic geometry. We discuss numerical examples to illustrate the significance of our optimality conditions.     

Lagrange multiplier necessary conditions for global optimality for non-convex minimization over a quadratic constraint via S-lemma

In this paper, we present Lagrange multiplier necessary conditions for global optimality that apply to non-convex optimization problems beyond quadratic optimization problems subject to a single quadratic constraint. In particular, we show that our optimality conditions apply to problems where the objective function is the difference of quadratic and convex functions over a quadratic constraint, and to certain class of fractional programming problems. Our necessary conditions become necessary and sufficient conditions for global optimality for quadratic minimization subject to quadratic constraint. As an application, we also obtain global optimality conditions for a class of trust-region problems. Our approach makes use of outer-estimators, and the powerful S-lemma which has played key role in control theory and semidefinite optimization. We discuss numerical examples to illustrate the significance of our optimality conditions. The authors are grateful to the referees for their useful comments which have contributed to the final preparation of the paper.     

Reference sharing: a new collaboration model for cooperative coevolution

Cooperative coevolutionary algorithms have been a popular and effective learning approach to solve optimization problems through problem decomposition. However, their performance is highly sensitive to the degree of problem separability. Different collaboration mechanisms usually have to be chosen for particular problems. In the paper, we aim to design a collaboration model that can be successfully applied to a wide range of problems. We present a novel collaboration mechanism that offers this type of potential, along with a new sorting strategy for individuals that are assigned multiple fitness values. Furthermore, we demonstrate and analyze our algorithm through comparison studies with other popular cooperative coevolutionary models on a suite of standard function optimization problems.     

A New Scalarization Technique to Approximate Pareto Fronts of Problems with Disconnected Feasible Sets

We introduce and analyze a novel scalarization technique and an associated algorithm for generating an approximation of the Pareto front (i.e., the efficient set) of nonlinear multiobjective optimization problems. Our approach is applicable to nonconvex problems, in particular to those with disconnected Pareto fronts and disconnected domains (i.e., disconnected feasible sets). We establish the theoretical properties of our new scalarization technique and present an algorithm for its implementation. By means of test problems, we illustrate the strengths and advantages of our approach over existing scalarization techniques such as those derived from the Pascoletti–Serafini method, as well as the popular weighted-sum method.     

Higher-order radial derivatives and optimality conditions in nonsmooth vector optimization

We propose notions of higher-order outer and inner radial derivatives of set-valued maps and obtain main calculus rules. Some direct applications of these rules in proving optimality conditions for particular optimization problems are provided. Then we establish higher-order optimality necessary conditions and sufficient ones for a general set-valued vector optimization problem with inequality constraints. A number of examples illustrate both the calculus rules and the optimality conditions. In particular, they explain some advantages of our results over earlier existing ones and why we need higher-order radial derivatives.     

What makes an optimization problem hand?

We address the question: “Are some classes of combinatorial optimization problems intrinsically harder than others, without regard to the algorithm one uses, or can difficulty only be assessed relative to particular algorithms?” We provide a measure of the hardness of a particular optimization problem for a particular optimization algorithm and present two algorithm-independent quantities that use this measure to provide answers to our question. In the first of these we average hardness over all possible algorithms and show that according to this quantity, there are no distinctions between optimization problems. In this sense no problems are intrinsically harder than others. For the second quantity, rather than average over all algorithms, we consider the level of hardness of a problem (or class of problems) for the associated optimal algorithm. By this criteria there are classes of problems that are intrinsically harder than others.     

Completely positive reformulations for polynomial optimization

Polynomial optimization encompasses a very rich class of problems in which both the objective and constraints can be written in terms of polynomials on the decision variables. There is a well established body of research on quadratic polynomial optimization problems based on reformulations of the original problem as a conic program over the cone of completely positive matrices, or its conic dual, the cone of copositive matrices. As a result of this reformulation approach, novel solution schemes for quadratic polynomial optimization problems have been designed by drawing on conic programming tools, and the extensively studied cones of completely positive and of copositive matrices. In particular, this approach has been applied to solve key combinatorial optimization problems. Along this line of research, we consider polynomial optimization problems that are not necessarily quadratic. For this purpose, we use a natural extension of the cone of completely positive matrices; namely, the cone of completely positive tensors. We provide a general characterization of the class of polynomial optimization problems that can be formulated as a conic program over the cone of completely positive tensors. As a consequence of this characterization, it follows that recent related results for quadratic problems can be further strengthened and generalized to higher order polynomial optimization problems. Also, we show that the conditions underlying the characterization are conceptually the same, regardless of the degree of the polynomials defining the problem. To illustrate our results, we discuss in further detail special and relevant instances of polynomial optimization problems.     

Fuzzy criteria for feature selection

The presence of less relevant or highly correlated features often decrease classification accuracy. Feature selection in which most informative variables are selected for model generation is an important step in data-driven modeling. In feature selection, one often tries to satisfy multiple criteria such as feature discriminating power, model performance or subset cardinality. Therefore, a multi-objective formulation of the feature selection problem is more appropriate. In this paper, we propose to use fuzzy criteria in feature selection by using a fuzzy decision making framework. This formulation allows for a more flexible definition of the goals in feature selection, and avoids the problem of weighting different goals is classical multi-objective optimization. The optimization problem is solved using an ant colony optimization algorithm proposed in our previous work. We illustrate the added value of the approach by applying our proposed fuzzy feature selection algorithm to eight benchmark problems.     

Smooth exact penalty functions: a general approach

In the article, we present a new perspective on the method of smooth exact penalty functions that is becoming more and more popular tool for solving constrained optimization problems. In particular, our approach to smooth exact penalty functions allows one to apply previously unused tools (namely, parametric optimization) to the study of these functions. We give a new simple proof of local exactness of smooth penalty functions that significantly generalizes all similar results existing in the literature. We also provide new necessary and sufficient conditions for a smooth penalty function to be globally exact.     

A new approach for modeling and solving set packing problems

In recent years the unconstrained quadratic binary program has emerged as a unified modeling and solution framework for a variety of combinatorial optimization problems. Experience reported in the literature with several problem classes has demonstrated that this unified approach works surprisingly well in terms of solution quality and computational times, often rivaling and sometimes surpassing special purpose methods. In this paper we report on the application of this unified framework to set packing problems with a variety of coefficient distributions. Computational experience is reported illustrating the attractiveness of the approach. In particular, our results highlight both the effectiveness and robustness of our approach across a wide array of test problems.     

Employing different loss functions for the classification of images via supervised learning

Supervised learning methods are powerful techniques to learn a function from a given set of labeled data, the so-called training data. In this paper the support vector machines approach is applied to an image classification task. Starting with the corresponding Tikhonov regularization problem, reformulated as a convex optimization problem, we introduce a conjugate dual problem to it and prove that, whenever strong duality holds, the function to be learned can be expressed via the dual optimal solutions. Corresponding dual problems are then derived for different loss functions. The theoretical results are applied by numerically solving a classification task using high dimensional real-world data in order to obtain optimal classifiers. The results demonstrate the excellent performance of support vector classification for this particular problem.     

Ensemble feature selection for high dimensional data: a new method and a comparative study

The curse of dimensionality is based on the fact that high dimensional data is often difficult to work with. A large number of features can increase the noise of the data and thus the error of a learning algorithm. Feature selection is a solution for such problems where there is a need to reduce the data dimensionality. Different feature selection algorithms may yield feature subsets that can be considered local optima in the space of feature subsets. Ensemble feature selection combines independent feature subsets and might give a better approximation to the optimal subset of features. We propose an ensemble feature selection approach based on feature selectors’ reliability assessment. It aims at providing a unique and stable feature selection without ignoring the predictive accuracy aspect. A classification algorithm is used as an evaluator to assign a confidence to features selected by ensemble members based on their associated classification performance. We compare our proposed approach to several existing techniques and to individual feature selection algorithms. Results show that our approach often improves classification performance and feature selection stability for high dimensional data sets.     

Book Review of Introduction to the theory of toeplitz operators with infinite index,Vladimir Dybin and Sergei M. Grudsky (Translated from Russian by Andrei Jacob), operator theory: advances and applications,Birkhäuser Verlag,Basel-Boston-Berlin, 2002, volume 137, 299 pages. ISBN 3-7643-6728-8.

We consider a special class of optimization problems that we call a Mathematical Programme with Vanishing Constraints. It has a number of important applications in structural and topology optimization, but typically does not satisfy standard constraint qualifications like the linear independence and the Mangasarian–Fromovitz constraint qualification. We therefore investigate the Abadie and Guignard constraint qualifications in more detail. In particular, it follows from our results that also the Abadie constraint qualification is typically not satisfied, whereas the Guignard constraint qualification holds under fairly mild assumptions for our particular class of optimization problems.     

A New Conical Regularization for Some Optimization and Optimal Control Problems: Convergence Analysis and Finite Element Discretization

In this article, we study an abstract constrained optimization problem that appears commonly in the optimal control of linear partial differential equations. The main emphasis of the present study is on the case when the ordering cone for the optimization problem has an empty interior. To circumvent this major difficulty, we propose a new conical regularization approach in which the main idea is to replace the ordering cone by a family of dilating cones. We devise a general regularization approach and use it to give a detailed convergence analysis for the conical regularization as well as a related regularization approach. We showed that the conical regularization approach leads to a family of optimization problems that admit regular multipliers. The approach remains valid in the setting of general Hilbert spaces and it does not require any sort of compactness or positivity condition on the operators involved. One of the main advantages of the approach is that it is amenable for numerical computations. We consider four different examples, two of them elliptic control problems with state constraints, and present numerical results that completely support our theoretical results and confirm the numerical feasibility of our approach. The motivation for the conical regularization is to overcome the difficulties associated with the lack of Slater's type constraint qualification, which is a common hurdle in numerous branches of applied mathematics including optimal control, inverse problems, vector optimization, set-valued optimization, sensitivity analysis, variational inequalities, among others.     

Reinforced Angle-Based Multicategory Support Vector Machines

The support vector machine (SVM) is a very popular classification tool with many successful applications. It was originally designed for binary problems with desirable theoretical properties. Although there exist various multicategory SVM (MSVM) extensions in the literature, some challenges remain. In particular, most existing MSVMs make use of k classification functions for a k-class problem, and the corresponding optimization problems are typically handled by existing quadratic programming solvers. In this article, we propose a new group of MSVMs, namely, the reinforced angle-based MSVMs (RAMSVMs), using an angle-based prediction rule with k ? 1 functions directly. We prove that RAMSVMs can enjoy Fisher consistency. Moreover, we show that the RAMSVM can be implemented using the very efficient coordinate descent algorithm on its dual problem. Numerical experiments demonstrate that our method is highly competitive in terms of computational speed, as well as classification prediction performance. Supplemental materials for the article are available online.     

A global optimization method for the design of space trajectories

The problem of optimally designing a trajectory for a space mission is considered in this paper. Actual mission design is a complex, multi-disciplinary and multi-objective activity with relevant economic implications. In this paper we will consider some simplified models proposed by the European Space Agency as test problems for global optimization (GTOP database). We show that many trajectory optimization problems can be quite efficiently solved by means of relatively simple global optimization techniques relying on standard methods for local optimization. We show in this paper that our approach has been able to find trajectories which in many cases outperform those already known. We also conjecture that this problem displays a “funnel structure” similar, in some sense, to that of molecular optimization problems.     

On pseudocyclic table algebras and applications to pseudocyclic association schemes

We effect a complete study of the thermodynamic formalism, the entropy spectrum of Birkhoff averages, and the ergodic optimization problem for a family of parabolic horseshoes. We consider a large class of potentials that are not necessarily regular, and we describe both the uniqueness of equilibrium measures and the occurrence of phase transitions for nonregular potentials in this class. Our approach consists in reducing the problems to the study of renewal shifts. We also describe applications of this approach to hyperbolic horseshoes as well as to noninvertible maps, both parabolic (with the Manneville-Pomeau map) and uniformly expanding. This allows us to recover in a unified manner several results scattered in the literature. For the family of hyperbolic horseshoes, we also describe the dimension spectrum of equilibrium measures of a class of potentials that are not necessarily regular. In particular, the dimension spectra need not be strictly convex.     

Solving Fractional Multicriteria Optimization Problems with Sum of Squares Convex Polynomial Data

This paper focuses on the study of finding efficient solutions in fractional multicriteria optimization problems with sum of squares convex polynomial data. We first relax the fractional multicriteria optimization problems to fractional scalar ones. Then, using the parametric approach, we transform the fractional scalar problems into non-fractional problems. Consequently, we prove that, under a suitable regularity condition, the optimal solution of each non-fractional scalar problem can be found by solving its associated single semidefinite programming problem. Finally, we show that finding efficient solutions in the fractional multicriteria optimization problems is tractable by employing the epsilon constraint method. In particular, if the denominators of each component of the objective functions are same, then we observe that efficient solutions in such a problem can be effectively found by using the hybrid method. Some numerical examples are given to illustrate our results.     

Fast interior point solvers for H1-regularized PDE-constrained optimization problems

We consider Newton systems arising from the interior point solution of PDE-constrained optimization problems. In particular, we examine problems where the control variable is regularized by an H¹-norm within the cost functional. We present preconditioned iterative methods for the resulting matrix systems, and justify the potency of our approach through numerical experiments. (© 2016 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim)