Two Types of Approximate Saddle Points

In this paper, we study two types of approximate solutions for a vector optimization problem in Banach space setting. Our main concern is to define two new concepts of approximate saddle points and relate them to the above solution concepts. As a result, a dual is formulated, and duality results are established.     

Set-Valued Systems with Infinite-Dimensional Image and Applications

In infinite-dimensional spaces, we investigate a set-valued system from the image perspective. By exploiting the quasi-interior and the quasi-relative interior, we give some new equivalent characterizations of (proper, regular) linear separation and establish some new sufficient and necessary conditions for the impossibility of the system under new assumptions, which do not require the set to have nonempty interior. We also present under mild assumptions the equivalence between (proper, regular) linear separation and saddle points of Lagrangian functions for the system. These results are applied to obtain some new saddle point sufficient and necessary optimality conditions of vector optimization problems.     

ε-Mixed type duality for nonconvex multiobjective programs with an infinite number of constraints

Using a scalarization method, approximate optimality conditions of a multiobjective nonconvex optimization problem which has an infinite number of constraints are established. Approximate duality theorems for mixed duality are given. Results on approximate duality in Wolfe type and Mond-Weir type are also derived. Approximate saddle point theorems of an approximate vector Lagrangian function are investigated.     

Subgradient Methods for Saddle-Point Problems

We study subgradient methods for computing the saddle points of a convex-concave function. Our motivation comes from networking applications where dual and primal-dual subgradient methods have attracted much attention in the design of decentralized network protocols. We first present a subgradient algorithm for generating approximate saddle points and provide per-iteration convergence rate estimates on the constructed solutions. We then focus on Lagrangian duality, where we consider a convex primal optimization problem and its Lagrangian dual problem, and generate approximate primal-dual optimal solutions as approximate saddle points of the Lagrangian function. We present a variation of our subgradient method under the Slater constraint qualification and provide stronger estimates on the convergence rate of the generated primal sequences. In particular, we provide bounds on the amount of feasibility violation and on the primal objective function values at the approximate solutions. Our algorithm is particularly well-suited for problems where the subgradient of the dual function cannot be evaluated easily (equivalently, the minimum of the Lagrangian function at a dual solution cannot be computed efficiently), thus impeding the use of dual subgradient methods.     

A preconditioning technique for a class of PDE-constrained optimization problems

We investigate the use of a preconditioning technique for solving linear systems of saddle point type arising from the application of an inexact Gauss?CNewton scheme to PDE-constrained optimization problems with a hyperbolic constraint. The preconditioner is of block triangular form and involves diagonal perturbations of the (approximate) Hessian to insure nonsingularity and an approximate Schur complement. We establish some properties of the preconditioned saddle point systems and we present the results of numerical experiments illustrating the performance of the preconditioner on a model problem motivated by image registration.     

Generic uniqueness theorems with some applications

We first present a proper condition under which the image of a set-valued mapping becomes a singleton and then obtain several generic uniqueness theorems which can be applied to study the uniqueness of the solutions for nonlinear problems. As applications, we prove that, in the sense of Baire category, most optimization problems (respectively, saddle point problems and variational inequality problems) have unique solution.     

近似锥一次类凸集值向量优化问题强有效解的广义鞍点刻画

本文研究了近似锥一次类凸集值向量优化强有效解的广义鞍点表示问题.利用择一定理,得到了近似锥-次类凸集值优化问题强有效解为广义鞍点的充分条件和必要条件.所得结果丰富了集值优化理论,并且拓广了广义鞍点的应用.     

集值映射多目标半定规划的Kuhn-Tucker鞍点最优性条件

首先在序拓扑线性空间中定义了集值映射多目标半定规划问题的KuhnTucker鞍点,在广义锥-次类凸条件下,讨论了此集值优化问题的弱有效解和Benson真有效性解与Kuhn-Tucker鞍点之间的关系.     

集值优化问题的强有效解与Kuhn-Tucker鞍点

首先在局部凸Hausdorff拓扑向量空间中定义了集值优化问题的Kuhn-Tucker鞍点,在近似锥-次类凸集值映射下,讨论了集值优化问题的强有效解与Kuhn-Tucker鞍点之间的关系.     

集值优化问题的强有效解与Kuhn—Tucker鞍点

首先在局部凸Hausdorff拓扑向量空间中定义了集值优化问题的Kuhn—Tucker鞍点，在近似锥一次类凸集值映射下，讨论了集值优化问题的强有效解与Kuhn—Tucker鞍点之间的关系．     

Minimax feature selection problem for constructing a classifier using support vector machines

A minimax feature selection problem for constructing a classifier using support vector machines is considered. Properties of the solutions of this problem are analyzed. An improvement of the saddle point search algorithm based on extending the bound for the step parameter is proposed. A new nondifferential optimization algorithm is developed that, together with the saddle point search algorithm, forms a hybrid feature selection algorithm. The efficiency of the algorithm for computing Dykstra’s projections as applied for the feature selection problem is experimentally estimated.     

Super efficiency in vector optimization with nearly convexlike set-valued maps

In this paper, we establish a scalarization theorem and a Lagrange multiplier theorem for super efficiency in vector optimization problem involving nearly convexlike set-valued maps. A dual is proposed and duality results are obtained in terms of super efficient solutions. A new type of saddle point, called super saddle point, of an appropriate set-valued Lagrangian map is introduced and is used to characterize super efficiency.     

序线性空间中向量优化问题的K-T型定理

本文在广义凸性条件下,研究实线性空间中一类向量优化问题的最优性条件,我们引入F ritz-John鞍点,、“K-T鞍点”讨论它们与有效解、弱有效解之间的关系.     

Lagrange Multipliers and saddle points in multiobjective programming

In this paper, we present several conditions for the existence of a Lagrange multiplier or a weak saddle point in multiobjective optimization. Relations between a Lagrange multiplier and a weak saddle point are established. A sufficient condition is also given for the equivalence of the Benson proper efficiency and the Borwein proper efficiency.This research was supported by NSFC under Grant No. 78900011 and by BMADIS. The authors are grateful to two referees for supplying valuable comments and pointing out detailed corrections to the draft paper. The authors also wish to thank Dr. P. L. Yu for valuable comments and suggestions.The revised version of this paper was completed while the second author visited the Faculty of Technical Mathematics and Informatics, Delft University of Technology, Delft, The Netherlands.     

向量映射的鞍点和Lagrange对偶问题

本文研究拓扑向量空间广义锥-次类凸映射向量优化问题的鞍点最优性条件和Lagrange对偶问题,建立向量优化问题的Fritz John鞍点和Kuhn-Tucker鞍点的最优性条件及其与向量优化问题的有效解和弱有效解之间的联系。通过对偶问题和向量优化问题的标量化刻画各解之间的关系,给出目标映射是广义锥-次类凸的向量优化问题在其约束映射满足广义Slater约束规格的条件下的对偶定理。     

Limiting accuracy of segregated solution methods for nonsymmetric saddle point problems

Nonsymmetric saddle point problems arise in a wide variety of applications in computational science and engineering. The aim of this paper is to discuss the numerical behavior of several nonsymmetric iterative methods applied for solving the saddle point systems via the Schur complement reduction or the null-space projection approach. Krylov subspace methods often produce the iterates which fluctuate rather strongly. Here we address the question whether large intermediate approximate solutions reduce the final accuracy of these two-level (inner–outer) iteration algorithms. We extend our previous analysis obtained for symmetric saddle point problems and distinguish between three mathematically equivalent back-substitution schemes which lead to a different numerical behavior when applied in finite precision arithmetic. Theoretical results are then illustrated on a simple model example.     

Approximate Duality

We extend the Lagrangian duality theory for convex optimization problems to incorporate approximate solutions. In particular, we generalize well-known relationships between minimizers of a convex optimization problem, maximizers of its Lagrangian dual, saddle points of the Lagrangian, Kuhn–Tucker vectors, and Kuhn–Tucker conditions to incorporate approximate versions. As an application, we show how the theory can be used for convex quadratic programming and then apply the results to support vector machines from learning theory.     

A simplicial algorithm for the nonlinear complementarity problem

A triangulation of the nonnegative orthant and a special labeling of the vertices lead to a combinatorial procedure for seeking solutions or approximate solutions to the nonlinear complementarity problem under coercive-like assumptions on the problem functions. Derivatives are not required. Convergence is proved, computational considerations are discussed, and some preliminary applications to convex programming and saddle point computation, along with numerical results, are presented.     

序线性空间中广义半似凸集值映射向量优化的Benson真有效性

在序线性空间中建立了广义半似凸集值映射的择一定理.利用向量闭包,引进了集值优化的Benson真有效解.在广义半似凸的假设下,获得了Benson真有效性意义下的标量化定理,Lagrangian乘子定理和鞍点定理.     

Efficient and Adaptive Lagrange-Multiplier Methods for Nonlinear Continuous Global Optimization

Lagrangian methods are popular in solving continuous constrained optimization problems. In this paper, we address three important issues in applying Lagrangian methods to solve optimization problems with inequality constraints.First, we study methods to transform inequality constraints into equality constraints. An existing method, called the slack-variable method, adds a slack variable to each inequality constraint in order to transform it into an equality constraint. Its disadvantage is that when the search trajectory is inside a feasible region, some satisfied constraints may still pose some effect on the Lagrangian function, leading to possible oscillations and divergence when a local minimum lies on the boundary of the feasible region. To overcome this problem, we propose the MaxQ method that carries no effect on satisfied constraints. Hence, minimizing the Lagrangian function in a feasible region always leads to a local minimum of the objective function. We also study some strategies to speed up its convergence.Second, we study methods to improve the convergence speed of Lagrangian methods without affecting the solution quality. This is done by an adaptive-control strategy that dynamically adjusts the relative weights between the objective and the Lagrangian part, leading to better balance between the two and faster convergence.Third, we study a trace-based method to pull the search trajectory from one saddle point to another in a continuous fashion without restarts. This overcomes one of the problems in existing Lagrangian methods that converges only to one saddle point and requires random restarts to look for new saddle points, often missing good saddle points in the vicinity of saddle points already found.Finally, we describe a prototype Novel (Nonlinear Optimization via External Lead) that implements our proposed strategies and present improved solutions in solving a collection of benchmarks.