A Smoothing Objective Penalty Function Algorithm for Inequality Constrained Optimization Problems

In this article, a smoothing objective penalty function for inequality constrained optimization problems is presented. The article proves that this type of the smoothing objective penalty functions has good properties in helping to solve inequality constrained optimization problems. Moreover, based on the penalty function, an algorithm is presented to solve the inequality constrained optimization problems, with its convergence under some conditions proved. Two numerical experiments show that a satisfactory approximate optimal solution can be obtained by the proposed algorithm.     

Inexact subgradient methods for quasi-convex optimization problems

In this paper, we consider a generic inexact subgradient algorithm to solve a nondifferentiable quasi-convex constrained optimization problem. The inexactness stems from computation errors and noise, which come from practical considerations and applications. Assuming that the computational errors and noise are deterministic and bounded, we study the effect of the inexactness on the subgradient method when the constraint set is compact or the objective function has a set of generalized weak sharp minima. In both cases, using the constant and diminishing stepsize rules, we describe convergence results in both objective values and iterates, and finite convergence to approximate optimality. We also investigate efficiency estimates of iterates and apply the inexact subgradient algorithm to solve the Cobb–Douglas production efficiency problem. The numerical results verify our theoretical analysis and show the high efficiency of our proposed algorithm, especially for the large-scale problems.     

Cone contraction and reference point methods for multi-criteria mixed integer optimization

Interactive approaches employing cone contraction for multi-criteria mixed integer optimization are introduced. In each iteration, the decision maker (DM) is asked to give a reference point (new aspiration levels). The subsequent Pareto optimal point is the reference point projected on the set of admissible objective vectors using a suitable scalarizing function. Thereby, the procedures solve a sequence of optimization problems with integer variables. In such a process, the DM provides additional preference information via pair-wise comparisons of Pareto optimal points identified. Using such preference information and assuming a quasiconcave and non-decreasing value function of the DM we restrict the set of admissible objective vectors by excluding subsets, which cannot improve over the solutions already found. The procedures terminate if all Pareto optimal solutions have been either generated or excluded. In this case, the best Pareto point found is an optimal solution. Such convergence is expected in the special case of pure integer optimization; indeed, numerical simulation tests with multi-criteria facility location models and knapsack problems indicate reasonably fast convergence, in particular, under a linear value function. We also propose a procedure to test whether or not a solution is a supported Pareto point (optimal under some linear value function).     

Proximal Point Algorithm On Riemannian Manifolds

Abstract

In this paper we consider the minimization problem with constraints. We will show that if the set of constraints is a Riemannian manifold of nonpositive sectional curvature, and the objective function is convex in this manifold, then the proximal point method in Euclidean space is naturally extended to solve that class of problems. We will prove that the sequence generated by our method is well defined and converge to a minimizer point. In particular we show how tools of Riemannian geometry, more specifically the convex analysis in Riemannian manifolds, can be used to solve nonconvex constrained problem in Euclidean, space.     

Modified active set projected spectral gradient method for bound constrained optimization

In this paper, by means of an active set strategy, we present a projected spectral gradient algorithm for solving large-scale bound constrained optimization problems. A nice property of the active set estimation technique is that it can identify the active set at the optimal point without requiring strict complementary condition, which is potentially used to solve degenerated optimization problems. Under appropriate conditions, we show that this proposed method is globally convergent. We also do some numerical experiments by using some bound constrained problems from CUTEr library. The numerical comparisons with SPG, TRON, and L-BFGS-B show that the proposed method is effective and promising.     

一种具有非线性约束线性规划全局优化算法

本文提出了一种新的适用于处理非线性约束下线性规划问题的全局优化算法。该算法通过构造子问题来寻找优于当前局部最优解的可行解。该子问题可通过模拟退火算法来解决。通过求解一系列的子问题,当前最优解被不断地更新,最终求得全局最优解。最后,本算法应用于几个典型例题,并与罚函数法相比较,数值结果表明该算法是可行的,有效的。     

An extension of proximal methods for quasiconvex minimization on the nonnegative orthant

In this paper we propose an extension of proximal methods to solve minimization problems with quasiconvex objective functions on the nonnegative orthant. Assuming that the function is bounded from below and lower semicontinuous and using a general proximal distance, it is proved that the iterations given by our algorithm are well defined and stay in the positive orthant. If the objective function is quasiconvex we obtain the convergence of the iterates to a certain set which contains the set of optimal solutions and convergence to a KKT point if the function is continuously differentiable and the proximal parameters are bounded. Furthermore, we introduce a sufficient condition on the proximal distance such that the sequence converges to an optimal solution of the problem.     

Steepest descent method with a generalized Armijo search for quasiconvex functions on Riemannian manifolds

This paper extends the full convergence of the steepest descent method with a generalized Armijo search and a proximal regularization to solve minimization problems with quasiconvex objective functions on complete Riemannian manifolds. Previous convergence results are obtained as particular cases and some examples in non-Euclidian spaces are given. In particular, our approach can be used to solve constrained minimization problems with nonconvex objective functions in Euclidian spaces if the set of constraints is a Riemannian manifold and the objective function is quasiconvex in this manifold.     

A convergent decomposition algorithm for support vector machines

In this work we consider nonlinear minimization problems with a single linear equality constraint and box constraints. In particular we are interested in solving problems where the number of variables is so huge that traditional optimization methods cannot be directly applied. Many interesting real world problems lead to the solution of large scale constrained problems with this structure. For example, the special subclass of problems with convex quadratic objective function plays a fundamental role in the training of Support Vector Machine, which is a technique for machine learning problems. For this particular subclass of convex quadratic problem, some convergent decomposition methods, based on the solution of a sequence of smaller subproblems, have been proposed. In this paper we define a new globally convergent decomposition algorithm that differs from the previous methods in the rule for the choice of the subproblem variables and in the presence of a proximal point modification in the objective function of the subproblems. In particular, the new rule for sequentially selecting the subproblems appears to be suited to tackle large scale problems, while the introduction of the proximal point term allows us to ensure the global convergence of the algorithm for the general case of nonconvex objective function. Furthermore, we report some preliminary numerical results on support vector classification problems with up to 100 thousands variables.     

Proximal Point Method on Finslerian Manifolds and the “Effort–Accuracy” Trade-off

In this paper, we consider minimization problems with constraints. We show that, if the set of constraints is a Finslerian manifold of non-positive flag curvature, and the objective function is differentiable and satisfies the Kurdyka-Lojasiewicz property, then the proximal point method can be naturally extended to solve this class of problems. We prove that the sequence generated by our method is well defined and converges to a critical point. We show how tools of Finslerian geometry, specifically non-symmetrical metrics, can be used to solve non-convex constrained problems in Euclidean spaces. As an application, we give one result regarding decision-making speed and costs related to change.     

Augmented Lagrangian Objective Penalty Function

Augmented Lagrangian function is one of the most important tools used in solving some constrained optimization problems. In this article, we study an augmented Lagrangian objective penalty function and a modified augmented Lagrangian objective penalty function for inequality constrained optimization problems. First, we prove the dual properties of the augmented Lagrangian objective penalty function, which are at least as good as the traditional Lagrangian function's. Under some conditions, the saddle point of the augmented Lagrangian objective penalty function satisfies the first-order Karush-Kuhn-Tucker condition. This is especially so when the Karush-Kuhn-Tucker condition holds for convex programming of its saddle point existence. Second, we prove the dual properties of the modified augmented Lagrangian objective penalty function. For a global optimal solution, when the exactness of the modified augmented Lagrangian objective penalty function holds, its saddle point exists. The sufficient and necessary stability conditions used to determine whether the modified augmented Lagrangian objective penalty function is exact for a global solution is proved. Based on the modified augmented Lagrangian objective penalty function, an algorithm is developed to find a global solution to an inequality constrained optimization problem, and its global convergence is also proved under some conditions. Furthermore, the sufficient and necessary calmness condition on the exactness of the modified augmented Lagrangian objective penalty function is proved for a local solution. An algorithm is presented in finding a local solution, with its convergence proved under some conditions.     

A derivative-free algorithm for systems of nonlinear inequalities

Recently a new derivative-free algorithm has been proposed for the solution of linearly constrained finite minimax problems. This derivative-free algorithm is based on a smoothing technique that allows one to take into account the non-smoothness of the max function. In this paper, we investigate, both from a theoretical and computational point of view, the behavior of the minmax algorithm when used to solve systems of nonlinear inequalities when derivatives are unavailable. In particular, we show an interesting property of the algorithm, namely, under some mild conditions regarding the regularity of the functions defining the system, it is possible to prove that the algorithm locates a solution of the problem after a finite number of iterations. Furthermore, under a weaker regularity condition, it is possible to show that an accumulation point of the sequence generated by the algorithm exists which is a solution of the system. Moreover, we carried out numerical experimentation and comparison of the method against a standard pattern search minimization method. The obtained results confirm that the good theoretical properties of the method correspond to interesting numerical performance. Moreover, the algorithm compares favorably with a standard derivative-free method, and this seems to indicate that extending the smoothing technique to pattern search algorithms can be beneficial.     

Inner Approximation Method for a Reverse Convex Programming Problem

In this paper, we consider a reverse convex programming problem constrained by a convex set and a reverse convex set, which is defined by the complement of the interior of a compact convex set X. We propose an inner approximation method to solve the problem in the case where X is not necessarily a polytope. The algorithm utilizes an inner approximation of X by a sequence of polytopes to generate relaxed problems. It is shown that every accumulation point of the sequence of optimal solutions of the relaxed problems is an optimal solution of the original problem.     

Approximation of solutions for location problems

This paper is devoted to a class of location problems with polyhedral norms. The objective function is shown to be a piecewise convex function which has to be maximized. We prove that the optimal locations belong to a finite set of intersection points, and we present an efficient method operating upon this finite set and providing a strict local maximum with few computational effort.     

An Adaptive Newton Algorithm for Optimal Control Problems with Application to Optimal Electrode Design

In this work, we present an adaptive Newton-type method to solve nonlinear constrained optimization problems, in which the constraint is a system of partial differential equations discretized by the finite element method. The adaptive strategy is based on a goal-oriented a posteriori error estimation for the discretization and for the iteration error. The iteration error stems from an inexact solution of the nonlinear system of first-order optimality conditions by the Newton-type method. This strategy allows one to balance the two errors and to derive effective stopping criteria for the Newton iterations. The algorithm proceeds with the search of the optimal point on coarse grids, which are refined only if the discretization error becomes dominant. Using computable error indicators, the mesh is refined locally leading to a highly efficient solution process. The performance of the algorithm is shown with several examples and in particular with an application in the neurosciences: the optimal electrode design for the study of neuronal networks.     

On Extensions of the Frank-Wolfe Theorems

In this paper we consider optimization problems defined by a quadratic objective function and a finite number of quadratic inequality constraints. Given that the objective function is bounded over the feasible set, we present a comprehensive study of the conditions under which the optimal solution set is nonempty, thus extending the so-called Frank-Wolfe theorem. In particular, we first prove a general continuity result for the solution set defined by a system of convex quadratic inequalities. This result implies immediately that the optimal solution set of the aforementioned problem is nonempty when all the quadratic functions involved are convex. In the absence of the convexity of the objective function, we give examples showing that the optimal solution set may be empty either when there are two or more convex quadratic constraints, or when the Hessian of the objective function has two or more negative eigenvalues. In the case when there exists only one convex quadratic inequality constraint (together with other linear constraints), or when the constraint functions are all convex quadratic and the objective function is quasi-convex (thus allowing one negative eigenvalue in its Hessian matrix), we prove that the optimal solution set is nonempty.     

Singularities of Monotone Vector Fields and an Extragradient-type Algorithm

Bearing in mind the notion of monotone vector field on Riemannian manifolds, see [12--16], we study the set of their singularities and for a particularclass of manifolds develop an extragradient-type algorithm convergent to singularities of such vector fields. In particular, our method can be used forsolving nonlinear constrained optimization problems in Euclidean space, with a convex objective function and the constraint set a constant curvature Hadamard manifold. Our paper shows how tools of convex analysis on Riemannian manifolds can be used to solve some nonconvex constrained problem in a Euclidean space.O.P. Ferreira- was supported in part by CAPES, FUNAPE (UFG) and (CNPq).S.Z. Németh- was supported in part by grant No.T029572 of the National Research Foundation of Hungary.     

Relaxations and approximations of chance constraints under finite distributions

Optimization problems with constraints involving stochastic parameters that are required to be satisfied with a prespecified probability threshold arise in numerous applications. Such chance constrained optimization problems involve the dual challenges of stochasticity and nonconvexity. In the setting of a finite distribution of the stochastic parameters, an optimization problem with linear chance constraints can be formulated as a mixed integer linear program (MILP). The natural MILP formulation has a weak relaxation bound and is quite difficult to solve. In this paper, we review some recent results on improving the relaxation bounds and constructing approximate solutions for MILP formulations of chance constraints. We also discuss a recently introduced bicriteria approximation algorithm for covering type chance constrained problems. This algorithm uses a relaxation to construct a solution whose (constraint violation) risk level may be larger than the pre-specified threshold, but is within a constant factor of it, and whose objective value is also within a constant factor of the true optimal value. Finally, we present some new results that improve on the bicriteria approximation factors in the finite scenario setting and shed light on the effect of strong relaxations on the approximation ratios.     

AN IMPLEMENTABLE ALGORITHM AND ITS CONVERGENCE FOR GLOBAL MINIMIZATION WITH CONSTRAINS

With the integral-level approach to global optimization, a class of discontinuous penalty functions is proposed to solve constrained minimization problems. In this paper we propose an implementable algorithm by means of the good point set of uniform distribution which conquers the default of Monte-Carlo method. At last we prove the convergence of the implementable algorithm.     

An Objective Penalty Function of Bilevel Programming

Penalty methods are very efficient in finding an optimal solution to constrained optimization problems. In this paper, we present an objective penalty function with two penalty parameters for inequality constrained bilevel programming under the convexity assumption to the lower level problem. Under some conditions, an optimal solution to a bilevel programming defined by the objective penalty function is proved to be an optimal solution to the original bilevel programming. Moreover, based on the objective penalty function, an algorithm is developed to obtain an optimal solution to the original bilevel programming, with its convergence proved under some conditions.