A cutting plane method from analytic centers for stochastic programming

The stochastic linear programming problem with recourse has a dual block-angular structure. It can thus be handled by Benders' decomposition or by Kelley's method of cutting planes; equivalently the dual problem has a primal block-angular structure and can be handled by Dantzig-Wolfe decomposition—the two approaches are in fact identical by duality. Here we shall investigate the use of the method of cutting planes from analytic centers applied to similar formulations. The only significant difference form the aforementioned methods is that new cutting planes (or columns, by duality) will be generated not from the optimum of the linear programming relaxation, but from the analytic center of the set of localization.This research has been supported by the Fonds National de la Recherche Scientifique Suisse (grant # 12-26434.89), NSERC-Canada and FCAR-Quebec.Corresponding author.     

Barrier method in nonsmooth convex optimization without convex representation

In this article we want to demonstrate that under mild conditions the barrier method is an effective solution approach for convex optimization problems whose objective is nonsmooth and whose feasible set is described by smooth inequality constraints in which all the constraint functions need not be convex.     

The minimization of lower subdifferentiable functions under nonlinear constraints: An all feasible cutting plane algorithm

Nonlinear, possibly nonsmooth, minimization problems are considered with boundedly lower subdifferentiable objective and constraints. An algorithm of the cutting plane type is developed, which has the property that the objective needs to be considered at feasible points only. It generates automatically a nondecreasing sequence of lower bounds converging to the optimal function value, thus admitting a rational rule for stopping the calculations when sufficient precision in the objective value has been obtained. Details are given concerning the efficient implementation of the algorithm. Computational results are reported concerning the algorithm as applied to continuous location problems with distance constraints. The author thanks the referees for several constructive remarks and for pointing out an error in an earlier version of the proof of Lemma 2.1.     

Analytic center of spherical shells and its application to analytic center machine

The two-case pattern recognition problem aims to find the best way of linearly separate two different classes of data points with a good generalization performance. In the context of learning machines proposed to solve the pattern recognition problem, the analytic center machine (ACM) uses the analytic center cutting plane method restricted to spherical shells. In this work we prove existence and uniqueness of the analytic center of a spherical surface, which guarantees the well definedness of ACM problem. We also propose and analyze new primal, dual and primal-dual formulations based on interior point methods for the analytic center machine. Further, we provide a complexity bound on the number of iterations for the primal approach. F.M.P. Raupp was partially supported by CNPq Grant 475647/2006-8 and FAPERJ/CNPq through PRONEX-Computational Modeling. B.F. Svaiter was partially supported by CNPq Grants 300755/2005-8, 475647/2006-8 and by FAPERJ/CNPq through PRONEX-Optimization.     

Solving dual problems using a coevolutionary optimization algorithm

In solving certain optimization problems, the corresponding Lagrangian dual problem is often solved simply because in these problems the dual problem is easier to solve than the original primal problem. Another reason for their solution is the implication of the weak duality theorem which suggests that under certain conditions the optimal dual function value is smaller than or equal to the optimal primal objective value. The dual problem is a special case of a bilevel programming problem involving Lagrange multipliers as upper-level variables and decision variables as lower-level variables. Another interesting aspect of dual problems is that both lower and upper-level optimization problems involve only box constraints and no other equality of inequality constraints. In this paper, we propose a coevolutionary dual optimization (CEDO) algorithm for co-evolving two populations—one involving Lagrange multipliers and other involving decision variables—to find the dual solution. On 11 test problems taken from the optimization literature, we demonstrate the efficacy of CEDO algorithm by comparing it with a couple of nested smooth and nonsmooth algorithms and a couple of previously suggested coevolutionary algorithms. The performance of CEDO algorithm is also compared with two classical methods involving nonsmooth (bundle) optimization methods. As a by-product, we analyze the test problems to find their associated duality gap and classify them into three categories having zero, finite or infinite duality gaps. The development of a coevolutionary approach, revealing the presence or absence of duality gap in a number of commonly-used test problems, and efficacy of the proposed coevolutionary algorithm compared to usual nested smooth and nonsmooth algorithms and other existing coevolutionary approaches remain as the hallmark of the current study.     

On solving the Lagrangian dual of integer programs via an incremental approach

The Lagrangian dual of an integer program can be formulated as a min-max problem where the objective function is convex, piecewise affine and, hence, nonsmooth. It is usually tackled by means of subgradient algorithms, or multiplier adjustment techniques, or even more sophisticated nonsmooth optimization methods such as bundle-type algorithms. Recently a new approach to solving unconstrained convex finite min-max problems has been proposed, which has the nice property of working almost independently of the exact evaluation of the objective function at every iterate-point.     

A Polynomial Cutting Surfaces Algorithm for the Convex Feasibility Problem Defined by Self-Concordant Inequalities

Consider a nonempty convex set in ^m which is defined by a finite number of smooth convex inequalities and which admits a self-concordant logarithmic barrier. We study the analytic center based column generation algorithm for the problem of finding a feasible point in this set. At each iteration the algorithm computes an approximate analytic center of the set defined by the inequalities generated in the previous iterations. If this approximate analytic center is a solution, then the algorithm terminates; otherwise either an existing inequality is shifted or a new inequality is added into the system. As the number of iterations increases, the set defined by the generated inequalities shrinks and the algorithm eventually finds a solution of the problem. The algorithm can be thought of as an extension of the classical cutting plane method. The difference is that we use analytic centers and convex cuts instead of arbitrary infeasible points and linear cuts. In contrast to the cutting plane method, the algorithm has a polynomial worst case complexity of O(Nlog 1/) on the total number of cuts to be used, where N is the number of convex inequalities in the original problem and is the maximum common slack of the original inequality system.     

Augmented self-concordant barriers and nonlinear optimization problems with finite complexity

In this paper we study special barrier functions for convex cones, which are the sum of a self-concordant barrier for the cone and a positive-semidefinite quadratic form. We show that the central path of these augmented barrier functions can be traced with linear speed. We also study the complexity of finding the analytic center of the augmented barrier, a problem that has some interesting applications. We show that for some special classes of quadratic forms and some convex cones, the computation of the analytic center requires an amount of operations independent of the particular data set. We argue that these problems form a class that is endowed with a property which we call finite polynomial complexity. This paper presents research results of the Belgian Program on Interuniversity Poles of Attraction initiated by the Belgian State, Prime Ministers Office, Science Policy Programming.This author acknowledges the support of the grant 1214-057093-99 of the Swiss National Science Foundation.     

On the volumetric path

We consider the logarithmic and the volumetric barrier functions used in interior point methods. In the case of the logarithmic barrier function, the analytic center of a level set is the point at which the central path intersects that level set. We prove that this also holds for the volumetric path. For the central path, it is also true that the analytic center of the optimal level set is the limit point of the central path. The only known case where this last property for the logarithmic barrier function fails occurs in case of semidefinite optimization in the absence of strict complementarity. For the volumetric path, we show with an example that this property does not hold even for a linear optimization problem in canonical form.     

A New Approach to a Multicriteria Optimization Problem

We present a new approach to a multicriteria optimization problem, where the objective and the constraints are linear functions. From an equivalent equilibrium problem, first suggested in [5,6,8], we show new characterizations of weakly efficient points based on the partial order induced by a nonempty closed convex cone in a finite-dimensional linear space, as in [7]. Thus, we are able to apply the analytic center cutting plane algorithm that finds equilibrium points approximately, by Raupp and Sosa [10], in order to find approximate weakly efficient solutions of MOP.     

On Well Definedness of the Central Path

We study the well definedness of the central path for a linearly constrained convex programming problem with smooth objective function. We prove that, under standard assumptions, existence of the central path is equivalent to the nonemptiness and boundedness of the optimal set. Other equivalent conditions are given. We show that, under an additional assumption on the objective function, the central path converges to the analytic center of the optimal set.     

Observability with bearing-only observations and smoothness of the attainable set

The attainable set of a linear control system can have both smooth and nonsmooth boundary. This smoothness property is known to be used to classify such systems. One approach, suggested by A. I. Ovseevich in the case of a smooth control set, is based on connecting smoothness of the attainable set with spherical observability of the dual system. This paper generalizes these results to the case of nonsmooth control sets. The corresponding property of the spherical observability notion can be treated as observability with several bearing-only observations. __________ Translated from Fundamentalnaya i Prikladnaya Matematika, Vol. 11, No. 8, pp. 119–130, 2005.     

On the convergence rate of a forward-backward type primal-dual splitting algorithm for convex optimization problems

In this paper, we analyse the convergence rate of the sequence of objective function values of a primal-dual proximal-point algorithm recently introduced in the literature for solving a primal convex optimization problem having as objective the sum of linearly composed infimal convolutions, nonsmooth and smooth convex functions and its Fenchel-type dual one. The theoretical part is illustrated by numerical experiments in image processing.     

On the convergence of the entropy-exponential penalty trajectories and generalized proximal point methods in semidefinite optimization

The convergence of primal and dual central paths associated to entropy and exponential functions, respectively, for semidefinite programming problem are studied in this paper. It is proved that the primal path converges to the analytic center of the primal optimal set with respect to the entropy function, the dual path converges to a point in the dual optimal set and the primal-dual path associated to this paths converges to a point in the primal-dual optimal set. As an application, the generalized proximal point method with the Kullback-Leibler distance applied to semidefinite programming problems is considered. The convergence of the primal proximal sequence to the analytic center of the primal optimal set with respect to the entropy function is established and the convergence of a particular weighted dual proximal sequence to a point in the dual optimal set is obtained.     

Proximal-gradient algorithms for fractional programming

In this paper, we propose two proximal-gradient algorithms for fractional programming problems in real Hilbert spaces, where the numerator is a proper, convex and lower semicontinuous function and the denominator is a smooth function, either concave or convex. In the iterative schemes, we perform a proximal step with respect to the nonsmooth numerator and a gradient step with respect to the smooth denominator. The algorithm in case of a concave denominator has the particularity that it generates sequences which approach both the (global) optimal solutions set and the optimal objective value of the underlying fractional programming problem. In case of a convex denominator the numerical scheme approaches the set of critical points of the objective function, provided the latter satisfies the Kurdyka-?ojasiewicz property.     

Barrier subgradient method

In this paper we develop a new affine-invariant primal–dual subgradient method for nonsmooth convex optimization problems. This scheme is based on a self-concordant barrier for the basic feasible set. It is suitable for finding approximate solutions with certain relative accuracy. We discuss some applications of this technique including fractional covering problem, maximal concurrent flow problem, semidefinite relaxations and nonlinear online optimization. For all these problems, the rate of convergence of our method does not depend on the problem’s data.     

Quasidifferentiability in nonsmooth,nonconvex mechanics

Nonconvex and nonsmooth optimization problems arise in advanced engineering analysis and structural analysis applications. In fact the set of inequality and complementarity relations that describe the structural analysis problem are generated as optimality conditions by the quasidifferential potential energy optimization problem. Thus new kind of variational expressions arise for these problems, which generalize the classical variational equations of smooth mechanics, the variational inequalities of convex, nonsmooth mechanics and give a solid, computationally efficient explication of hemivariational inequalities of nonconvex, nonsmooth mechanics. Moreover quasidifferential calculus and optimization software make this approach applicable for a large number of problems. The connection of quasidifferential optimization and nonsmooth, nonconvex mechanics is discussed in this paper. A number of representative examples from elastostatic analysis applications are treated in details. Numerical examples illustrate the theory.     

A Geometric Integration Approach to Nonsmooth,Nonconvex Optimisation

The optimisation of nonsmooth, nonconvex functions without access to gradients is a particularly challenging problem that is frequently encountered, for example in model parameter optimisation problems. Bilevel optimisation of parameters is a standard setting in areas such as variational regularisation problems and supervised machine learning. We present efficient and robust derivative-free methods called randomised Itoh–Abe methods. These are generalisations of the Itoh–Abe discrete gradient method, a well-known scheme from geometric integration, which has previously only been considered in the smooth setting. We demonstrate that the method and its favourable energy dissipation properties are well defined in the nonsmooth setting. Furthermore, we prove that whenever the objective function is locally Lipschitz continuous, the iterates almost surely converge to a connected set of Clarke stationary points. We present an implementation of the methods, and apply it to various test problems. The numerical results indicate that the randomised Itoh–Abe methods can be superior to state-of-the-art derivative-free optimisation methods in solving nonsmooth problems while still remaining competitive in terms of efficiency.

     

A nonlinear descent method for a variational inequality on a nonconvex set

In this paper, we consider a generalized variational inequality problem which involves the integrable cost mapping and a nonsmooth mapping with convex components. We propose a new gradient-type method which determines a stepsize by using the smooth part of the cost function. Thus, the method does not utilize analogs of derivatives of nonsmooth functions. We show that its convergence does not require additional assumptions.     

New Optimality Conditions for the Semivectorial Bilevel Optimization Problem

The paper is concerned with the optimistic formulation of a bilevel optimization problem with multiobjective lower-level problem. Considering the scalarization approach for the multiobjective program, we transform our problem into a scalar-objective optimization problem with inequality constraints by means of the well-known optimal value reformulation. Completely detailed first-order necessary optimality conditions are then derived in the smooth and nonsmooth settings while using the generalized differentiation calculus of Mordukhovich. Our approach is different from the one previously used in the literature and the conditions obtained are new. Furthermore, they reduce to those of a usual bilevel program, if the lower-level objective function becomes single-valued.