Convergence and Computational Analyses for Some Variable Target Value and Subgradient Deflection Methods

We consider two variable target value frameworks for solving large-scale nondifferentiable optimization problems. We provide convergence analyses for various combinations of these variable target value frameworks with several direction-finding and step-length strategies including the pure subgradient method, the volume algorithm, the average direction strategy, and a generalized Polyak-Kelley cutting plane method. In addition, we suggest a further enhancement via a projected quadratic-fit line-search whenever any of these algorithmic procedures experiences an improvement in the objective value. Extensive computational results on different classes of problems reveal that these modifications and enhancements significantly improve the effectiveness of the algorithms to solve Lagrangian duals of linear programs, even yielding a favorable comparison against the commercial software CPLEX 8.1.     

The prize collecting Steiner tree problem: models and Lagrangian dual optimization approaches

We propose a generalized version of the Prize Collecting Steiner Tree Problem (PCSTP), which offers a fundamental unifying model for several well-known -hard tree optimization problems. The PCSTP also arises naturally in a variety of network design applications including cable television and local access networks. We reformulate the PCSTP as a minimum spanning tree problem with additional packing and knapsack constraints and we explore various nondifferentiable optimization algorithms for solving its Lagrangian dual. We report computational results for nine variants of deflected subgradient strategies, the volume algorithm (VA), and the variable target value method used in conjunction with the VA and with a generalized Polyak–Kelley cutting plane technique. The performance of these approaches is also compared with an exact stabilized constraint generation procedure.     

A Projected Subgradient Method for Nondifferentiable Quasiconvex Multiobjective Optimization Problems

In this paper, we propose a projected subgradient method for solving constrained nondifferentiable quasiconvex multiobjective optimization problems. The algorithm is based on the Plastria subdifferential to overcome potential shortcomings known from algorithms based on the classical gradient. Under suitable, yet rather general assumptions, we establish the convergence of the full sequence generated by the algorithm to a Pareto efficient solution of the problem. Numerical results are presented to illustrate our findings.

     

Making Augmented Lagrangian Methods Computer Amenable for Equilibrium Problems

ABSTRACT

We develop three algorithms to solve the subproblems generated by the augmented Lagrangian methods introduced by Iusem-Nasri (2010) for the equilibrium problem. The first algorithm that we propose incorporates the Newton method and the other two are instances of the subgradient projection method. One of our algorithms is also capable of solving nondifferentiable equilibrium problems. Using well-known test problems, all algorithms introduced here are implemented and numerical results are reported to compare their performances.     

On embedding the volume algorithm in a variable target value method

We employ the volume algorithm as a subgradient deflection strategy in a variable target value method for solving nondifferentiable optimization problems. Focusing on Lagrangian duals for LPs, we exhibit primal nonconvergence of the original method, establish convergence of the proposed algorithm in the dual space, and present related computational results.     

The Euclidean Steiner tree problem in Rn: A mathematical programming formulation

A nonconvex mixed-integer programming formulation for the Euclidean Steiner Tree Problem (ESTP) in Rⁿ is presented. After obtaining separability between integer and continuous variables in the objective function, a Lagrange dual program is proposed. To solve this dual problem (and obtaining a lower bound for ESTP) we use subgradient techniques. In order to evaluate a subgradient at each iteration we have to solve three optimization problems, two in polynomial time, and one is a special convex nondifferentiable programming problem. This revised version was published online in June 2006 with corrections to the Cover Date.     

A class of convergent primal-dual subgradient algorithms for decomposable convex programs

In this paper we develop a primal-dual subgradient algorithm for preferably decomposable, generally nondifferentiable, convex programming problems, under usual regularity conditions. The algorithm employs a Lagrangian dual function along with a suitable penalty function which satisfies a specified set of properties, in order to generate a sequence of primal and dual iterates for which some subsequence converges to a pair of primal-dual optimal solutions. Several classical types of penalty functions are shown to satisfy these specified properties. A geometric convergence rate is established for the algorithm under some additional assumptions. This approach has three principal advantages. Firstly, both primal and dual solutions are available which prove to be useful in several contexts. Secondly, the choice of step sizes, which plays an important role in subgradient optimization, is guided more determinably in this method via primal and dual information. Thirdly, typical subgradient algorithms suffer from the lack of an appropriate stopping criterion, and so the quality of the solution obtained after a finite number of steps is usually unknown. In contrast, by using the primal-dual gap, the proposed algorithm possesses a natural stopping criterion.     

A subgradient method for multiobjective optimization

A method for solving quasiconvex nondifferentiable unconstrained multiobjective optimization problems is proposed in this paper. This method extends to the multiobjective case of the classical subgradient method for real-valued minimization. Assuming the basically componentwise quasiconvexity of the objective components, full convergence (to Pareto optimal points) of all the sequences produced by the method is established.     

基于异步次梯度法的LR算法及其在多阶段HFSP的应用

为降低求解复杂度和缩短计算时间,针对多阶段混合流水车间总加权完成时间问题,提出了一种结合异步次梯度法的改进拉格朗日松弛算法。建立综合考虑有限等待时间和工件释放时间的整数规划数学模型,将异步次梯度法嵌入到拉格朗日松弛算法中,从而通过近似求解拉格朗日松弛问题得到一个合理的异步次梯度方向,沿此方向进行搜索,逐渐降低到最优点的距离。通过仿真实验,验证了所提算法的有效性。对比所提算法与传统的基于次梯度法的拉格朗日松弛算法,结果表明,就综合解的质量和计算效率而言,所提算法能在较短的计算时间内获得更好的近优解,尤其是对大规模问题。     

A subgradient algorithm for certain minimax and minisum problems

We present a subgradient algorithm for minimizing the maximum of a finite collection of functions. It is assumed that each function is the sum of a finite collection of basic convex functions and that the number of different subgradient sets associated with nondifferentiable points of each basic function is finite on any bounded set. Problems belonging to this class include the linear approximation problem and both the minimax and minisum problems of location theory. Convergence of the algorithm to an epsilon-optimal solution is proven and its effectiveness is demonstrated by solving a number of location problems and linear approximation problems.This research was partially supported by the Army Research Office, Triangle Park, NC, under contract number DAH-CO4-75-G-0150, and by NSF grants ENG 16-24294 and ENG 75-10225.     

A Direct Splitting Method for Nonsmooth Variational Inequalities

We propose a direct splitting method for solving a nonsmooth variational inequality in Hilbert spaces. The weak convergence is established when the operator is the sum of two point-to-set and monotone operators. The proposed method is a natural extension of the incremental subgradient method for nondifferentiable optimization, which strongly explores the structure of the operator using projected subgradient-like techniques. The advantage of our method is that any nontrivial subproblem must be solved, like the evaluation of the resolvent operator. The necessity to compute proximal iterations is the main difficulty of other schemes for solving this kind of problem.     

On a space extension algorithm for nondifferentiable optimization

This paper presents a version of an algorithm, due to Shor, for solving unconstrained nondifferentiable optimization problems. The algorithm uses a space extension operator in the direction of the difference between two successive subgradients. The search direction is determined by multiplying the negative of a subgradient by a positive-definite and symmetric matrix. This matrix is updated by a formula similar to the DFP updating formula for differentiable problems. An approximate line search due to Wolfe is presented. A linear rate of convergence of the successive function values is established.     

Portfolio optimization by minimizing conditional value-at-risk via nondifferentiable optimization

Conditional Value-at-Risk (CVaR) is a portfolio evaluation function having appealing features such as sub-additivity and convexity. Although the CVaR function is nondifferentiable, scenario-based CVaR minimization problems can be reformulated as linear programs (LPs) that afford solutions via widely-used commercial softwares. However, finding solutions through LP formulations for problems having many financial instruments and a large number of price scenarios can be time-consuming as the dimension of the problem greatly increases. In this paper, we propose a two-phase approach that is suitable for solving CVaR minimization problems having a large number of price scenarios. In the first phase, conventional differentiable optimization techniques are used while circumventing nondifferentiable points, and in the second phase, we employ a theoretically convergent, variable target value nondifferentiable optimization technique. The resultant two-phase procedure guarantees infinite convergence to optimality. As an optional third phase, we additionally perform a switchover to a simplex solver starting with a crash basis obtained from the second phase when finite convergence to an exact optimum is desired. This three phase procedure substantially reduces the effort required in comparison with the direct use of a commercial stand-alone simplex solver (CPLEX 9.0). Moreover, the two-phase method provides highly-accurate near-optimal solutions with a significantly improved performance over the interior point barrier implementation of CPLEX 9.0 as well, especially when the number of scenarios is large. We also provide some benchmarking results on using an alternative popular proximal bundle nondifferentiable optimization technique.     

Generalized concavity and duality with a square root term

Duality results for a class of nondifferentiable mathematical programming problems are given. These results allow for the weakening of the usual convexity conditions required for duality to hold. A pair of symmetric and self dual nondifferentiable programs under weaker convexity conditions are also given. A subgradient symmetric duality is proposed and its limitations discussed. Finally, a pair of nondifferentiable mathematical programs containing arbitrary norms is presented.     

Inexact subgradient methods for quasi-convex optimization problems

In this paper, we consider a generic inexact subgradient algorithm to solve a nondifferentiable quasi-convex constrained optimization problem. The inexactness stems from computation errors and noise, which come from practical considerations and applications. Assuming that the computational errors and noise are deterministic and bounded, we study the effect of the inexactness on the subgradient method when the constraint set is compact or the objective function has a set of generalized weak sharp minima. In both cases, using the constant and diminishing stepsize rules, we describe convergence results in both objective values and iterates, and finite convergence to approximate optimality. We also investigate efficiency estimates of iterates and apply the inexact subgradient algorithm to solve the Cobb–Douglas production efficiency problem. The numerical results verify our theoretical analysis and show the high efficiency of our proposed algorithm, especially for the large-scale problems.     

A polynomial algorithm for minimum quadratic cost flow problems

Network flow problems with quadratic separable costs appear in a number of important applications such as; approximating input-output matrices in economy; projecting and forecasting traffic matrices in telecommunication networks; solving nondifferentiable cost flow problems by subgradient algorithms. It is shown that the scaling technique introduced by Edmonds and Karp (1972) in the case of linear cost flows for deriving a polynomial complexity bound for the out-of-kilter method, may be extended to quadratic cost flows and leads to a polynomial algorithm for this class of problems. The method may be applied to the solution of singly constrained quadratic programs and thus provides an alternative approach to the polynomial algorithm suggested by Helgason, Kennington and Lall (1980).     

A Subgradient Method Based on Gradient Sampling for Solving Convex Optimization Problems

Based on the gradient sampling technique, we present a subgradient algorithm to solve the nondifferentiable convex optimization problem with an extended real-valued objective function. A feature of our algorithm is the approximation of subgradient at a point via random sampling of (relative) gradients at nearby points, and then taking convex combinations of these (relative) gradients. We prove that our algorithm converges to an optimal solution with probability 1. Numerical results demonstrate that our algorithm performs favorably compared with existing subgradient algorithms on applications considered.     

Abstract convergence theorem for quasi-convex optimization problems with applications

Abstract

Quasi-convex optimization is fundamental to the modelling of many practical problems in various fields such as economics, finance and industrial organization. Subgradient methods are practical iterative algorithms for solving large-scale quasi-convex optimization problems. In the present paper, focusing on quasi-convex optimization, we develop an abstract convergence theorem for a class of sequences, which satisfy a general basic inequality, under some suitable assumptions on parameters. The convergence properties in both function values and distances of iterates from the optimal solution set are discussed. The abstract convergence theorem covers relevant results of many types of subgradient methods studied in the literature, for either convex or quasi-convex optimization. Furthermore, we propose a new subgradient method, in which a perturbation of the successive direction is employed at each iteration. As an application of the abstract convergence theorem, we obtain the convergence results of the proposed subgradient method under the assumption of the Hölder condition of order p and by using the constant, diminishing or dynamic stepsize rules, respectively. A preliminary numerical study shows that the proposed method outperforms the standard, stochastic and primal-dual subgradient methods in solving the Cobb–Douglas production efficiency problem.     

An adaptive finite element method for the sparse optimal control of fractional diffusion

We propose and analyze an a posteriori error estimator for a partial differential equation (PDE)-constrained optimization problem involving a nondifferentiable cost functional, fractional diffusion, and control-constraints. We realize fractional diffusion as the Dirichlet-to-Neumann map for a nonuniformly PDE and propose an equivalent optimal control problem with a local state equation. For such an equivalent problem, we design an a posteriori error estimator which can be defined as the sum of four contributions: two contributions related to the approximation of the state and adjoint equations and two contributions that account for the discretization of the control variable and its associated subgradient. The contributions related to the discretization of the state and adjoint equations rely on anisotropic error estimators in weighted Sobolev spaces. We prove that the proposed a posteriori error estimator is locally efficient and, under suitable assumptions, reliable. We design an adaptive scheme that yields, for the examples that we perform, optimal experimental rates of convergence.     

A modified subgradient extragradient method for solving the variational inequality problem

The subgradient extragradient method for solving the variational inequality (VI) problem, which is introduced by Censor et al. (J. Optim. Theory Appl. 148, 318–335, 2011), replaces the second projection onto the feasible set of the VI, in the extragradient method, with a subgradient projection onto some constructible half-space. Since the method has been introduced, many authors proposed extensions and modifications with applications to various problems. In this paper, we introduce a modified subgradient extragradient method by improving the stepsize of its second step. Convergence of the proposed method is proved under standard and mild conditions and primary numerical experiments illustrate the performance and advantage of this new subgradient extragradient variant.