Surrogate Gradient Algorithm for Lagrangian Relaxation

The subgradient method is used frequently to optimize dual functions in Lagrangian relaxation for separable integer programming problems. In the method, all subproblems must be solved optimally to obtain a subgradient direction. In this paper, the surrogate subgradient method is developed, where a proper direction can be obtained without solving optimally all the subproblems. In fact, only an approximate optimization of one subproblem is needed to get a proper surrogate subgradient direction, and the directions are smooth for problems of large size. The convergence of the algorithm is proved. Compared with methods that take effort to find better directions, this method can obtain good directions with much less effort and provides a new approach that is especially powerful for problems of very large size.     

On the Surrogate Gradient Algorithm for Lagrangian Relaxation

When applied to large-scale separable optimization problems, the recently developed surrogate subgradient method for Lagrangian relaxation (Zhao et al.: J. Optim. Theory Appl. 100, 699–712, 1999) does not need to solve optimally all the subproblems to update the multipliers, as the traditional subgradient method requires. Based on it, the penalty surrogate subgradient algorithm was further developed to address the homogenous solution issue (Guan et al.: J. Optim. Theory Appl. 113, 65–82, 2002; Zhai et al.: IEEE Trans. Power Syst. 17, 1250–1257, 2002). There were flaws in the proofs of Zhao et al., Guan et al., and Zhai et al.: for problems with inequality constraints, projection is necessary to keep the multipliers nonnegative; however, the effects of projection were not properly considered. This note corrects the flaw, completes the proofs, and asserts the correctness of the methods. This work is supported by the NSFC Grant Nos. 60274011, 60574067, the NCET program (No. NCET-04-0094) of China. The third author was supported in part by US National Science Foundation under Grants ECS-0323685 and DMI-0423607.     

Comments on “Surrogate Gradient Algorithm for Lagrangian Relaxation”

This note presents not only a surrogate subgradient method, but also a framework of surrogate subgradient methods. Furthermore, the framework can be used not only for separable problems, but also for coupled subproblems. The note delineates such a framework and shows that the algorithm can converges for a larger stepsize. The author thanks Professor Ching-An Lin from the Department of Electrical and Control Engineering of National Chiao Tung University, Hsinchu, Taiwan for valuable discussions.     

Using Surrogate Constraints in a Lagrangian Relaxation Approach to Set-Covering Problems

Lagrangian relaxations have been used in a variety of IP problem settings. The main thrust of such efforts is to obtain bounding information for use in a branch-and-bound procedure. This paper examines the effect of adding a single surrogate constraint to Lagrangian subproblems in an attempt to improve upon the bounds produced by conventional Lagrangian relaxation. Computational results on some randomly generated set-covering problems are reported.     

Convergence Rate of the Augmented Lagrangian SQP Method

In this paper, the augmented Lagrangian SQP method is considered for the numerical solution of optimization problems with equality constraints. The problem is formulated in a Hilbert space setting. Since the augmented Lagrangian SQP method is a type of Newton method for the nonlinear system of necessary optimality conditions, it is conceivable that q-quadratic convergence can be shown to hold locally in the pair (x, ). Our interest lies in the convergence of the variable x alone. We improve convergence estimates for the Newton multiplier update which does not satisfy the same convergence properties in x as for example the least-square multiplier update. We discuss these updates in the context of parameter identification problems. Furthermore, we extend the convergence results to inexact augmented Lagrangian methods. Numerical results for a control problem are also presented.     

求解二次分配问题的拉格朗日松弛新方法

以改进的拉格朗日松弛(Lagrangian relaxation,LR)方法和二次分配问题(quadratic assignment problem,QAP)的线性化模型为基础,给出了求解QAP的拉格朗日松弛新方法,这为有效求解QAP提供了一种新的解决方案.通过求解二次分配基准问题库(QAPLIB)中的实际算例,从实验的角度说明了拉格朗日松弛新方法求解QAP的可行性及存在的不足之处,并对今后进一步的研究工作指明了方向.     

Convergence of the Augmented Lagrangian Method for Nonlinear Optimization Problems over Second-Order Cones

The paper analyzes the rate of local convergence of the augmented Lagrangian method for nonlinear second-order cone optimization problems. Under the constraint nondegeneracy condition and the strong second order sufficient condition, we demonstrate that the sequence of iterate points generated by the augmented Lagrangian method locally converges to a local minimizer at a linear rate, whose ratio constant is proportional to 1/τ with penalty parameter τ not less than a threshold . Importantly and interestingly enough, the analysis does not require the strict complementarity condition. Supported by the National Natural Science Foundation of China under Project 10771026 and by the Scientific Research Foundation for the Returned Overseas Chinese Scholars, State Education Ministry.     

基于异步次梯度法的LR算法及其在多阶段HFSP的应用

为降低求解复杂度和缩短计算时间,针对多阶段混合流水车间总加权完成时间问题,提出了一种结合异步次梯度法的改进拉格朗日松弛算法。建立综合考虑有限等待时间和工件释放时间的整数规划数学模型,将异步次梯度法嵌入到拉格朗日松弛算法中,从而通过近似求解拉格朗日松弛问题得到一个合理的异步次梯度方向,沿此方向进行搜索,逐渐降低到最优点的距离。通过仿真实验,验证了所提算法的有效性。对比所提算法与传统的基于次梯度法的拉格朗日松弛算法,结果表明,就综合解的质量和计算效率而言,所提算法能在较短的计算时间内获得更好的近优解,尤其是对大规模问题。     

Lagrangian Relaxation and the Single-Source Capacitated Facility-Location Problem

Lagrangian relaxation has been widely used in solving a number of hard combinatorial optimization problems. The success of the approach depends on the structure of the problem and on the values assigned to the Lagrange multipliers. A recent paper on the single-source capacitated facility-location problem proposed the use of Lagrangian relaxation in which the capacity constraints were relaxed. In this paper, a class of such problems is defined for which the proposed relaxation is guaranteed to result in an infeasible solution, irrespective of the values assigned to the Lagrange multipliers. In these cases, the bounds on the optimal solution, obtained from the relaxation, are generally poor. It is concluded that, when using Lagrangian relaxation, it may be worthwhile carrying out a preliminary analysis to determine the potential viability of the approach before extensive development takes place.     

线性随机微分方程的离散波形松弛方法的几乎必然收敛性

提出了随机微分方程的离散型波形松弛方法,证明了它是几乎必然收敛的.此外,通过数值实验验证了所得结果.     

关于USSOR迭代法收敛的充要条件和最佳松弛因子

本文在线性方程组系数矩阵A为相容次序矩阵及A的Jacobi迭代矩阵的特征值μj均为实数的条件下,得出了USSOR迭代法收敛的充分必要性定理.并给出了USSOR迭代矩阵之谱半径ρ(ψω,-ω)的表达式及ρ(ψω,-ω)的最佳松弛因子.     

Convergence Properties of the Successive Extrapolated Relaxation (S.E.R.) Method

Extrapolation methods to accelerate convergence of a sequenceof iterates are investigated. A transformation formula derivedfrom the related deterministic sequence is modified so thatit may be used for the stochastic sequences. The S.E.R. method,which is related to Aitken's ² process, is discussed. For linearlyconvergent sequences it is shown that S.E.R. not only will convergeif the original sequence converges, but will converge to thesame limit. An analysis of the bounds for the convergence andthe perturbations is made for Aitken's ² process, S.E.R. andS.E.O.R. The method is applicable to convergent and locallyconvergent vector sequences.     

Constraint Programming Based Lagrangian Relaxation for the Automatic Recording Problem

Whereas CP methods are strong with respect to the detection of local infeasibilities, OR approaches have powerful optimization abilities that ground on tight global bounds on the objective. An integration of propagation ideas from CP and Lagrangian relaxation techniques from OR combines the merits of both approaches. We introduce a general way of how linear optimization constraints can strengthen their propagation abilities via Lagrangian relaxation. The method is evaluated on a set of benchmark problems stemming from a multimedia application. The experiments show the superiority of the combined method compared with a pure OR approach and an algorithm based on two independent optimization constraints.     

On the Global Convergence of a Modified Augmented Lagrangian Linesearch Interior-Point Newton Method for Nonlinear Programming

We consider a linesearch globalization of the local primal-dual interior-point Newton method for nonlinear programming introduced by El-Bakry, Tapia, Tsuchiya, and Zhang. The linesearch uses a new merit function that incorporates a modification of the standard augmented Lagrangian function and a weak notion of centrality. We establish a global convergence theory and present promising numerical experimentation.     

New Bundle Methods for Solving Lagrangian Relaxation Dual Problems

Bundle methods have been used frequently to solve nonsmooth optimization problems. In these methods, subgradient directions from past iterations are accumulated in a bundle, and a trial direction is obtained by performing quadratic programming based on the information contained in the bundle. A line search is then performed along the trial direction, generating a serious step if the function value is improved by or a null step otherwise. Bundle methods have been used to maximize the nonsmooth dual function in Lagrangian relaxation for integer optimization problems, where the subgradients are obtained by minimizing the performance index of the relaxed problem. This paper improves bundle methods by making good use of near-minimum solutions that are obtained while solving the relaxed problem. The bundle information is thus enriched, leading to better search directions and less number of null steps. Furthermore, a simplified bundle method is developed, where a fuzzy rule is used to combine linearly directions from near-minimum solutions, replacing quadratic programming and line search. When the simplified bundle method is specialized to an important class of problems where the relaxed problem can be solved by using dynamic programming, fuzzy dynamic programming is developed to obtain efficiently near-optimal solutions and their weights for the linear combination. This method is then applied to job shop scheduling problems, leading to better performance than previously reported in the literature.     

Dynamic Slope Scaling Procedure and Lagrangian Relaxation with Subproblem Approximation

The dynamic slope scaling procedure (DSSP) is an efficient heuristic algorithm that provides good solutions to the fixed-charge transportation or network flow problem. However, the procedure is graphically motivated and appears unrelated to other optimization techniques. In this paper, we formulate the fixed-charge problem as a mathematical program with complementarity constraints (MPCC) and show that DSSP is equivalent to solving MPCC using Lagrangian relaxation with subproblem approximation.     

Convergence of augmented Lagrangian methods in extensions beyond nonlinear programming

Mathematical Programming - The augmented Lagrangian method (ALM) is extended to a broader-than-ever setting of generalized nonlinear programming in convex and nonconvex optimization that is capable...     

A Lagrangian Relaxation Heuristic for Capacitated Facility Location with Single-Source Constraints

Facility location models are applicable to problems in many diverse areas, such as distribution systems and communication networks. In capacitated facility location problems, a number of facilities with given capacities must be chosen from among a set of possible facility locations and then customers assigned to them. We describe a Lagrangian relaxation heuristic algorithm for capacitated problems in which each customer is served by a single facility. By relaxing the capacity constraints, the uncapacitated facility location problem is obtained as a subproblem and solved by the well-known dual ascent algorithm. The Lagrangian relaxations are complemented by an add heuristic, which is used to obtain an initial feasible solution. Further, a final adjustment heuristic is used to attempt to improve the best solution generated by the relaxations. Computational results are reported on examples generated from the Kuehn and Hamburger test problems.