Two-level primal-dual decomposition technique for large-scale nonconvex optimization problems with constraints

A two-level decomposition method for nonconvex separable optimization problems with additional local constraints of general inequality type is presented and thoroughly analyzed in the paper. The method is of primal-dual type, based on an augmentation of the Lagrange function. Previous methods of this type were in fact three-level, with adjustment of the Lagrange multipliers at one of the levels. This level is eliminated in the present approach by replacing the multipliers by a formula depending only on primal variables and Kuhn-Tucker multipliers for the local constraints. The primal variables and the Kuhn-Tucker multipliers are together the higher-level variables, which are updated simultaneously. Algorithms for this updating are proposed in the paper, together with their convergence analysis, which gives also indications on how to choose penalty coefficients of the augmented Lagrangian. Finally, numerical examples are presented.     

Decomposition in multi-item inventory control

This paper considers a Lagrangian decomposition approach to a stochastic demand multi-item inventory control problem with a single resource constraint. The work is a generalization of existing decomposition methods.Three decomposition methods are proposed, and bounds on the loss of optimality for each are given in terms of the Lagrange multiplier used. One method allows the calculation of the complete decision rule in advance of the realization of the states, but is expected to perform worse than the other two methods. The second and third method allow the determination of decisions as an optimization problem as the states are realized. Since, in any problem with many states, only a small proportion will actually be realized even in a large time-horizon problem, there may be some advantage in taking this approach.     

A Modified Barrier-Augmented Lagrangian Method for Constrained Minimization

We present and analyze an interior-exterior augmented Lagrangian method for solving constrained optimization problems with both inequality and equality constraints. This method, the modified barrier—augmented Lagrangian (MBAL) method, is a combination of the modified barrier and the augmented Lagrangian methods. It is based on the MBAL function, which treats inequality constraints with a modified barrier term and equalities with an augmented Lagrangian term. The MBAL method alternatively minimizes the MBAL function in the primal space and updates the Lagrange multipliers. For a large enough fixed barrier-penalty parameter the MBAL method is shown to converge Q-linearly under the standard second-order optimality conditions. Q-superlinear convergence can be achieved by increasing the barrier-penalty parameter after each Lagrange multiplier update. We consider a dual problem that is based on the MBAL function. We prove a basic duality theorem for it and show that it has several important properties that fail to hold for the dual based on the classical Lagrangian.     

A hybrid approach to 3-level FETI

Finite Element Tearing and Interconnecting (FETI) methods are nonoverlapping domain decomposition methods which have been proven to be very robust and parallel scalable for a class of elliptic partial differential equations. These methods are also called dual domain decomposition methods since the continuity accross the subdomain boundaries is enforced by Lagrange multipliers and, after elimination of the primal variables, the remaining Schur complement system is solved iteratively in the Lagrange multiplier space using a Krylov space method. Domain decomposition methods iterating on the primal variables are called primal substructuring methods. FETI and FETI–DP methods are different members of the family of dual domain decomposition methods. Their standard versions have in common that the local subproblems and a small global problem are solved exactly by a direct method, essentially representing two different levels within the algorithm. Several extensions of dual and primal iterative substructuring beyond two levels have been proposed in the past, see, e.g., [7] for FETI–DP, and, e.g., Tu [13,12,11] or [9] and [1] for BDDC. In the present article, a hybrid FETI/FETI–DP method is considered and some numerical results are presented. It is noted that independently, there is ongoing research on hybrid FETI methods by Jungho Lee of the Courant Institute. (© 2008 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim)     

Analysis on a superlinearly convergent augmented Lagrangian method

The augmented Lagrangian method is a classical method for solving constrained optimization.Recently,the augmented Lagrangian method attracts much attention due to its applications to sparse optimization in compressive sensing and low rank matrix optimization problems.However,most Lagrangian methods use first order information to update the Lagrange multipliers,which lead to only linear convergence.In this paper,we study an update technique based on second order information and prove that superlinear convergence can be obtained.Theoretical properties of the update formula are given and some implementation issues regarding the new update are also discussed.     

Spatial and time decomposition algorithms for dynamic nonlinear network optimization using duality

The structural and computational aspects of two decomposition algorithms suitable for dynamic optimization of nonlinear interconnected networks are examined. Both methods arise from a decomposition based on Lagrangian duality theory of the addressed dynamic optimization problem, which is the minimization of energy costs over a given time period, subject to the requirement that the network equations and inequality restrictions are satisfied. The first algorithm uses a spatial decomposition of the state space into subgroups of state variables associated with particular network zones. This leads to a number of lower-dimensional optimization problems which can be solved individually at one level and coordinated at a higher level to account for interactions between these zones. The second algorithm uses time decomposition to solve a sequence of static optimization problems, one for each time increment into which the interval is subdivided, which are then coordinated to take account of dynamic interaction between the time increments. Computational results from an actual network in the United Kingdom are presented for both methods.     

A nonlinear Lagrangian for constrained optimization problems

A novel nonlinear Lagrangian is presented for constrained optimization problems with both inequality and equality constraints, which is nonlinear with respect to both functions in problem and Lagrange multipliers. The nonlinear Lagrangian inherits the smoothness of the objective and constraint functions and has positive properties. The algorithm on the nonlinear Lagrangian is demonstrated to possess local and linear convergence when the penalty parameter is less than a threshold (the penalty parameter in the penalty method has to approximate zero) under a set of suitable conditions, and be super-linearly convergent when the penalty parameter is decreased following Lagrange multiplier update. Furthermore, the dual problem based on the nonlinear Lagrangian is discussed and some important properties are proposed, which fail to hold for the dual problem based on the classical Lagrangian. At last, the preliminary and comparing numerical results for several typical test problems by using the new nonlinear Lagrangian algorithm and the other two related nonlinear Lagrangian algorithms, are reported, which show that the given nonlinear Lagrangian is promising.     

Lagrangian Relaxation and the Single-Source Capacitated Facility-Location Problem

Lagrangian relaxation has been widely used in solving a number of hard combinatorial optimization problems. The success of the approach depends on the structure of the problem and on the values assigned to the Lagrange multipliers. A recent paper on the single-source capacitated facility-location problem proposed the use of Lagrangian relaxation in which the capacity constraints were relaxed. In this paper, a class of such problems is defined for which the proposed relaxation is guaranteed to result in an infeasible solution, irrespective of the values assigned to the Lagrange multipliers. In these cases, the bounds on the optimal solution, obtained from the relaxation, are generally poor. It is concluded that, when using Lagrangian relaxation, it may be worthwhile carrying out a preliminary analysis to determine the potential viability of the approach before extensive development takes place.     

An Augmented Lagrangian Method for a Class of Inverse Quadratic Programming Problems

We consider an inverse quadratic programming (QP) problem in which the parameters in the objective function of a given QP problem are adjusted as little as possible so that a known feasible solution becomes the optimal one. We formulate this problem as a minimization problem with a positive semidefinite cone constraint and its dual is a linearly constrained semismoothly differentiable (SC¹) convex programming problem with fewer variables than the original one. We demonstrate the global convergence of the augmented Lagrangian method for the dual problem and prove that the convergence rate of primal iterates, generated by the augmented Lagrange method, is proportional to 1/r, and the rate of multiplier iterates is proportional to  $1/\sqrt{r}$ , where r is the penalty parameter in the augmented Lagrangian. As the objective function of the dual problem is a SC¹ function involving the projection operator onto the cone of symmetrically semi-definite matrices, the analysis requires extensive tools such as the singular value decomposition of matrices, an implicit function theorem for semismooth functions, and properties of the projection operator in the symmetric-matrix space. Furthermore, the semismooth Newton method with Armijo line search is applied to solve the subproblems in the augmented Lagrange approach, which is proven to have global convergence and local quadratic rate. Finally numerical results, implemented by the augmented Lagrangian method, are reported.     

Experiments with primal - dual decomposition and subgradient methods for the uncapacitatied facility location problem

For optimization problems that are structured both with respect to the constraints and with respect to the variables, it is possible to use primal–dual solution approaches, based on decomposition principles. One can construct a primal subproblem, by fixing some variables, and a dual subproblem, by relaxing some constraints and king their Lagrange multipliers, so that both these problems are much easier to solve than the original problem. We study methods based on these subproblems, that do not include the difficult Benders or Dantzig-Wolfe master problems, namely primal–dual subgradient optimization methods, mean value cross decomposition, and several comtbinations of the different techniques. In this paper, these solution approaches are applied to the well-known uncapacitated facility location problem. Computational tests show that some combination methods yield near-optimal solutions quicker than the classical dual ascent method of Erlenkotter     

Network planning under uncertainty with an application to hydropower generation

Summary We present a general modeling framework for therobust optimization of linear network problems with uncertainty in the values of the right-hand side. In contrast to traditional approaches in mathematical programming, we use scenarios to characterize the uncertainty. Solutions are obtained for each scenario and these individual scenarios are aggregated to yield a nonanticipative or implementable policy that minimizes the regret of wrong decisions. A given solution is termed robust if it minimizes the sum over the scenarios of the weighted upper difference between the objective function value for the solution and the objective function value for the optimal solution for each scenario, while satisfying certain nonanticipativity constraints. This approach results in a huge model with a network submodel per scenario plus coupling constraints. Several decomposition approaches are considered, namely Dantzig-Wolfe decomposition, various types of Benders decomposition and different quadratic network approaches for approximating Augmented Lagrangian decomposition. We present computational results for these methods, including two implementation versions of the Lagrangian based method: a sequential implementation and a parallel implementation on a network of three workstations.     

Decomposition and Dynamic Cut Generation in Integer Linear Programming

Decomposition algorithms such as Lagrangian relaxation and Dantzig-Wolfe decomposition are well-known methods that can be used to generate bounds for mixed-integer linear programming problems. Traditionally, these methods have been viewed as distinct from polyhedral methods, in which bounds are obtained by dynamically generating valid inequalities to strengthen an initial linear programming relaxation. Recently, a number of authors have proposed methods for integrating dynamic cut generation with various decomposition methods to yield further improvement in computed bounds. In this paper, we describe a framework within which most of these methods can be viewed from a common theoretical perspective. We then discuss how the framework can be extended to obtain a decomposition-based separation technique we call decompose and cut. As a by-product, we describe how these methods can take advantage of the fact that solutions with known structure, such as those to a given relaxation, can frequently be separated much more easily than arbitrary real vectors.     

A second-order optimality condition with first- and second-order complementarity associated with global convergence of algorithms

We develop a new notion of second-order complementarity with respect to the tangent subspace related to second-order necessary optimality conditions by the introduction of so-called tangent multipliers. We prove that around a local minimizer, a second-order stationarity residual can be driven to zero while controlling the growth of Lagrange multipliers and tangent multipliers, which gives a new second-order optimality condition without constraint qualifications stronger than previous ones associated with global convergence of algorithms. We prove that second-order variants of augmented Lagrangian (under an additional smoothness assumption based on the Lojasiewicz inequality) and interior point methods generate sequences satisfying our optimality condition. We present also a companion minimal constraint qualification, weaker than the ones known for second-order methods, that ensures usual global convergence results to a classical second-order stationary point. Finally, our optimality condition naturally suggests a definition of second-order stationarity suitable for the computation of iteration complexity bounds and for the definition of stopping criteria.     

Global properties of integrable Hamiltonian systems

This paper deals with Lagrangian bundles which are symplectic torus bundles that occur in integrable Hamiltonian systems. We review the theory of obstructions to triviality, in particular monodromy, as well as the ensuing classification problems which involve the Chern and Lagrange class. Our approach, which uses simple ideas from differential geometry and algebraic topology, reveals the fundamental role of the integer affine structure on the base space of these bundles. We provide a geometric proof of the classification of Lagrangian bundles with fixed integer affine structure by their Lagrange class.      

Alternating direction splitting for block Angular parallel optimization

We develop and compare three decomposition algorithms derived from the method of alternating directions. They may be viewed as block Gauss-Seidel variants of augmented Lagrangian approaches that take advantage of block angular structure. From a parallel computation viewpoint, they are ideally suited to a data parallel environment. Numerical results for large-scale multicommodity flow problems are presented to demonstrate the effectiveness of these decomposition algorithmims on the Thinking Machines CM-5 parallel supercomputer relative to the widely-used serial optimization package MINOS 5.4.This material is based on research supported by the Air Force Office of Scientific Research, Grants AFORS-89-0410 and F49620-1-0036, and by NSF Grants CCR-89-07671, CDA-90-24618, and CCR-93-06807. The work of the second author was supported partially by Grant 95.00732.CT01 from the Italian National Research Council (CNR).     

A note on augmented Lagrangian-based parallel splitting method

We consider the linearly constrained separable convex minimization problem, whose objective function consists of the sum of \(m\) individual convex functions in the absence of any coupling variables. While augmented Lagrangian-based decomposition methods have been well developed in the literature for solving such problems, a noteworthy requirement of these methods is that an additional correction step is a must to guarantee their convergence. This note shows that a straightforward Jacobian decomposition of the augmented Lagrangian method is globally convergent if the involved functions are further assumed to be strongly convex.     

Augmented Lagrangian Theory,Duality and Decomposition Methods for Variational Inequality Problems

In this paper, we develop the augmented Lagrangian theory and duality theory for variational inequality problems. We propose also decomposition methods based on the augmented Lagrangian for solving complex variational inequality problems with coupling constraints.     

Convergence of an augmented Lagrange algorithm for nonlinear optimizations with second-order cone constraints

An augmented Lagrange algorithm for nonlinear optimizations with second-order cone constraints is proposed based on a Löwner operator associated with a potential function for the optimization problems with inequality constraints. The favorable properties of both the Löwner operator and the corresponding augmented Lagrangian are discussed. And under some mild assumptions, the rate of convergence of the augmented Lagrange algorithm is studied in detail.     

本刊英文版Vol.30(2014),No.1论文摘要

<正>Analysis on a Superlinearly Convergent Augmented Lagrangian Method Ya Xiang YUAN Abstract The augmented Lagrangian method is a classical method for solving constrained optimization.Recently,the augmented Lagrangian method attracts much attention due to its applications to sparse optimization in compressive sensing and low rank matrix optimization problems.However,most Lagrangian methods use first order information to update the Lagrange multipliers,which lead to only linear convergence.In this paper,we study an update technique