Two-level primal–dual proximal decomposition technique to solve large scale optimization problems

We describe a primal–dual application of the proximal point algorithm to nonconvex minimization problems. Motivated by the work of Spingarn and more recently by the work of Hamdi et al. about the primal resource-directive decomposition scheme to solve nonlinear separable problems. This paper discusses some local results of a primal–dual regularization approach that leads to a decomposition algorithm.     

Stepwise decomposition approaches for large scale cell formation problems

Cell formation is one of the major steps in cellular manufacturing system (CMS) design. In this paper, two stepwise decomposition approaches are proposed to solve large scale industrial problems. Both of them analyze the part-machine relations, decompose the original system to several large subsystems and then use an optimal solution technique to solve each. Several results are proved to show the conditions under which optimal solutions are obtained.     

Solving mixed-integer robust optimization problems with interval uncertainty using Benders decomposition

Uncertainty and integer variables often exist together in economics and engineering design problems. The goal of robust optimization problems is to find an optimal solution that has acceptable sensitivity with respect to uncertain factors. Including integer variables with or without uncertainty can lead to formulations that are computationally expensive to solve. Previous approaches for robust optimization problems under interval uncertainty involve nested optimization or are not applicable to mixed-integer problems where the objective or constraint functions are neither quadratic, nor linear. The overall objective in this paper is to present an efficient robust optimization method that does not contain nested optimization and is applicable to mixed-integer problems with quasiconvex constraints (? type) and convex objective funtion. The proposed method is applied to a variety of numerical examples to test its applicability and numerical evidence is provided for convergence in general as well as some theoretical results for problems with linear constraints.     

Regularized robust optimization: the optimal portfolio execution case

An uncertainty set is a crucial component in robust optimization. Unfortunately, it is often unclear how to specify it precisely. Thus it is important to study sensitivity of the robust solution to variations in the uncertainty set, and to develop a method which improves stability of the robust solution. In this paper, to address these issues, we focus on uncertainty in the price impact parameters in an optimal portfolio execution problem. We first illustrate that a small variation in the uncertainty set may result in a large change in the robust solution. We then propose a regularized robust optimization formulation which yields a solution with a better stability property than the classical robust solution. In this approach, the uncertainty set is regularized through a regularization constraint, defined by a linear matrix inequality using the Hessian of the objective function and a regularization parameter. The regularized robust solution is then more stable with respect to variation in the uncertainty set specification, in addition to being more robust to estimation errors in the price impact parameters. The regularized robust optimal execution strategy can be computed by an efficient method based on convex optimization. Improvement in the stability of the robust solution is analyzed. We also study implications of the regularization on the optimal execution strategy and its corresponding execution cost. Through the regularization parameter, one can adjust the level of conservatism of the robust solution.     

Regularized optimization methods for convex MINLP problems

We propose regularized cutting-plane methods for solving mixed-integer nonlinear programming problems with nonsmooth convex objective and constraint functions. The given methods iteratively search for trial points in certain localizer sets, constructed by employing linearizations of the involved functions. New trial points can be chosen in several ways; for instance, by minimizing a regularized cutting-plane model if functions are costly. When dealing with hard-to-evaluate functions, the goal is to solve the optimization problem by performing as few function evaluations as possible. Numerical experiments comparing the proposed algorithms with classical methods in this area show the effectiveness of our approach.     

Monotone and nonmonotone trust-region-based algorithms for large scale unconstrained optimization problems

Two trust regions algorithms for unconstrained nonlinear optimization problems are presented: a monotone and a nonmonotone one. Both of them solve the trust region subproblem by the spectral projected gradient (SPG) method proposed by Birgin, Martínez and Raydan (in SIAM J. Optim. 10(4):1196?C1211, 2000). SPG is a nonmonotone projected gradient algorithm for solving large-scale convex-constrained optimization problems. It combines the classical projected gradient method with the spectral gradient choice of steplength and a nonmonotone line search strategy. The simplicity (only requires matrix-vector products, and one projection per iteration) and rapid convergence of this scheme fits nicely with globalization techniques based on the trust region philosophy, for large-scale problems. In the nonmonotone algorithm the trial step is evaluated by acceptance via a rule which can be considered a generalization of the well known fraction of Cauchy decrease condition and a generalization of the nonmonotone line search proposed by Grippo, Lampariello and Lucidi (in SIAM J. Numer. Anal. 23:707?C716, 1986). Convergence properties and extensive numerical results are presented. Our results establish the robustness and efficiency of the new algorithms.     

A Newton inexact interior-point method for large scale nonlinear optimization problems

In this paper, we describe a variant of the Newton Interior-Point method in [8] for nonlinear programming problems. In this scheme, the perturbation parameter can be chosen within a range of, values and we can use an iterative method for approximately solving the reduced linear system arising at each step. We have devised the inner termination rule which guarantees the global convergence of this Newton Inexact Interior-Point method. We remark that the required assumptions are weaker than those stated in [8], as shown by some numerical examples. This research was supported by the Italian Ministry for Education, University and Research (MIUR), FIRB Project No. RBAU01JYPN.     

Likelihood robust optimization for data-driven problems

We consider optimal decision-making problems in an uncertain environment. In particular, we consider the case in which the distribution of the input is unknown, yet there is some historical data drawn from the distribution. In this paper, we propose a new type of distributionally robust optimization model called the likelihood robust optimization (LRO) model for this class of problems. In contrast to previous work on distributionally robust optimization that focuses on certain parameters (e.g., mean, variance, etc.) of the input distribution, we exploit the historical data and define the accessible distribution set to contain only those distributions that make the observed data achieve a certain level of likelihood. Then we formulate the targeting problem as one of optimizing the expected value of the objective function under the worst-case distribution in that set. Our model avoids the over-conservativeness of some prior robust approaches by ruling out unrealistic distributions while maintaining robustness of the solution for any statistically likely outcomes. We present statistical analyses of our model using Bayesian statistics and empirical likelihood theory. Specifically, we prove the asymptotic behavior of our distribution set and establish the relationship between our model and other distributionally robust models. To test the performance of our model, we apply it to the newsvendor problem and the portfolio selection problem. The test results show that the solutions of our model indeed have desirable performance.     

Quadratic optimization problems in robust beamforming

This paper presents a class of constrained optimization problems whereby a quadratic cost function is to be minimized with respect to a weight vector subject to an inequality quadratic constraint on the weight vector. This class of constrained optimization problems arises as a result of a motivation for designing robust antenna array processors in the field of adaptive array processing. The constrained optimization problem is first solved by using the primal-dual method. Numerical techniques are presented to reduce the computational complexity of determining the optimal Lagrange multiplier and hence the optimal weight vector. Subsequently, a set of linear constraints or at most linear plus norm constraints are developed for approximating the performance achievable with the quadratic constraint. The use of linear constraints is very attractive, since they reduce the computational burden required to determine the optimal weight vector.     

Efficient Kriging-based robust optimization of unconstrained problems

A novel methodology, based on Kriging and expected improvement, is proposed for applying robust optimization on unconstrained problems affected by implementation error. A modified expected improvement measure which reflects the need for robust instead of nominal optimization is used to provide new sampling point locations. A new sample is added at each iteration by finding the location at which the modified expected improvement measure is maximum. By means of this process, the algorithm iteratively progresses towards the robust optimum. It is demonstrated that the algorithm performs significantly better than current techniques for robust optimization using response surface modeling.     

Global optimization of robust chance constrained problems

We propose a stochastic algorithm for the global optimization of chance constrained problems. We assume that the probability measure with which the constraints are evaluated is known only through its moments. The algorithm proceeds in two phases. In the first phase the probability distribution is (coarsely) discretized and solved to global optimality using a stochastic algorithm. We only assume that the stochastic algorithm exhibits a weak* convergence to a probability measure assigning all its mass to the discretized problem. A diffusion process is derived that has this convergence property. In the second phase, the discretization is improved by solving another nonlinear programming problem. It is shown that the algorithm converges to the solution of the original problem. We discuss the numerical performance of the algorithm and its application to process design.     

Ambiguous chance constrained problems and robust optimization

In this paper we study ambiguous chance constrained problems where the distributions of the random parameters in the problem are themselves uncertain. We focus primarily on the special case where the uncertainty set of the distributions is of the form where ρ_p denotes the Prohorov metric. The ambiguous chance constrained problem is approximated by a robust sampled problem where each constraint is a robust constraint centered at a sample drawn according to the central measure The main contribution of this paper is to show that the robust sampled problem is a good approximation for the ambiguous chance constrained problem with a high probability. This result is established using the Strassen-Dudley Representation Theorem that states that when the distributions of two random variables are close in the Prohorov metric one can construct a coupling of the random variables such that the samples are close with a high probability. We also show that the robust sampled problem can be solved efficiently both in theory and in practice. Research partially supported by NSF grant CCR-00-09972. Research partially supported by NSF grants CCR-00-09972, DMS-01-04282, and ONR grant N000140310514.     

Duality for constrained robust sum optimization problems

Mathematical Programming - Given an infinite family of extended real-valued functions $$f_{i}$$ , $$i\in I,$$ and a family $${\mathcal {H}}$$ of nonempty finite subsets of I,  the...     

On large scale nonlinear Network optimization

Partial separability and partitioned quasi-Newton updating have been recently introduced and experimented with success in large scale nonlinear optimization, large nonlinear least squares calculations and in large systems of nonlinear equations. It is the purpose of this paper to apply this idea to large dimensional nonlinear network optimization problems. The method proposed thus uses these techniques for handling the cost function, while more classical tools as variable partitioning and specialized data structures are used in handling the network constraints. The performance of a code implementing this method, as well as more classical techniques, is analyzed on several numerical examples.     

Solving large scale crew scheduling problems

The crew pairing problem is posed as a set partitioning zero-one integer program. Variables are generated as legal pairings meeting all work rules. Dual values obtained from solving successive large linear program relaxations are used to prune the search tree. In this paper we present a graph based branching heuristic applied to a restricted set partitioning problem representing a collection of ‘best’ pairings. The algorithm exploits the natural integer properties of the crew pairing problem. Computational results are presented to show realized crew cost savings.     

Regularized Lagrangian duality for linearly constrained quadratic optimization and trust-region problems

In this paper we first establish a Lagrange multiplier condition characterizing a regularized Lagrangian duality for quadratic minimization problems with finitely many linear equality and quadratic inequality constraints, where the linear constraints are not relaxed in the regularized Lagrangian dual. In particular, in the case of a quadratic optimization problem with a single quadratic inequality constraint such as the linearly constrained trust-region problems, we show that the Slater constraint qualification (SCQ) is necessary and sufficient for the regularized Lagrangian duality in the sense that the regularized duality holds for each quadratic objective function over the constraints if and only if (SCQ) holds. A new theorem of the alternative for systems involving both equality constraints and two quadratic inequality constraints plays a key role. We also provide classes of quadratic programs, including a class of CDT-subproblems with linear equality constraints, where (SCQ) ensures regularized Lagrangian duality.     

Exact relaxations for parametric robust linear optimization problems

We first show that the closedness of the characteristic cone of the constraint system of a parametric robust linear optimization problem is a necessary and sufficient condition for each robust linear program with the finite optimal value to admit exact semidefinite linear programming relaxations. We then provide the weakest regularity condition that guarantees exact second-order cone programming relaxations for parametric robust linear programs.     

Variable-sized uncertainty and inverse problems in robust optimization

In robust optimization, the general aim is to find a solution that performs well over a set of possible parameter outcomes, the so-called uncertainty set. In this paper, we assume that the uncertainty size is not fixed, and instead aim at finding a set of robust solutions that covers all possible uncertainty set outcomes. We refer to these problems as robust optimization with variable-sized uncertainty. We discuss how to construct smallest possible sets of min–max robust solutions and give bounds on their size.A special case of this perspective is to analyze for which uncertainty sets a nominal solution ceases to be a robust solution, which amounts to an inverse robust optimization problem. We consider this problem with a min–max regret objective and present mixed-integer linear programming formulations that can be applied to construct suitable uncertainty sets.Results on both variable-sized uncertainty and inverse problems are further supported with experimental data.     

Auxiliary problem principle and decomposition of optimization problems

The auxiliary problem principle allows one to find the solution of a problem (minimization problem, saddle-point problem, etc.) by solving a sequence of auxiliary problems. There is a wide range of possible choices for these problems, so that one can give special features to them in order to make them easier to solve. We introduced this principle in Ref. 1 and showed its relevance to decomposing a problem into subproblems and to coordinating the subproblems. Here, we derive several basic or abstract algorithms, already given in Ref. 1, and we study their convergence properties in the framework of i infinite-dimensional convex programming.     

The robust binomial approach to chance-constrained optimization problems with application to stochastic partitioning of large process networks

In this paper, we study an interpretation of the sample-based approach to chance-constrained programming problems grounded in statistical testing theory. On top of being simple and pragmatic, this approach is theoretically well founded, non parametric and leads to a general method for leveraging existing heuristic algorithms for the deterministic case to their chance-constrained counterparts. Throughout this paper, this algorithm design approach is illustrated on a real world graph partitioning problem which crops up in the field of compilation for parallel systems. Extensive computational results illustrate the practical relevance of the approach, as well as the robustness of the obtained solutions.