CP decomposition for tensors via alternating least squares with QR decomposition

The CP tensor decomposition is used in applications such as machine learning and signal processing to discover latent low-rank structure in multidimensional data. Computing a CP decomposition via an alternating least squares (ALS) method reduces the problem to several linear least squares problems. The standard way to solve these linear least squares subproblems is to use the normal equations, which inherit special tensor structure that can be exploited for computational efficiency. However, the normal equations are sensitive to numerical ill-conditioning, which can compromise the results of the decomposition. In this paper, we develop versions of the CP-ALS algorithm using the QR decomposition and the singular value decomposition, which are more numerically stable than the normal equations, to solve the linear least squares problems. Our algorithms utilize the tensor structure of the CP-ALS subproblems efficiently, have the same complexity as the standard CP-ALS algorithm when the input is dense and the rank is small, and are shown via examples to produce more stable results when ill-conditioning is present. Our MATLAB implementation achieves the same running time as the standard algorithm for small ranks, and we show that the new methods can obtain lower approximation error.     

Experiments with primal - dual decomposition and subgradient methods for the uncapacitatied facility location problem

For optimization problems that are structured both with respect to the constraints and with respect to the variables, it is possible to use primal–dual solution approaches, based on decomposition principles. One can construct a primal subproblem, by fixing some variables, and a dual subproblem, by relaxing some constraints and king their Lagrange multipliers, so that both these problems are much easier to solve than the original problem. We study methods based on these subproblems, that do not include the difficult Benders or Dantzig-Wolfe master problems, namely primal–dual subgradient optimization methods, mean value cross decomposition, and several comtbinations of the different techniques. In this paper, these solution approaches are applied to the well-known uncapacitated facility location problem. Computational tests show that some combination methods yield near-optimal solutions quicker than the classical dual ascent method of Erlenkotter     

A property of certain multistage linear programs and some applications

In this paper, we present a property of certain linear multistage problems. To solve them, a method which takes this property into account is presented. It requires the resolution of 2N–1 subproblems, if there areN stages in the original problem. A sufficient condition is given on the matrix of the constraints for the property to be true. When only a submatrix has this property, we propose to use the Dantzig-Wolfe decomposition principle. We then can solve the subproblem with the proposed method. Applications to linear and nonlinear programming are presented.This work was done while the author was Visiting Scholar at the Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, California.     

A heuristic lagrangean algorithm for the capacitated plant location problem

Lagrangean techniques have been widely applied to the uncapacitated plant location problem, and in some cases they have proven to be successfull even when capacitated problems with additional constraints are taken into account. In our paper we study the application of these techniques to the capacitated plant location problem when the model considered is a pure integer one. Several lagrangean decompositions are considered and for some of them heuristic algorithms have been designed to solve the resulting lagrangean subproblems, the heuristics consisting of a two phase procedure. The first (location phase) defines a set of multipliers from the analysis of the dual LP relaxation, and makes a choice of the plants considering the resulting subproblems as a particular case of the general assignment problems. Several heuristics have been studied for this second phase, based either on a decomposition of knapsack type subproblems through a definition of a set of penalties, or of looking into the duality gap and trying to reduce it. Computational experience is reported.     

Proximal minimizations with D-functions and the massively parallel solution of linear network programs

This paper discusses the massively parallel solution of linear network programs. It integrates the general algorithmic framework of proximal minimization with D-functions (PMD) with primal-dual row-action algorithms. Three alternative algorithmic schemes are studied: quadratic proximal point, entropic proximal point, and least 2-norm perturbations. Each is solving a linear network problem by solving a sequence of nonlinear approximations. The nonlinear subproblems decompose for massively parallel computing. The three algorithms are implemented on a Connection Machine CM-2 with up to 32K processing elements, and problems with up to 16 million variables are solved. A comparison of the three algorithms establishes their relative efficiency. Numerical experiments also establish the best internal tactics which can be used when implementing proximal minimization algorithms. Finally, the new algorithms are compared with an implementation of the network simplex algorithm executing on a CRAY Y-MP vector supercomputer.     

A Decomposition Method for Global and Local Quadratic Minimization

We present a decomposition method for indefinite quadratic programming problems having n variables and m linear constraints. The given problem is decomposed into at most m QP subproblems each having m linear constraints and n-1 variables. All global minima, all isolated local minima and some of the non-isolated local minima for the given problem are obtained from those of the lower dimensional subproblems. One way to continue solving the given problem is to apply the decomposition method again to the subproblems and repeatedly doing so until subproblems of dimension 1 are produced and these can be solved directly. A technique to reduce the potentially large number of subproblems is formulated.     

Outer-approximation algorithms for nonsmooth convex MINLP problems

In this work, we combine outer-approximation (OA) and bundle method algorithms for dealing with mixed-integer non-linear programming (MINLP) problems with nonsmooth convex objective and constraint functions. As the convergence analysis of OA methods relies strongly on the differentiability of the involved functions, OA algorithms may fail to solve general nonsmooth convex MINLP problems. In order to obtain OA algorithms that are convergent regardless the structure of the convex functions, we solve the underlying OA’s non-linear subproblems by a specialized bundle method that provides necessary information to cut off previously visited (non-optimal) integer points. This property is crucial for proving (finite) convergence of OA algorithms. We illustrate the numerical performance of the given proposal on a class of hybrid robust and chance-constrained problems that involve a random variable with finite support.     

Heuristics Based on Partial Enumeration for the Unrelated Parallel Processor Scheduling Problem

The classical deterministic scheduling problem of minimizing the makespan on unrelated parallel processors is known to be NP-hard in the strong sense. Given the mixed integer linear model with binary decision variables, this paper presents heuristic algorithms based on partial enumeration. Basically, they consist in the construction of mixed integer subproblems, considering the integrality of some subset of variables, formulated using the information obtained from the solution of the linear relaxed problem. Computational experiments are reported for a collection of test problems, showing that some of the proposed algorithms achieve better solutions than other relevant approximation algorithms published up to now.     

Medium-term hydro-thermal coordination via hydro and thermal subsystem decomposition

Summary This paper addresses the medium-term hydro-thermal coordination problem in an electric energy system. That is, the problem of finding the energy production of every power plant (hydro or thermal) in every subperiod of a given planning period, so that the customer load is supplied at minimum cost. The planning horizon is typically one to two months and the first week of this planning period is modeled in detail. The solution method proposed decomposes the problem in two subproblems corresponding to the hydro and thermal subsystems. These two subproblems are coordinated using a coordinating function for every subperiod. The coordinating function of a given subperiod expresses total production cost in that subperiod as a function of the total hydro production in that subperiod. The decomposition proposed makes it possible to use specialized algorithms to solve the hydro and thermal subproblems. This results in a very efficient computational procedure. From an experimental point of view the coordinating mechanism is robust. A case study is provided. It considers 61 thermal plants, a hydro system including 8 cascaded hydro plants and a 48 subperiods planning period.     

Parallel bundle-based decomposition for large-scale structured mathematical programming problems

In this paper, we present parallel bundle-based decomposition algorithms to solve a class of structured large-scale convex optimization problems. An example in this class of problems is the block-angular linear programming problem. By dualizing, we transform the original problem to an unconstrained nonsmooth concave optimization problem which is in turn solved by using a modified bundle method. Further, this dual problem consists of a collection of smaller independent subproblems which give rise to the parallel algorithms. We discuss the implementation on the CRYSTAL multi-computer. Finally, we present computational experience with block-angular linear programming problems and observe that more than 70% efficiency can be obtained using up to eleven processors for one group of test problems, and more than 60% efficiency can be obtained for relatively smaller problems using up to five processors for another group of problems.     

The Optimal Preconditioning in the Domain Decomposition Method for Wilson Element

1.TheCollstructionofPreconditionerLetfil)eapolygolldolllaillillR',feL'(fl).Consi(lertheholllogeneousDiricllletboulldaryvalueProblenlofPoissonequation,Assllmethat,fordomainfi,thereareacoarsersubdivisionTHwitllIneshsizeHalldananotheroneThwithmeshsizeh,whichisobtainedbyrefiningTH'Thebotllsubdivisionssatisfythequasi-uniformityandtheillversehypothesis.FOragivenelemelltT,Pm(T)dellotesthespaceofallpolynomialswiththedegreenotgreaterthanm,Qm(T)denotesthespaceofallpolynomialswiththedegreecorres…     

An algorithm using the finite volume element method and its splitting extrapolation

This paper is to present a new efficient algorithm by using the finite volume element method and its splitting extrapolation. This method combines the local conservation property of the finite volume element method and the advantages of splitting extrapolation, such as a high order of accuracy, a high degree of parallelism, less computational complexity and more flexibility than a Richardson extrapolation. Because the splitting extrapolation formulas only require us to solve a set of smaller discrete subproblems on different coarser grids in parallel instead of on the globally fine grid, a large scale multidimensional problem is turned into a set of smaller discrete subproblems. Additionally, this method is efficient for solving interface problems if we regard the interfaces of the problems as the interfaces of the initial domain decomposition.     

A class of Dantzig–Wolfe type decomposition methods for variational inequality problems

We consider a class of decomposition methods for variational inequalities, which is related to the classical Dantzig–Wolfe decomposition of linear programs. Our approach is rather general, in that it can be used with certain types of set-valued or nonmonotone operators, as well as with various kinds of approximations in the subproblems of the functions and derivatives in the single-valued case. Also, subproblems may be solved approximately. Convergence is established under reasonable assumptions. We also report numerical experiments for computing variational equilibria of the game-theoretic models of electricity markets. Our numerical results illustrate that the decomposition approach allows to solve large-scale problem instances otherwise intractable if the widely used PATH solver is applied directly, without decomposition.     

Dynamic Programming Method for Constrained Discrete-Time Optimal Control

A dynamic programming method is presented for solving constrained, discrete-time, optimal control problems. The method is based on an efficient algorithm for solving the subproblems of sequential quadratic programming. By using an interior-point method to accommodate inequality constraints, a modification of an existing algorithm for equality constrained problems can be used iteratively to solve the subproblems. Two test problems and two application problems are presented. The application examples include a rest-to-rest maneuver of a flexible structure and a constrained brachistochrone problem.     

Numerical Comparison of Augmented Lagrangian Algorithms for Nonconvex Problems

Augmented Lagrangian algorithms are very popular tools for solving nonlinear programming problems. At each outer iteration of these methods a simpler optimization problem is solved, for which efficient algorithms can be used, especially when the problems are large. The most famous Augmented Lagrangian algorithm for minimization with inequality constraints is known as Powell-Hestenes-Rockafellar (PHR) method. The main drawback of PHR is that the objective function of the subproblems is not twice continuously differentiable. This is the main motivation for the introduction of many alternative Augmented Lagrangian methods. Most of them have interesting interpretations as proximal point methods for solving the dual problem, when the original nonlinear programming problem is convex. In this paper a numerical comparison between many of these methods is performed using all the suitable problems of the CUTE collection.This author was supported by ProNEx MCT/CNPq/FAPERJ 171.164/2003, FAPESP (Grants 2001/04597-4 and 2002/00094-0 and 2003/09169-6) and CNPq (Grant 302266/2002-0).This author was partially supported by CNPq-Brasil and CDCHT-Venezuela.This author was supported by ProNEx MCT/CNPq/FAPERJ 171.164/2003, FAPESP (Grant 2001/04597-4) and CNPq.     

Decomposition methods for a spatial model for long-term energy pricing problem

We consider an energy production network with zones of production and transfer links. Each zone representing an energy market (a country, part of a country or a set of countries) has to satisfy the local demand using its hydro and thermal units and possibly importing and exporting using links connecting the zones. Assuming that we have the appropriate tools to solve a single zonal problem (approximate dynamic programming, dual dynamic programming, etc.), the proposed algorithm allows us to coordinate the productions of all zones. We propose two reformulations of the dynamic model which lead to different decomposition strategies. Both algorithms are adaptations of known monotone operator splitting methods, namely the alternating direction method of multipliers and the proximal decomposition algorithm which have been proved to be useful to solve convex separable optimization problems. Both algorithms present similar performance in theory but our numerical experimentation on real-size dynamic models have shown that proximal decomposition is better suited to the coordination of the zonal subproblems, becoming a natural choice to solve the dynamic optimization of the European electricity market.     

Arc routing problems with time-dependent service costs

This paper studies an arc routing problem with capacity constraints and time-dependent service costs. This problem is motivated by winter gritting applications where the “timing” of each intervention is crucial. The exact problem-solving approach reported here first transforms the arc routing problem into an equivalent node routing problem. Then, a column generation scheme is used to solve the latter. The master problem is a classical set covering problem, while the subproblems are time-dependent shortest path problems with resource constraints. These subproblems are solved using an extension of a previously developed algorithm. Computational results are reported on problems derived from a set of classical instances of the vehicle routing problem with time windows.     

A new risk criterion in fuzzy environment and its application

Since the observed values of security returns in real-world problems are sometimes imprecise or vague, an increasing effort in research is devoted to study the properties of risk measures in fuzzy portfolio optimization problems. In this paper, a new risk measure is suggested to gauge the risk resulted from fuzzy uncertainty. For this purpose, the absolute deviation and absolute semi-deviation are first defined for fuzzy variable by nonlinear fuzzy integrals. To compute effectively the absolute semi-deviations of single fuzzy variable as well as its functions, this paper discusses the methods of computing the absolute semi-deviation by classical Lebesgue–Stieltjes (L–S) integral. After that, several useful absolute deviation and absolute semi-deviation formulas are established for common triangular, trapezoidal and normal fuzzy variables. Applying the absolute semi-deviation as a new risk measure in portfolio optimization, three classes of fuzzy portfolio optimization models are developed by combining the absolute semi-deviation with expected value operator and credibility measure. Based on the analytical representation of absolute semi-deviations, the established fuzzy portfolio selection models can be turned into their equivalent piecewise linear or fractional programming problems. Since the absolute semi-deviation is a piecewise fractional function and pseudo-convex on the feasible subregions of deterministic programming models, we take advantage of the structural characteristics to design a domain decomposition method to separate a deterministic programming problem into three convex subproblems, which can be solved by conventional solution methods or general-purpose software. Finally, some numerical experiments are performed to demonstrate the new modeling idea and the effectiveness of the solution method.     

A simplicial decomposition algorithm for solving the variational inequality formulation of the general traffic assignment problem for large scale networks

Summary The class of simplicial decomposition methods has been shown to constitute efficient tools for the solution of the variational inequality formulation of the general traffic assignment problem. This paper presents a particular implementation of such an algorithm, with emphasis on its ability to solve large scale problems efficiently. The convergence of the algorithm is monitored by the primal gap function, which arises naturally in simplicial decomposition schemes. The gap function also serves as an instrument for maintaining a reasonable subproblem size, through its use in column dropping criteria. The small dimension and special structure of the subproblems also allows for the use of very efficient algorithms; several algorithms in the class of linearization methods are presented. When restricting the number of retained extremal flows in a simplicial decomposition scheme, the number of major iterations tends to increase. For large networks the shortest path calculations, leading to new extremal flow generation, require a large amount of the total computation time. A special study is therefore made in order to choose the most efficient extremal flow generation technique. Computational results on symmetric problems are presented for networks of some large cities, and on asymmetric problems for some of the networks used in the literature. Computational results for bimodal models of some large cities leading to asymmetric problems are also discussed.