Parallel computing in nonconvex programming

In this paper, we are concerned with the development of parallel algorithms for solving some classes of nonconvex optimization problems. We present an introductory survey of parallel algorithms that have been used to solve structured problems (partially separable, and large-scale block structured problems), and algorithms based on parallel local searches for solving general nonconvex problems. Indefinite quadratic programming posynomial optimization, and the general global concave minimization problem can be solved using these approaches. In addition, for the minimum concave cost network flow problem, we are going to present new parallel search algorithms for large-scale problems. Computational results of an efficient implementation on a multi-transputer system will be presented.     

STORE AND GENERATION OF LARGE-SCALE PRIME LIST ON THE COMPUTER

A scheme of very compact store for large-scale prime list is given and two algorithms for rapid generating the list are provided.     

Parallel Synchronous and Asynchronous Space-Decomposition Algorithms for Large-Scale Minimization Problems

Three parallel space-decomposition minimization (PSDM) algorithms, based on the parallel variable transformation (PVT) and the parallel gradient distribution (PGD) algorithms (O.L. Mangasarian, SIMA Journal on Control and Optimization, vol. 33, no. 6, pp. 1916–1925.), are presented for solving convex or nonconvex unconstrained minimization problems. The PSDM algorithms decompose the variable space into subspaces and distribute these decomposed subproblems among parallel processors. It is shown that if all decomposed subproblems are uncoupled of each other, they can be solved independently. Otherwise, the parallel algorithms presented in this paper can be used. Numerical experiments show that these parallel algorithms can save processor time, particularly for medium and large-scale problems. Up to six parallel processors are connected by Ethernet networks to solve four large-scale minimization problems. The results are compared with those obtained by using sequential algorithms run on a single processor. An application of the PSDM algorithms to the training of multilayer Adaptive Linear Neurons (Madaline) and a new parallel architecture for such parallel training are also presented.     

求解中大规模复杂凸二次整数规划问题的新型分枝定界算法

针对现有分枝定界算法在求解高维复杂二次整数规划问题时所存在的诸多不足，本文通过充分挖掘二次整数规划问题的结构特性来设计选择分枝变量与分枝方向的新方法，并将HNF算法与原问题松弛问题的求解相结合来寻求较好的初始整数可行解，由此导出可用于有效求解中大规模复杂二次整数规划问题的改进型分枝定界算法．数值试验结果表明所给算法大大改进了已有相关的分枝定界算法，并具有较好的稳定性与广泛的适用性．     

Automatic identification of embedded network rows in large-scale optimization models

The solution of a large-scale linear, integer, or mixed integer programming problem is often facilitated by the exploitation of special structure in the model. This paper presents heuristic algorithms for identifying embedded network rows within the coefficient matrix of such models. The problem of identifying a maximum-size embedded pure network is shown to be among the class of NP-hard problems. The polynomially-bounded, efficient algorithms presented here do not guarantee network sets of maximum size. However, upper bounds on the size of the maximum network set are developed and used to show that our algorithms identify embedded networks of close to maximum size. Computational tests with large-scale, real-world models are presented.     

分支定界算法求解带有释放时间的单机双代理调度问题

本文研究了带有释放时间的单机双代理调度问题,目标函数为极小化最大完工时间和。为了便于利用优化软件求解,建立了混合整数规划模型。考虑到该问题具有NP困难性,因此采用近似与精确算法分别求解不同规模问题。针对大规模问题,提出了优势代理优先启发式算法,并证明了其渐近最优性。针对小规模问题,设计了分支定界法进行最优求解,其中基于释放时间的分支规则和基于加工中断的下界有效地减少了运算时间。最后,通过数值测试验证了分支定界算法的有效性以及启发式算法的收敛性。     

Parallel optimization for traffic assignment

Most large-scale optimization problems exhibit structures that allow the possibility of attack via algorithms that exhibit a high level of parallelism. The emphasis of this paper is the development of parallel optimization algorithms for a class of convex, block-structured problems. Computational experience is cited for some large-scale problems arising from traffic assignment applications. The algorithms considered here have the property that they allow such problems to be decomposed into a set of smaller optimization problems at each major iteration. These smaller problems correspond to linear single-commodity networks in the traffic assignment case, and they may be solved in parallel. Results are given for the distributed solution of such problems on the CRYSTAL multicomputer.This research was supported in part by NSF grant CCR-8709952 and AFOSR grant AFOSR-86-0194.     

An evaluation of mathematical programming and minicomputers

The availability of efficient mathematical software on minicomputers could greatly increase the use of operations research techniques in industry and government. The objective of this paper is to demonstrate the feasibility of implementing a particular class of mathematical programming algorithms, namely shortest path algorithms, on “typical” minicomputers. Two distinct shortest path algorithms were tested on four computer systems using a common set of test problems. Computational results are presented which verify the feasibility of implementing these algorithms in a minicomputer environment, and also show the relative efficiency of each algorithm to be the same when tested on a minicomputer as when tested on a large-scale computer system.     

Facilitating analysis of Monte Carlo dense matrix inversion algorithm scaling behaviour through simulation

With the latest developments in the area of advanced computer architectures, we are already seeing large-scale machines at petascale level and are faced with the exascale computing challenge. All these require scalability at system, algorithmic and mathematical model levels. In particular, efficient scalable algorithms are required to bridge the performance gap. Being able to predict application demeanour, performance and scalability of currently used software on new supercomputers of different architectures, varying sizes, and utilising distinct ways of intercommunication, can be of great benefit for researchers as well as application developers. This paper is concerned with scaling characteristics of Monte Carlo based algorithms for matrix inversion. The algorithmic behaviour on both, a shared memory and a large-scale cluster system will be predicted with the help of an extreme-scale high-performance computing (HPC) simulator.     

On the choice of explicit stabilizing terms in column generation

Column generation algorithms are instrumental in many areas of applied optimization, where linear programs with an enormous number of columns need to be solved. Although successfully employed in many applications, these approaches suffer from well-known instability issues that somewhat limit their efficiency. Building on the theory developed for nondifferentiable optimization algorithms, a large class of stabilized column generation algorithms can be defined which avoid the instability issues by using an explicit stabilizing term in the dual; this amounts at considering a (generalized) augmented Lagrangian of the primal master problem. Since the theory allows for a great degree of flexibility in the choice and in the management of the stabilizing term, one can use piecewise-linear or quadratic functions that can be efficiently dealt with using off-the-shelf solvers. The practical effectiveness of this approach is demonstrated by extensive computational experiments on large-scale Vehicle and Crew Scheduling problems. Also, the results of a detailed computational study on the impact of the different choices in the stabilization term (shape of the function, parameters), and their relationships with the quality of the initial dual estimates, on the overall effectiveness of the approach are reported, providing practical guidelines for selecting the most appropriate variant in different situations.     

Integrating memetic search into the BioHEL evolutionary learning system for large-scale datasets

Local search methods are widely used to improve the performance of evolutionary computation algorithms in all kinds of domains. Employing advanced and efficient exploration mechanisms becomes crucial in complex and very large (in terms of search space) problems, such as when employing evolutionary algorithms to large-scale data mining tasks. Recently, the GAssist Pittsburgh evolutionary learning system was extended with memetic operators for discrete representations that use information from the supervised learning process to heuristically edit classification rules and rule sets. In this paper we first adapt some of these operators to BioHEL, a different evolutionary learning system applying the iterative learning approach, and afterwards propose versions of these operators designed for continuous attributes and for dealing with noise. The performance of all these operators and their combination is extensively evaluated on a broad range of synthetic large-scale datasets to identify the settings that present the best balance between efficiency and accuracy. Finally, the identified best configurations are compared with other classes of machine learning methods on both synthetic and real-world large-scale datasets and show very competent performance.     

Randomized first order algorithms with applications to ℓ 1-minimization

In this paper we propose randomized first-order algorithms for solving bilinear saddle points problems. Our developments are motivated by the need for sublinear time algorithms to solve large-scale parametric bilinear saddle point problems where cheap online assessment of the solution quality is crucial. We present the theoretical efficiency estimates of our algorithms and discuss a number of applications, primarily to the problem of ? ₁ minimization arising in sparsity-oriented signal processing. We demonstrate, both theoretically and by numerical examples, that when seeking for medium-accuracy solutions of large-scale ? ₁ minimization problems, our randomized algorithms outperform significantly (and progressively as the sizes of the problem grow) the state-of-the art deterministic methods.     

Structured minimal-memory inexact quasi-Newton method and secant preconditioners for augmented Lagrangian optimization

Augmented Lagrangian methods for large-scale optimization usually require efficient algorithms for minimization with box constraints. On the other hand, active-set box-constraint methods employ unconstrained optimization algorithms for minimization inside the faces of the box. Several approaches may be employed for computing internal search directions in the large-scale case. In this paper a minimal-memory quasi-Newton approach with secant preconditioners is proposed, taking into account the structure of Augmented Lagrangians that come from the popular Powell–Hestenes–Rockafellar scheme. A combined algorithm, that uses the quasi-Newton formula or a truncated-Newton procedure, depending on the presence of active constraints in the penalty-Lagrangian function, is also suggested. Numerical experiments using the Cute collection are presented.     

Global Optimization of Multiplicative Programs

This paper develops global optimization algorithms for linear multiplicative and generalized linear multiplicative programs based upon the lower bounding procedure of Ryoo and Sahinidis [30] and new greedy branching schemes that are applicable in the context of any rectangular branch-and-bound algorithm. Extensive computational results are presented on a wide range of problems from the literature, including quadratic and bilinear programs, and randomly generated large-scale multiplicative programs. It is shown that our algorithms make possible for the first time the solution of large and complex multiplicative programs to global optimality.     

Symmetric Perry conjugate gradient method

A family of new conjugate gradient methods is proposed based on Perry’s idea, which satisfies the descent property or the sufficient descent property for any line search. In addition, based on the scaling technology and the restarting strategy, a family of scaling symmetric Perry conjugate gradient methods with restarting procedures is presented. The memoryless BFGS method and the SCALCG method are the special forms of the two families of new methods, respectively. Moreover, several concrete new algorithms are suggested. Under Wolfe line searches, the global convergence of the two families of the new methods is proven by the spectral analysis for uniformly convex functions and nonconvex functions. The preliminary numerical comparisons with CG_DESCENT and SCALCG algorithms show that these new algorithms are very effective algorithms for the large-scale unconstrained optimization problems. Finally, a remark for further research is suggested.     

Experiments on the practical I/O efficiency of geometric algorithms: Distribution sweep versus plane sweep

We present an extensive experimental study comparing the performance of four algorithms for the following orthogonal segment intersection problem: given a set of horizontal and vertical line segments in the plane, report all intersecting horizontal-vertical pairs. The problem has important applications in VLSI layout and graphics, which are large-scale in nature. The algorithms under evaluation are our implementations of distribution sweep and three variations of plane sweep. Distribution sweep is specifically designed for the situations in which the problem is too large to be solved in internal memory, and theoretically has optimal I/O cost. Plane sweep is a well-known and powerful technique in computational geometry, and is optimal for this particular problem in terms of internal computation. The three variations of plane sweep differ by the sorting methods (external versus internal sorting) used in the preprocessing phase and the dynamic data structures (B-tree versus 2-3-4-tree) used in the sweeping phase. We generate the test data by three programs that use a random number generator while producing some interesting properties that are predicted by our theoretical analysis. The sizes of the test data range from 250 thousand segments to 2.5 million segments. The experiments provide detailed quantitative evaluation of the performance of the four algorithms, and the observed behavior of the algorithms is consistent with their theoretical properties. This is, to the best of our knowledge, the first experimental algorithmic study comparing the practical performance between external-memory algorithms and conventional algorithms with large-scale test data.     

考虑泊位疏浚的连续型泊位和动态岸桥联合调度

针对集装箱码头泊位需要定期维护的实际特征,研究了泊位疏浚情况下连续型泊位和动态岸桥联合调度问题。首先,建立了一个以船舶周转时间最小为目标的整数线性规划模型;其次,针对问题特性设计了三种启发式算法。为了分析泊位疏浚对码头工作的影响并验证模型正确性和算法有效性,分别对未考虑泊位疏浚和考虑泊位疏浚两种调度情形,进行了小规模与大规模问题输入的多组测试。三种算法在小规模输入上均取得了相同于CPLEX的精确解,从而验证了算法的有效性;进一步通过对比分析这些算法在大规模输入中的运行结果,验证其有效性能。     

Stochastic modeling and optimization of garbage collection algorithms in solid-state drive systems

Markov chains and mean-field analysis are powerful tools and widely used for performance analysis in large-scale computer and communication systems. In this paper, we consider the application of Markov modeling and mean-field analysis to solid-state drives (SSDs). SSDs are now widely deployed in mobiles, desktops, and data centers due to their high I/O performance and low energy consumption. In particular, we focus on characterizing the performance–durability tradeoff of garbage collection (GC) algorithms in SSDs. Specifically, we first develop a stochastic Markov chain model to capture the I/O dynamics of large-scale SSDs, then adapt mean-field analysis to derive the asymptotic steady state, based on which we are able to easily analyze the performance–durability tradeoff of a large family of GC algorithms. We further prove the model convergence and generalize the model for all types of workload. Inspired by this model, we also propose a randomized greedy algorithm (RGA) which has a single tunable parameter to trade between performance and durability. Using trace-driven simulation on DiskSim with SSD add-ons, we demonstrate how RGA can be parameterized to realize the performance–durability tradeoff.     

Computationally Efficient Methods for Calculating the Ideal Solution for MOLP Problems

This paper presents the results of an investigation into computational considerations that are relevant to large-scale multiobjective linear programming (MOLP) problems. Four approaches to obtaining a representation of the ideal solution are compared. Statistics on the number of simplex iterations and CPU time required are analysed for a set of randomly generated multiobjective linear programming problems. Recommendations are made based on the analysis of these results which are applicable to many MOLP solution algorithms.     

An Accelerated Coordinate Gradient Descent Algorithm for Non-separable Composite Optimization

Journal of Optimization Theory and Applications - Coordinate descent algorithms are popular in machine learning and large-scale data analysis problems due to their low computational cost iterative...