Enhancing performance of particle swarm optimization through an algorithmic link with genetic algorithms

Evolutionary Algorithms (EAs) are emerging as competitive and reliable techniques for several optimization tasks. Juxtapositioning their higher-level and implicit correspondence; it is provocative to query if one optimization algorithm can benefit from another by studying underlying similarities and dissimilarities. This paper establishes a clear and fundamental algorithmic linking between particle swarm optimization (PSO) algorithm and genetic algorithms (GAs). Specifically, we select the task of solving unimodal optimization problems, and demonstrate that key algorithmic features of an effective Generalized Generation Gap based Genetic Algorithm can be introduced into the PSO by leveraging this algorithmic linking while significantly enhance the PSO’s performance. However, the goal of this paper is not to solve unimodal problems, neither is to demonstrate that the modified PSO algorithm resembles a GA, but to highlight the concept of algorithmic linking in an attempt towards designing efficient optimization algorithms. We intend to emphasize that the evolutionary and other optimization researchers should direct more efforts in establishing equivalence between different genetic, evolutionary and other nature-inspired or non-traditional algorithms. In addition to achieving performance gains, such an exercise shall deepen the understanding and scope of various operators from different paradigms in Evolutionary Computation (EC) and other optimization methods.     

Self Balanced Differential Evolution

Differential Evolution (DE) is a well known and simple population based probabilistic approach for global optimization. It has reportedly outperformed a few Evolutionary Algorithms (EAs) and other search heuristics like the Particle Swarm Optimization (PSO) when tested over both benchmark and real world problems. But, DE, like other probabilistic optimization algorithms, sometimes behave prematurely in convergence. Therefore, in order to avoid stagnation while keeping a good convergence speed for DE, two modifications are proposed: one is the introduction of a new control parameter, Cognitive Learning Factor (CLF) and the other is dynamic setting of scale factor. Both modifications are proposed in mutation process of DE. Cognitive learning is a powerful mechanism that adjust the current position of individuals by a means of some specified knowledge. The proposed strategy, named as Self Balanced Differential Evolution (SBDE), balances the exploration and exploitation capability of the DE. To prove efficiency and efficacy of SBDE, it is tested over 30 benchmark optimization problems and compared the results with the basic DE and advanced variants of DE namely, SFLSDE, OBDE and jDE. Further, a real-world optimization problem, namely, Spread Spectrum Radar Polly phase Code Design, is solved to show the wide applicability of the SBDE.     

Differential Evolution algorithm with Separated Groups for multi-dimensional optimization problems

The classical Differential Evolution (DE) algorithm, one of population-based Evolutionary Computation methods, proved to be a successful approach for relatively simple problems, but does not perform well for difficult multi-dimensional non-convex functions. A number of significant modifications of DE have been proposed in recent years, including very few approaches referring to the idea of distributed Evolutionary Algorithms. The present paper presents a new algorithm to improve optimization performance, namely DE with Separated Groups (DE-SG), which distributes population into small groups, defines rules of exchange of information and individuals between the groups and uses two different strategies to keep balance between exploration and exploitation capabilities. The performance of DE-SG is compared to that of eight algorithms belonging to the class of Evolutionary Strategies (Covariance Matrix Adaptation ES), Particle Swarm Optimization (Comprehensive Learning PSO and Efficient Population Utilization Strategy PSO), Differential Evolution (Distributed DE with explorative-exploitative population families, Self-adaptive DE, DE with global and local neighbours and Grouping Differential Evolution) and multi-algorithms (AMALGAM). The comparison is carried out for a set of 10-, 30- and 50-dimensional rotated test problems of varying difficulty, including 10- and 30-dimensional composition functions from CEC2005. Although slow for simple functions, the proposed DE-SG algorithm achieves a great success rate for more difficult 30- and 50-dimensional problems.     

Process and content: two ways of using SSM

Soft systems methodology (SSM) includes several ways of gaining a rich appreciation of the problem situation addressed. ‘Analysis One’, exploration of the intervention itself, is the subject here, since it is sparsely covered in the literature. The analysis is conducted in terms of three roles: ‘client’, ‘problem solver’ and ‘problem owner’. Whoever is in the role of ‘problem solver’ is free to define a list of possible ‘problem owners’, which brings many perspectives to bear on the situation. It was realized that ‘client’ and ‘problem solver’ should themselves feature in the ‘problem owner’ list. The ‘problem’ owned by the ‘problem solver’ is that of undertaking the intervention. This led to a realization that SSM is relevant to both the content of a perceived situation (SSMc) and the process of dealing with that content (SSMp). This development is described and illustrated by work in the National Health Service. The focus of the SSM use was to define the intellectual process for a service specification project which NHS professionals would themselves carry out.     

Fitness based Differential Evolution

Differential Evolution (DE) is a well known and simple population based probabilistic approach for global optimization. It has reportedly outperformed a few Evolutionary Algorithms and other search heuristics like Particle Swarm Optimization when tested over both benchmark and real world problems. But, DE, like other probabilistic optimization algorithms, sometimes exhibits premature convergence and stagnates at suboptimal point. In order to avoid stagnation behavior while maintaining a good convergence speed, a new position update process is introduced, named fitness based position update process in DE. In the proposed strategy, position of the solutions are updated in two phases. In the first phase all the solutions update their positions using the basic DE and in the second phase, all the solutions update their positions based on their fitness. In this way, a better solution participates more times in the position update process. The position update equation is inspired from the Artificial Bee Colony algorithm. The proposed strategy is named as Fitness Based Differential Evolution ( $FBDE$ ). To prove efficiency and efficacy of $FBDE$ , it is tested over 22 benchmark optimization problems. A comparative analysis has also been carried out among proposed FBDE, basic DE, Simulated Annealing Differential Evolution and Scale Factor Local Search Differential Evolution. Further, $FBDE$ is also applied to solve a well known electrical engineering problem called Model Order Reduction problem for Single Input Single Output Systems.     

Against the Minimalistic Reading of Epistemic Contextualism: A Reply to Wolfgang Freitag

Several philosophers have argued that the factivity of knowledge poses a problem for epistemic contextualism (EC), which they have construed as a knowability problem. On a proposed minimalistic reading of EC’s commitments, Wolfgang Freitag argues that factivity yields no knowability problem for EC. I begin by explaining how factivity is thought to generate a contradiction out of paradigmatic contextualist cases on a certain reading of EC’s commitments. This reductio results in some kind of reflexivity problem for the contextualist when it comes to knowing her theory: either a knowability problem or a statability problem. Next, I set forth Freitag’s minimalistic reading of EC and explain how it avoids the reductio, the knowability problem and the statability problem. I argue that despite successfully evading these problems, Freitag’s minimalistic reading saddles EC with several other serious problems and should be rejected. I conclude by offering my own resolution to the problems.     

Cognitive learning in differential evolution and its application to model order reduction problem for single-input single-output systems

Differential evolution (DE) is a well known and simple population based probabilistic approach for global optimization over continuous spaces. It has reportedly outperformed a few evolutionary algorithms and other search heuristics like the particle swarm optimization when tested over both benchmark and real world problems. DE, like other probabilistic optimization algorithms, has inherent drawback of premature convergence and stagnation. Therefore, in order to find a trade-off between exploration and exploitation capability of DE algorithm, a new parameter namely, cognitive learning factor (CLF) is introduced in the mutation process. Cognitive learning is a powerful mechanism that adjust the current position of individuals by the means of some specified knowledge (previous experience of individuals). The proposed strategy is named as cognitive learning in differential evolution (CLDE). To prove the efficiency of various approaches of CLF in DE,?CLDE is tested over 25 benchmark problems. Further, to establish the wide applicability of CLF,?CLDE is applied to two advanced DE variants. CLDE is also applied to solve a well known electrical engineering problem called model order reduction problem for single input single output systems.     

GPU implementation of a Helmholtz Krylov solver preconditioned by a shifted Laplace multigrid method

A Helmholtz equation in two dimensions discretized by a second order finite difference scheme is considered. Krylov methods such as Bi-CGSTAB and IDR(s) have been chosen as solvers. Since the convergence of the Krylov solvers deteriorates with increasing wave number, a shifted Laplace multigrid preconditioner is used to improve the convergence. The implementation of the preconditioned solver on CPU (Central Processing Unit) is compared to an implementation on GPU (Graphics Processing Units or graphics card) using CUDA (Compute Unified Device Architecture). The results show that preconditioned Bi-CGSTAB on GPU as well as preconditioned IDR(s) on GPU is about 30 times faster than on CPU for the same stopping criterion.     

Gradient boosting for convex cone predict and optimize problems

Prediction models are traditionally optimized independently from decision-based optimization. Conversely, a ‘smart predict then optimize’ (SPO) framework optimizes prediction models to minimize downstream decision regret. In this paper we present dboost, the first general purpose implementation of smart gradient boosting for ‘predict, then optimize’ problems. The framework supports convex quadratic cone programming and gradient boosting is performed by implicit differentiation of a custom fixed-point mapping. Experiments comparing with state-of-the-art SPO methods show that dboost can further reduce out-of-sample decision regret.     

Completely unimodal numberings of a simple polytope

A completely unimodal numbering of the m vertices of a simple d-dimensional polytope is a numbering 0, 1, …,m−1 of the vertices such that on every k-dimensional face (2≤k≤d) there is exactly one local minimum (a vertex with no lower-numbered neighbors on that face). Such numberings are abstract objective functions in the sense of Adler and Saigal [1]. It is shown that a completely unimodal numbering of the vertices of a simple polytope induces a shelling of the facets of the dual simplicial polytope. The h-vector of the dual simplicial polytope is interpreted in terms of the numbering (with respect to using a local-improvement algorithm to locate the vertex numbered 0). In the case that the polytope is combinatorially equivalent to a d-dimensional cube, a ‘successor-tuple’ for each vertex is defined which carries the crucial information of the numbering for local-improvement algorithms. Combinatorial properties of these d-tuples are studied. Finally the running time of one particular local-improvement algorithm, the Random Algorithm, is studied for completely unimodal numberings of the d-cube. It is shown that for a certain class of numberings (which includes the example of Klee and Minty [8] showing that the simplex algorithm is not polynomial and all Hamiltonian saddle-free injective pseudo-Boolean functions [6]) this algorithm has expected running time that is at worst quadratic in the dimension d.     

On the GPGPU parallelization issues of finite element approximate inverse preconditioning

During the past decades, explicit finite element approximate inverse preconditioning methods have been extensively used for efficiently solving sparse linear systems on multiprocessor systems. The effectiveness of explicit approximate inverse preconditioning schemes relies on the use of efficient preconditioners that are close approximants to the coefficient matrix and are fast to compute in parallel. New parallel computational techniques are proposed for the parallelization of the Optimized Banded Generalized Approximate Inverse Finite Element Matrix (OBGAIFEM) algorithm, based on the concept of the “fish bone” computational approach, and for the Explicit Preconditioned Conjugate Gradient type methods on a General Purpose Graphics Processing Unit (GPGPU). The proposed parallel methods have been implemented using Compute Unified Device Architecture (CUDA) developed by NVIDIA. Finally, numerical results for the performance of the finite element explicit approximate inverse preconditioning for solving characteristic two dimensional boundary value problems on a massive multiprocessor interface on a GPU are presented. The CUDA implementation issues of the proposed methods are also discussed.     

On the GPGPU parallelization issues of finite element approximate inverse preconditioning

During the past decades, explicit finite element approximate inverse preconditioning methods have been extensively used for efficiently solving sparse linear systems on multiprocessor systems. The effectiveness of explicit approximate inverse preconditioning schemes relies on the use of efficient preconditioners that are close approximants to the coefficient matrix and are fast to compute in parallel. New parallel computational techniques are proposed for the parallelization of the Optimized Banded Generalized Approximate Inverse Finite Element Matrix (OBGAIFEM) algorithm, based on the concept of the “fish bone” computational approach, and for the Explicit Preconditioned Conjugate Gradient type methods on a General Purpose Graphics Processing Unit (GPGPU). The proposed parallel methods have been implemented using Compute Unified Device Architecture (CUDA) developed by NVIDIA. Finally, numerical results for the performance of the finite element explicit approximate inverse preconditioning for solving characteristic two dimensional boundary value problems on a massive multiprocessor interface on a GPU are presented. The CUDA implementation issues of the proposed methods are also discussed.     

Reflections on queue modelling from the last 50 years

Queueing theory continues to be one of the most researched areas of operational research, and has generated numerous review papers over the years. The phrase ‘queue modelling’ is used in the title to indicate a more practical emphasis. This paper uses work taken predominantly from the last 50 years of pages of the Operational Research Quarterly and the Journal of the Operational Research Society to offer a commentary on attempts of operational researchers to tackle real queueing problems, and on research foci past and future. A new discipline of ‘queue modelling’ is proposed, drawing upon the combined strengths of analytic and simulation approaches with the responsibility to derive meaningful insights for managers.     

On the Convergence of the Pshenichnyi-Pironneau-Polak Minimax Algorithm with an Active Set Strategy

The purpose of this technical note is to present a proof of convergence of the Pshenichnyi-Pironneau-Polak (PPP) minimax algorithm (see Algorithm 2.4.1 in Polak, Optimization: Algorithms and Consistent Approximations, Springer, [1997]), modified to use an active set strategy. This active set strategy was formally derived in Polak (Optimization: Algorithms and Consistent Approximations, Springer, [1997]) from those used in the methods of feasible directions developed by Zoutendijk (Methods of Feasible Directions, Elsevier, [1960]) and Polak (Computational Methods in Optimization: A Unified Approach, Academic, [1971]). The resulting ε-Active PPP algorithm was presented as Algorithm 2.4.34, in Polak (Optimization: Algorithms and Consistent Approximations, Springer, [1997]), without any proofs.     

On the order of deferred correction

New deferred correction methods for the numerical solution of initial value problems in ordinary differential equations have recently been introduced by Dutt, Greengard and Rokhlin. A convergence proof is presented for these methods, based on the abstract Stetter-Lindberg-Skeel framework and Spijker-type norms. It is shown that p corrections of an order-r one-step solver yield order-r(p+1) accuracy.     

An Evolutionary Algorithm With Applications to Statistics

Evolutionary algorithms are used to find maxima of functions. Their claim to fame is an ability to find a global maximum in the presence of local maxima. The computations do not require derivatives or convexity, but still may be fairly computationally intensive in larger dimensions. This article presents a new type of evolutionary algorithm that works well in many dimensions, with the added advantage that linear equality constraints are implemented in a natural way. Bounds on the coordinates of the solution are also easy to implement. Several examples are presented and the new algorithm is compared with standard versions of evolutionary algorithms. The first group consists of functions in one or two dimensions, chosen to be difficult to maximize by gradient or simplex-based methods. The second set of examples are from statistics problems that are known to be computationally difficult. The first is least absolute deviations nonlinear regression with bootstrapped confidence bounds on the mean response, the second is smoothed nonparametric unimodal density estimation requiring both a linear equality constraint and linear inequality constraints. The Fortran subroutine is available for the user.     

Review on the monograph Deterministic Global Optimization: An Introduction to the Diagonal Approach,Springer, 2017, written by Yaroslav D. Sergeyev and Dmitri E. Kvasov

This is a review of the book ‘Deterministic Global Optimization: An Introduction to the Diagonal Approach’.

     

DrAmpl: a meta solver for optimization problem analysis

Optimization problems modeled in the AMPL modeling language (Fourer et al., in AMPL: a modeling language for mathematical programming, 2002) may be examined by a set of tools found in the AMPL Solver Library (Gay, in Hooking your solver to AMPL, 1997). DrAmpl is a meta solver which, by use of the AMPL Solver Library, dissects such optimization problems, obtains statistics on their data, is able to symbolically prove or numerically disprove convexity of the functions involved and provides aid in the decision for an appropriate solver. A problem is associated with a number of relevant solvers available on the NEOS Server for Optimization (Czyzyk et al., in IEEE J Comput Sci Eng 5:68–75, 1998) by means of a relational database. We describe the need for such a tool, the design of DrAmpl and some of its consequences, and keep in mind that a similar tool could be developed for other algebraic modeling languages.     

Interactive Multiple Criteria Decision Making based on preference driven Evolutionary Multiobjective Optimization with controllable accuracy

We present an approach to interactive Multiple Criteria Decision Making based on preference driven Evolutionary Multiobjective Optimization with controllable accuracy.The approach relies on formulae for lower and upper bounds on coordinates of the outcome of an arbitrary efficient variant corresponding to preference information expressed by the Decision Maker. In contrast to earlier works on that subject, here lower and upper bounds can be calculated and their accuracy controlled entirely within evolutionary computation framework. This is made possible by exploration of not only the region of feasible variants - a standard within evolutionary optimization, but also the region of infeasible variants, the latter to our best knowledge being a novel approach within Evolutionary Multiobjective Optimization.To illustrate how this concept can be applied to interactive Multiple Criteria Decision Making, two algorithms employing evolutionary computations are proposed and their usefulness demonstrated by a numerical example.     

Singly-implicit stabilized extended one-step methods for second-order initial-value problems with oscillating solutions

We introduce a new class of singly-implicit extended one-step methods for the numerical integration of second-order initial-value problems y″ = f(t, y), y(t₀) = η₀, y′(t₀) = η₁, with oscillating solutions. We first show that for third order, with two stages there exists a uniquely determined ‘almost’ P-stable method. We then investigate stability of the general class of fourth-order one-step methods. We first look for stabilized fourth-order methods with two stages, and show the interesting result that there exist families of two-stage fourth-order P-stable methods. We also obtain some families of three-stage fourth-order P-stable methods. The obtained methods are computationally tested on problems of practical interest.