Optimal Parameter Tuning for Multiclass Support Vector Machines in Machinery Health State Estimation

The increasing demand for high reliability, safety and availability of technical systems calls for innovative maintenance strategies. The use of prognostic health management (PHM) approach where maintenance action is taken based on current and future health state of a component or system is rapidly gaining popularity in the maintenance industry. Multiclass support vector machines (MC-SVM) has been identified as a promising algorithm in PHM applications due to its high classification accuracy. However, it requires parameter tuning for each application, with the objective of minimizing the classification error. This is a single objective optimization problem which requires the use of optimization algorithms that are capable of exhaustively searching for the global optimum parameters. This work proposes the use of hybrid differential evolution (DE) and particle swarm optimization (PSO) in optimally tuning the MC-SVM parameters. DE identifies the search limit of the parameters while PSO finds the global optimum within the search limit. The feasibility of the approach is verified using bearing run-to-failure data and the results show that the proposed method significantly increases health state classification accuracy. (© 2014 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim)     

Global optimization of higher order moments in portfolio selection

We discuss the global optimization of the higher order moments of a portfolio of financial assets. The proposed model is an extension of the celebrated mean variance model of Markowitz. Asset returns typically exhibit excess kurtosis and are often skewed. Moreover investors would prefer positive skewness and try to reduce kurtosis of their portfolio returns. Therefore the mean variance model (assuming either normally distributed returns or quadratic utility functions) might be too simplifying. The inclusion of higher order moments has therefore been proposed as a possible augmentation of the classical model in order to make it more widely applicable. The resulting problem is non-convex, large scale, and highly relevant in financial optimization. We discuss the solution of the model using two stochastic algorithms. The first algorithm is Differential Evolution (DE). DE is a population based metaheuristic originally designed for continuous optimization problems. New solutions are generated by combining up to four existing solutions plus noise, and acceptance is based on evolutionary principles. The second algorithm is based on the asymptotic behavior of a suitably defined Stochastic Differential Equation (SDE). The SDE consists of three terms. The first term tries to reduce the value of the objective function, the second enforces feasibility of the iterates, while the third adds noise in order to enable the trajectory to climb hills.     

Adaptive variable space differential evolution algorithm based on population distribution

A novel memetic computing optimization algorithms, i.e. an adaptive variable space differential evolution algorithm (AVSDE), is proposed to improve the global optimization performance. AVSDE guides most individuals search in adaptive variable space (AVS) and employs adaptive mutation and adaptive control parameter. In AVSDE, AVS is determined by population global distribution information, and DE’s operators depend on the local information of the distance and direction. The performance of AVSDE is improved by integrating the global information with the local information. In addition, different mutation strategies are selected according to the evolution stage and random probability to balance AVSDE’s exploration and exploitation abilities, and adaptive control parameter is used to further enhance the performance of AVSDE. 19 scalable benchmark functions are employed to demonstrate the performance of AVSDE. Comparing with two well-tuned conventional DE and several state $-$ of-the $-$ art parameter adaptive DE variants, the whole performance of AVSDE is the best. Finally, two experiments are conducted to analyze the effect of the key parameters on AVSDE’s performance, and the optimal parameters are obtained.     

Differential evolution algorithm-based parameter estimation for chaotic systems

Parameter estimation for chaotic systems is an important issue in nonlinear science and has attracted increasing interests from various research fields, which could be essentially formulated as a multidimensional optimization problem. As a novel evolutionary computation technique, differential evolution algorithm (DE) has attracted much attention and wide applications, owing to its simple concept, easy implementation and quick convergence. However, to the best of our knowledge, there is no published work on DE for estimating parameters of chaotic systems. In this paper, a DE approach is applied to estimate the parameters of Lorenz system. Numerical simulation and the comparisons demonstrate the effectiveness and robustness of DE. Moreover, the effect of population size on the optimization performances is investigated as well.     

An efficient Differential Evolution based algorithm for solving multi-objective optimization problems

In the present study, a modified variant of Differential Evolution (DE) algorithm for solving multi-objective optimization problems is presented. The proposed algorithm, named Multi-Objective Differential Evolution Algorithm (MODEA) utilizes the advantages of Opposition-Based Learning for generating an initial population of potential candidates and the concept of random localization in mutation step. Finally, it introduces a new selection mechanism for generating a well distributed Pareto optimal front. The performance of proposed algorithm is investigated on a set of nine bi-objective and five tri-objective benchmark test functions and the results are compared with some recently modified versions of DE for MOPs and some other Multi Objective Evolutionary Algorithms (MOEAs). The empirical analysis of the numerical results shows the efficiency of the proposed algorithm.     

Self Balanced Differential Evolution

Differential Evolution (DE) is a well known and simple population based probabilistic approach for global optimization. It has reportedly outperformed a few Evolutionary Algorithms (EAs) and other search heuristics like the Particle Swarm Optimization (PSO) when tested over both benchmark and real world problems. But, DE, like other probabilistic optimization algorithms, sometimes behave prematurely in convergence. Therefore, in order to avoid stagnation while keeping a good convergence speed for DE, two modifications are proposed: one is the introduction of a new control parameter, Cognitive Learning Factor (CLF) and the other is dynamic setting of scale factor. Both modifications are proposed in mutation process of DE. Cognitive learning is a powerful mechanism that adjust the current position of individuals by a means of some specified knowledge. The proposed strategy, named as Self Balanced Differential Evolution (SBDE), balances the exploration and exploitation capability of the DE. To prove efficiency and efficacy of SBDE, it is tested over 30 benchmark optimization problems and compared the results with the basic DE and advanced variants of DE namely, SFLSDE, OBDE and jDE. Further, a real-world optimization problem, namely, Spread Spectrum Radar Polly phase Code Design, is solved to show the wide applicability of the SBDE.     

Differential evolution for sequencing and scheduling optimization

This paper presents a stochastic method based on the differential evolution (DE) algorithm to address a wide range of sequencing and scheduling optimization problems. DE is a simple yet effective adaptive scheme developed for global optimization over continuous spaces. In spite of its simplicity and effectiveness the application of DE on combinatorial optimization problems with discrete decision variables is still unusual. A novel solution encoding mechanism is introduced for handling discrete variables in the context of DE and its performance is evaluated over a plethora of public benchmarks problems for three well-known NP-hard scheduling problems. Extended comparisons with the well-known random-keys encoding scheme showed a substantially higher performance for the proposed. Furthermore, a simple slight modification in the acceptance rule of the original DE algorithm is introduced resulting to a more robust optimizer over discrete spaces than the original DE.     

Improved biogeography-based optimization with random ring topology and Powell's method

Biogeography-based optimization (BBO) is a competitive population optimization algorithm based on biogeography theory with inherently insufficient exploration capability and slow convergence speed. To overcome limitations, we propose an improved variant of BBO, named PRBBO, for solving global optimization problems. In PRBBO, a hybrid migration operator with random ring topology, a modified mutation operator, and a self-adaptive Powell's method are rational integrated together. The hybrid migration operator with random ring topology, denoted as RMO, is created by using local ring topology to replace global topology, which can avoid the asymmetrical migration operation and enhance potential population diversity. The self-adaptive Powell's method is amended by using self-adaptive parameters for suiting evolution process to enhance solution precision quickly. Extensive experimental tests are carried out on 24 benchmark functions to show effectiveness of the proposed algorithm. Simulation results were compared with original BBO, ABC, DE, other variants of the BBO, and other state-of-the-art evolutionary algorithms. Finally, the effectiveness of operators on the performance of PRBBO is also discussed.     

An improved harmony search algorithm for synchronization of discrete-time chaotic systems

The harmony search (HS) algorithm is a recently developed meta-heuristic algorithm, and has been very successful in a wide variety of optimization problems. HS was conceptualized using an analogy with music improvisation process where music players improvise the pitches of their instruments to obtain better harmony. The HS algorithm does not require initial values and uses a random search instead of a gradient search, so derivative information is unnecessary. Furthermore, the HS algorithm is simple in concept, few in parameters, easy in implementation, imposes fewer mathematical requirements, and does not require initial value settings of the decision variables. In recent years, the investigation of synchronization and control problem for discrete chaotic systems has attracted much attention, and many possible applications. The tuning of a proportional–integral–derivative (PID) controller based on an improved HS (IHS) algorithm for synchronization of two identical discrete chaotic systems subject the different initial conditions is investigated in this paper. Simulation results of the IHS to determine the PID parameters to synchronization of two Hénon chaotic systems are compared with other HS approaches including classical HS and global-best HS. Numerical results reveal that the proposed IHS method is a powerful search and controller design optimization tool for synchronization of chaotic systems.     

A new multi-threshold image segmentation approach using state transition algorithm

Thresholding plays an important role in image segmentation and image analysis. In this paper, the normalized histogram of an image is fitted by a linear combined normal distribution functions and each normal distribution function represents a class of pixels, whereas the parameters like the mean, the variance and the weights in the fitting function are undetermined. By transforming the fitting problem into a nonlinear and non-convex optimization problem, the state transition algorithm (STA) which is a new global optimization method is used to choose the optimal parameters of the fitting function. The effectiveness of proposed approach in multilevel thresholding problems is tested by several experimental results. By comparing with OTSU, particle swarm optimization (PSO), genetic algorithm (GA) and differential evolution (DE) algorithm, it has shown that STA has competitive performance in terms of both optimization results and thresholding segmentation.     

Analysis of estimation accuracy of the first moments of a Monte Carlo solution to an SDE with Wiener and Poisson components

In this paper, the estimation accuracy of the first moments of a numerical solution to an SDE with Wiener and Poisson components is investigated by a generalized explicit Euler method. Exact expressions for the mathematical expectation and variance of a test SDE solution are obtained. These expressions allow us to investigate the estimation accuracy obtained by a Monte Carlo method versus the SDE parameters, the integration step, and the size of the ensemble of simulated trajectories of the solution. The results of test numerical experiments are presented.     

Bare bones differential evolution

The barebones differential evolution (BBDE) is a new, almost parameter-free optimization algorithm that is a hybrid of the barebones particle swarm optimizer and differential evolution. Differential evolution is used to mutate, for each particle, the attractor associated with that particle, defined as a weighted average of its personal and neighborhood best positions. The performance of the proposed approach is investigated and compared with differential evolution, a Von Neumann particle swarm optimizer and a barebones particle swarm optimizer. The experiments conducted show that the BBDE provides excellent results with the added advantage of little, almost no parameter tuning. Moreover, the performance of the barebones differential evolution using the ring and Von Neumann neighborhood topologies is investigated. Finally, the application of the BBDE to the real-world problem of unsupervised image classification is investigated. Experimental results show that the proposed approach performs very well compared to other state-of-the-art clustering algorithms in all measured criteria.     

Hybridizing local search algorithms for global optimization

In this paper, we combine two types of local search algorithms for global optimization of continuous functions. In the literature, most of the hybrid algorithms are produced by combination of a global optimization algorithm with a local search algorithm and the local search is used to improve the solution quality, not to explore the search space to find independently the global optimum. The focus of this research is on some simple and efficient hybrid algorithms by combining the Nelder–Mead simplex (NM) variants and the bidirectional random optimization (BRO) methods for optimization of continuous functions. The NM explores the whole search space to find some promising areas and then the BRO local search is entered to exploit optimal solution as accurately as possible. Also a new strategy for shrinkage stage borrowed from differential evolution (DE) is incorporated in the NM variants. To examine the efficiency of proposed algorithms, those are evaluated by 25 benchmark functions designed for the special session on real-parameter optimization of CEC2005. A comparison study between the hybrid algorithms and some DE algorithms and non-parametric analysis of obtained results demonstrate that the proposed algorithms outperform most of other algorithms and their difference in most cases is statistically considerable. In a later part of the comparative experiments, a comparison of the proposed algorithms with some other evolutionary algorithms reported in the CEC2005 confirms a better performance of our proposed algorithms.     

基于种群自适应调整的多目标差分进化算法

为提高已有多目标进化算法在求解复杂多目标优化问题上的收敛性和解集分布性,提出一种基于种群自适应调整的多目标差分进化算法。该算法设计一个种群扩增策略,它在决策空间生成一些新个体帮助搜索更优的非支配解;设计了一个种群收缩策略,它依据对非支配解集的贡献程度淘汰较差的个体以减少计算负荷,并预留一些空间给新的带有种群多样性的扰动个体;引入精英学习策略,防止算法陷入局部收敛。通过典型的多目标优化函数对算法进行测试验证,结果表明所提算法相对于其他算法具有明显的优势,其性能优越,能够在保证良好收敛性的同时,使获得的Pareto最优解集具有更均匀的分布性和更广的覆盖范围,尤其适合于高维复杂多目标优化问题的求解。     

Massively parallel differential evolution—pattern search optimization with graphics hardware acceleration: an investigation on bound constrained optimization problems

This paper presents a novel parallel Differential Evolution (DE) algorithm with local search for solving function optimization problems, utilizing graphics hardware acceleration. As a population-based meta-heuristic, DE was originally designed for continuous function optimization. Graphics Processing Units (GPU) computing is an emerging desktop parallel computing technology that is becoming popular with its wide availability in many personal computers. In this paper, the classical DE was adapted in the data-parallel CPU-GPU heterogeneous computing platform featuring Single Instruction-Multiple Thread (SIMT) execution. The global optimal search of the DE was enhanced by the classical local Pattern Search (PS) method. The hybrid DE–PS method was implemented in the GPU environment and compared to a similar implementation in the common computing environment with a Central Processing Unit (CPU). Computational results indicate that the GPU-accelerated SIMT-DE-PS method is orders of magnitude faster than the corresponding CPU implementation. The main contribution of this paper is the parallelization analysis and performance analysis of the hybrid DE–PS with GPU acceleration. The research results demonstrate a promising direction for high speed optimization with desktop parallel computing on a personal computer.     

On the estimation of parameters for linear stochastic differential equations

The estimation of arbitrary number of parameters in linear stochastic differential equation (SDE) is investigated. The local asymptotic normality (LAN) of families of distributions corresponding to this SDE is established and the asymptotic efficiency of the maximum likelihood estimator (MLE) is obtained for the wide class of loss functions with polynomial majorants. As an example a single-degree of freedom mechanical system is considered. The results generalize [8], where all elements of the drift matrix are estimated and the asymptotic efficiency is proved only for the bounded loss functions. Received: 12 March 1997 / Revised version: 22 June 1998     

Differential evolution using a superior–inferior crossover scheme

Differential evolution (DE) is a new population-based stochastic optimization, which has difficulties in solving large-scale and multimodal optimization problems. The reason is that the population diversity decreases rapidly, which leads to the failure of the clustered individuals to reproduce better individuals. In order to improve the population diversity of DE, this paper aims to present a superior–inferior (SI) crossover scheme based on DE. Specifically, when population diversity degree is small, the SI crossover is performed to improve the search space of population. Otherwise, the superior–superior crossover is used to enhance its exploitation ability. In order to test the effectiveness of our SI scheme, we combine the SI with adaptive differential evolution (JADE), which is a recently developed DE variant for numerical optimization. In addition, the theoretical analysis of SI scheme is provided to show how the population’s diversity can be improved. In order to make the selection of parameters in our scheme more intelligently, a self-adaptive SI crossover scheme is proposed. Finally, comparative comprehensive experiments are given to illustrate the advantages of our proposed method over various DEs on a suite of 24 numerical optimization problems.     

A new hybrid algorithm based on optimal fuzzy controller in multimachine power system

In this article, a new methodology based on fuzzy proportional‐integral‐derivative (PID) controller is proposed to damp low frequency oscillation in multimachine power system where the parameters of proposed controller are optimized offline automatically by hybrid genetic algorithm (GA) and particle swarm optimization (PSO) techniques. This newly proposed method is more efficient because it cope with oscillations and different operating points. In this strategy, the controller is tuned online from the knowledge base and fuzzy interference. In the proposed method, for achieving the desired level of robust performance exact tuning of rule base and membership functions (MF) are very important. The motivation for using the GA and PSO as a hybrid method are to reduce fuzzy effort and take large parametric uncertainties in to account. This newly developed control strategy mixed the advantage of GA and PSO techniques to optimally tune the rule base and MF parameters of fuzzy controller that leads to a flexible controller with simple structure while is easy to implement. The proposed method is tested on three machine nine buses and 16 machine power systems with different operating conditions in present of disturbance and nonlinearity. The effectiveness of proposed controller is compared with robust PSS that tune using PSO and the fuzzy controller which is optimized rule base by GA through figure of demerit and integral of the time multiplied absolute value of the error performance indices. The results evaluation shows that the proposed method achieves good robust performance for a wide range of load change in the presents of disturbance and system nonlinearities and is superior to the other controllers. © 2014 Wiley Periodicals, Inc. Complexity 21: 78–93, 2015     

Monitoring change point for diffusion parameter based on discretely observed sample from stochastic differential equation models

Stochastic differential equation (SDE) models are useful in describing complex dynamical systems in science and engineering. In this study, we consider a monitoring procedure for an early detection of dispersion parameter change in SDE models. The proposed scheme provides a useful diagnostic analysis for phase I retrospective study and develops a flexible and effective control chart for phase II prospective monitoring. A standardized control chart is constructed, and a bootstrap method is used to estimate the mean and variance of the monitoring statistic. The control limit is obtained as an upper percentile of the maximum value of a standard Wiener process. The proposed procedure appears to have a manageable computational complexity for online implementation and also to be effective in detecting changes. We also investigate the performance of the exponentially weighted mean squared control charts for the continuous SDE processes. A simulation method is used to study the empirical sizes and the average run length characteristics of the proposed scheme, which also demonstrates the effectiveness of our method. Finally, we provide an empirical example for illustration. Copyright © 2014 John Wiley & Sons, Ltd.     

Intelligent active force control of a 3-RRR parallel manipulator incorporating fuzzy resolved acceleration control

This paper introduces a novel intelligent control scheme for robust and precise positioning and orientation of a class of highly non-linear 3-RRR (revolute-revolute-revolute) planar parallel manipulator. The primary objective is to force the manipulator to track accurately a prescribed Cartesian trajectory when the system is subjected to different types of disturbances in the forms of forced harmonic excitations. A two level fuzzy tuning resolved acceleration control (FLRAC) is first designed and implemented to the system to demonstrate the stable response of the manipulator in performing trajectory tracking tasks in the absence of the disturbances. In this scheme, the first level of fuzzy tuning is used to acquire the proportional-derivative (PD) gains linearly while the second level considers non-linear tuning for determining the other parameters of the fuzzy controller to increase its performance. Then, the controller is added in series with an active force controller (AFC) to create a novel two degree-of-freedom (DOF) controller known as FLRAC-AFC which is subsequently and rigorously tested for system robustness and accuracy in tracking the prescribed trajectory. The simulation study provides further insight into the potentials of the proposed robotic system in rejecting the disturbances for the given operating conditions. The results clearly show that the FLRAC-AFC scheme provides a much superior trajectory tracking capability compared to the conventional linear RAC alone.