Self-adaptive differential evolution incorporating a heuristic mixing of operators

A considerable number of differential evolution variants have been proposed in the last few decades. However, no variant was able to consistently perform over a wide range of test problems. In this paper, propose two novel differential evolution based algorithms are proposed for solving constrained optimization problems. Both algorithms utilize the strengths of multiple mutation and crossover operators. The appropriate mix of the mutation and crossover operators, for any given problem, is determined through an adaptive learning process. In addition, to further accelerate the convergence of the algorithm, a local search technique is applied to a few selected individuals in each generation. The resulting algorithms are named as Self-Adaptive Differential Evolution Incorporating a Heuristic Mixing of Operators. The algorithms have been tested by solving 60 constrained optimization test instances. The results showed that the proposed algorithms have a competitive, if not better, performance in comparison to the-state-of-the-art algorithms.     

Mesh adaptive direct search algorithms for mixed variable optimization

This paper introduces a new derivative-free class of mesh adaptive direct search (MADS) algorithms for solving constrained mixed variable optimization problems, in which the variables may be continuous or categorical. This new class of algorithms, called mixed variable MADS (MV-MADS), generalizes both mixed variable pattern search (MVPS) algorithms for linearly constrained mixed variable problems and MADS algorithms for general constrained problems with only continuous variables. The convergence analysis, which makes use of the Clarke nonsmooth calculus, similarly generalizes the existing theory for both MVPS and MADS algorithms, and reasonable conditions are established for ensuring convergence of a subsequence of iterates to a suitably defined stationary point in the nonsmooth and mixed variable sense.     

Viscosity Approximation Methods for a Class of Generalized Split Feasibility Problems with Variational Inequalities in Hilbert Space

Strong convergence theorem of viscosity approximation methods for nonexpansive mapping have been studied. We also know that CQ algorithm for solving the split feasibility problem (SFP) has a weak convergence result. In this paper, we use viscosity approximation methods and some related knowledge to solve a class of generalized SFP’s with monotone variational inequalities in Hilbert space. We propose some iterative algorithms based on viscosity approximation methods and get strong convergence theorems. As applications, we can use algorithms we proposed for solving split variational inequality problems (SVIP), split constrained convex minimization problems and some related problems in Hilbert space.     

关于不等式约束的信赖域算法

对于具有不等式约束的非线性优化问题，本文给出一个依赖域算法，由于算法中依赖区域约束采用向量的∞范数约束的形式，从而使子问题变二次规划，同时使算法变得更实用。在通常假设条件下，证明了算法的整体收敛性和超线性收敛性。     

SOR-like Methods for Augmented Systems

Several SOR-like methods are proposed for solving augmented systems. These have many different applications in scientific computing, for example, constrained optimization and the finite element method for solving the Stokes equation. The convergence and the choice of optimal parameter for these algorithms are studied. The convergence and divergence regions for some algorithms are given, and the new algorithms are applied to solve the Stokes equations as well.     

On penalty methods for minimax problems

Penalty algorithms for constrained minimax problems typically involve a sequence of unconstrained approximates which is pointwise monotone in each variable. This paper generalizes convergence results for a wider class of algorithms while imposing conditions which are close to being minimal.Sponsored in part by a grant from the Norwegian Council of Scientific and Technological Research.     

A genetic algorithm based augmented Lagrangian method for constrained optimization

Among the penalty based approaches for constrained optimization, augmented Lagrangian (AL) methods are better in at least three ways: (i) they have theoretical convergence properties, (ii) they distort the original objective function minimally, thereby providing a better function landscape for search, and (iii) they can result in computing optimal Lagrange multiplier for each constraint as a by-product. Instead of keeping a constant penalty parameter throughout the optimization process, these algorithms update the parameters (called multipliers) adaptively so that the corresponding penalized function dynamically changes its optimum from the unconstrained minimum point to the constrained minimum point with iterations. However, the flip side of these algorithms is that the overall algorithm requires a serial application of a number of unconstrained optimization tasks, a process that is usually time-consuming and tend to be computationally expensive. In this paper, we devise a genetic algorithm based parameter update strategy to a particular AL method. The proposed strategy updates critical parameters in an adaptive manner based on population statistics. Occasionally, a classical optimization method is used to improve the GA-obtained solution, thereby providing the resulting hybrid procedure its theoretical convergence property. The GAAL method is applied to a number of constrained test problems taken from the evolutionary algorithms (EAs) literature. The number of function evaluations required by GAAL in most problems is found to be smaller than that needed by a number of existing evolutionary based constraint handling methods. GAAL method is found to be accurate, computationally fast, and reliable over multiple runs. Besides solving the problems, the proposed GAAL method is also able to find the optimal Lagrange multiplier associated with each constraint for the test problems as an added benefit??a matter that is important for a sensitivity analysis of the obtained optimized solution, but has not yet been paid adequate attention in the past evolutionary constrained optimization studies.     

Continuous-time method and its discretization to inverse problem of intensity-modulated radiation therapy treatment planning

We propose a novel approach for solving box-constrained inverse problems in intensity-modulated radiation therapy (IMRT) treatment planning based on the idea of continuous dynamical methods and split-feasibility algorithms. Our method can compute a feasible solution without the second derivative of an objective function, which is required for gradient-based optimization algorithms. We prove theoretically that a double Kullback–Leibler divergence can be used as the Lyapunov function for the IMRT planning system.Moreover, we propose a non-negatively constrained iterative method formulated by discretizing a differential equation in the continuous method. We give proof for the convergence of a desired solution in the discretized system, theoretically. The proposed method not only reduces computational costs but also does not produce a solution with an unphysical negative radiation beam weight in solving IMRT planning inverse problems.The convergence properties of solutions for an ill-posed case are confirmed by numerical experiments using phantom data simulating a clinical setup.     

求解带二次约束的非凸二次规划的一种分支定界算法

本文提出了一种求解带二次约束和线性约束的二次规划的分支定界算法.在算法中,我们运用Lipschitz条件来确定目标函数和约束函数的在每个n矩形上的上下界,对于n矩形的分割,我们采用选择n矩形最长边的二分法,同时我们采用了一些矩形删除技术,在不大幅增加计算量的前提下,起到了加速算法收敛的效果.从理论上我们证明了算法的收敛性,同时数值实验表明该算法是有效的.     

Iterative Algorithm for Solving Triple-Hierarchical Constrained Optimization Problem

Many practical problems such as signal processing and network resource allocation are formulated as the monotone variational inequality over the fixed point set of a nonexpansive mapping, and iterative algorithms to solve these problems have been proposed. This paper discusses a monotone variational inequality with variational inequality constraint over the fixed point set of a nonexpansive mapping, which is called the triple-hierarchical constrained optimization problem, and presents an iterative algorithm for solving it. Strong convergence of the algorithm to the unique solution of the problem is guaranteed under certain assumptions.     

Random-Direction Optimization Algorithms with Applications to Threshold Controls

This work develops a class of stochastic optimization algorithms. It aims to provide numerical procedures for solving threshold-type optimal control problems. The main motivation stems from applications involving optimal or suboptimal hedging policies, for example, production planning of manufacturing systems including random demand and stochastic machine capacity. The proposed algorithm is a constrained stochastic approximation procedure that uses random-direction finite-difference gradient estimates. Under fairly general conditions, the convergence of the algorithm is established and the rate of convergence is also derived. A numerical example is reported to demonstrate the performance of the algorithm.     

On certain optimization methods with finite-step inner algorithms for convex finite-dimensional problems with inequality constraints

Numerical methods are proposed for solving finite-dimensional convex problems with inequality constraints satisfying the Slater condition. A method based on solving the dual to the original regularized problem is proposed and justified for problems having a strictly uniformly convex sum of the objective function and the constraint functions. Conditions for the convergence of this method are derived, and convergence rate estimates are obtained for convergence with respect to the functional, convergence with respect to the argument to the set of optimizers, and convergence to the g-normal solution. For more general convex finite-dimensional minimization problems with inequality constraints, two methods with finite-step inner algorithms are proposed. The methods are based on the projected gradient and conditional gradient algorithms. The paper is focused on finite-dimensional problems obtained by approximating infinite-dimensional problems, in particular, optimal control problems for systems with lumped or distributed parameters.     

Gauss–Newton methods with approximate projections for solving constrained nonlinear least squares problems

This paper is concerned with algorithms for solving constrained nonlinear least squares problems. We first propose a local Gauss–Newton method with approximate projections for solving the aforementioned problems and study, by using a general majorant condition, its convergence results, including results on its rate. By combining the latter method and a nonmonotone line search strategy, we then propose a global algorithm and analyze its convergence results. Finally, some preliminary numerical experiments are reported in order to illustrate the advantages of the new schemes.     

均衡及约束凸优化问题公共解的一般迭代算法

梯度投影法在解决约束凸极小化问题中起到了重要的作用. 基于Tian的一般迭代算法, 本文将梯度投影法和平均算子方法相结合, 首次提出隐式和显式的复合迭代算法, 寻求均衡问题和约束凸极小化问题的公共解. 在适当条件下, 获得了强收敛定理.     

Random algorithms for convex minimization problems

This paper deals with iterative gradient and subgradient methods with random feasibility steps for solving constrained convex minimization problems, where the constraint set is specified as the intersection of possibly infinitely many constraint sets. Each constraint set is assumed to be given as a level set of a convex but not necessarily differentiable function. The proposed algorithms are applicable to the situation where the whole constraint set of the problem is not known in advance, but it is rather learned in time through observations. Also, the algorithms are of interest for constrained optimization problems where the constraints are known but the number of constraints is either large or not finite. We analyze the proposed algorithm for the case when the objective function is differentiable with Lipschitz gradients and the case when the objective function is not necessarily differentiable. The behavior of the algorithm is investigated both for diminishing and non-diminishing stepsize values. The almost sure convergence to an optimal solution is established for diminishing stepsize. For non-diminishing stepsize, the error bounds are established for the expected distances of the weighted averages of the iterates from the constraint set, as well as for the expected sub-optimality of the function values along the weighted averages.     

AN INTERIOR TRUST REGION ALGORITHM FOR NONLINEAR MINIMIZATION WITH LINEAR CONSTRAINTS

AbstractAn interior trust-region-based algorithm for linearly constrained minimization problems is proposed and analyzed. This algorithm is similar to trust region algorithms for unconstrained minimization: a trust region subproblem on a subspace is solved in each iteration. We establish that the proposed algorithm has convergence properties analogous to those of the trust region algorithms for unconstrained minimization. Namely, every limit point of the generated sequence satisfies the Krush-Kuhn-Tucker (KKT) conditions and at least one limit point satisfies second order necessary optimality conditions. In addition, if one limit point is a strong local minimizer and the Hessian is Lipschitz continuous in a neighborhood of that point, then the generated sequence converges globally to that point in the rate of at least 2-step quadratic. We are mainly concerned with the theoretical properties of the algorithm in this paper. Implementation issues and adaptation to large-scale problems will be addressed in a     

Projected gradient methods for linearly constrained problems

The aim of this paper is to study the convergence properties of the gradient projection method and to apply these results to algorithms for linearly constrained problems. The main convergence result is obtained by defining a projected gradient, and proving that the gradient projection method forces the sequence of projected gradients to zero. A consequence of this result is that if the gradient projection method converges to a nondegenerate point of a linearly constrained problem, then the active and binding constraints are identified in a finite number of iterations. As an application of our theory, we develop quadratic programming algorithms that iteratively explore a subspace defined by the active constraints. These algorithms are able to drop and add many constraints from the active set, and can either compute an accurate minimizer by a direct method, or an approximate minimizer by an iterative method of the conjugate gradient type. Thus, these algorithms are attractive for large scale problems. We show that it is possible to develop a finite terminating quadratic programming algorithm without non-degeneracy assumptions. Work supported in part by the Applied Mathematical Sciences subprogram of the Office of Energy Research of the U.S. Department of Energy under Contract W-31-109-Eng-38. Work supported in part by the Applied Mathematical Sciences subprogram of the Office of Energy Research of the U.S. Department of Energy under Contract W-31-109-Eng-38.     

An improved whale optimization algorithm with armed force program and strategic adjustment

In this paper, we propose a new whale army optimization algorithm with a view to solving multifarious optimization problems. The key novelty of our approach is to modify the original whale optimization algorithm to make it effective to solve the complicated, large-scale and constrained optimization problems. Our modifications mainly embody two aspects: the beneficial strategic adjustment to set key parameters and to help establish base principles in the original optimizer and the introduction of armed force program which classifies the search whales into different categories to achieve efficient cooperation. We evaluate the performance of the proposed algorithm, using three simple benchmark test functions over thirty CEC-2014 real-parameter numerical optimization problems and three constraint engineering design problems. The test results indicate that this algorithm can provide a faster local convergence rate, a higher convergence accuracy, and a lower computational complexity in comparison to traditional whale optimization algorithms and other sophisticated state of the art whale optimizers. Performance wise, it also surpasses many advanced methods for large-scaled complex functions. Furthermore, in this paper we propose a variant of whale army optimization algorithm to specifically address and solve optimizing constrained problems with a high degree of precision.     

Convergence of Inexact Mann Iterations Generated by Nearly Nonexpansive Sequences and Applications

The Mann iterates behave well for nonexpansive mappings for any initial guess in the domain. Our aim in this article is to extend this method to a broad class of inexact fixed point algorithms generated by nearly nonexpansive sequences in Banach spaces and to locate the weak limit of the iterates by its initial guesses. Due to the inexactness, our algorithms become e?ciently applicable for a wider class of problems. As applications, we give convergence theorems for finding solutions of variational inclusion problems and constrained multiple-sets split feasibility problems. Our results are significant refinements and improvements of the corresponding results in the literature.     

The Extragradient Method for Convex Optimization in the Presence of Computational Errors

In this article, we study convergence of the extragradient method for constrained convex minimization problems in a Hilbert space. Our goal is to obtain an ε-approximate solution of the problem in the presence of computational errors, where ε is a given positive number. Most results known in the literature establish convergence of optimization algorithms, when computational errors are summable. In this article, the convergence of the extragradient method for solving convex minimization problems is established for nonsummable computational errors. We show that the the extragradient method generates a good approximate solution, if the sequence of computational errors is bounded from above by a constant.