Constrained optimization in expensive simulation: Novel approach

This article presents a novel heuristic for constrained optimization of computationally expensive random simulation models. One output is selected as objective to be minimized, while other outputs must satisfy given threshold values. Moreover, the simulation inputs must be integer and satisfy linear or nonlinear constraints. The heuristic combines (i) sequentialized experimental designs to specify the simulation input combinations, (ii) Kriging (or Gaussian process or spatial correlation modeling) to analyze the global simulation input/output data resulting from these designs, and (iii) integer nonlinear programming to estimate the optimal solution from the Kriging metamodels. The heuristic is applied to an (s,S)$(s\text{,}S)$ inventory system and a call-center simulation, and compared with the popular commercial heuristic OptQuest embedded in the Arena versions 11 and 12. In these two applications the novel heuristic outperforms OptQuest in terms of number of simulated input combinations and quality of the estimated optimum.     

A fast Pareto genetic algorithm approach for solving expensive multiobjective optimization problems

We present a new multiobjective evolutionary algorithm (MOEA), called fast Pareto genetic algorithm (FastPGA), for the simultaneous optimization of multiple objectives where each solution evaluation is computationally- and/or financially-expensive. This is often the case when there are time or resource constraints involved in finding a solution. FastPGA utilizes a new ranking strategy that utilizes more information about Pareto dominance among solutions and niching relations. New genetic operators are employed to enhance the proposed algorithm’s performance in terms of convergence behavior and computational effort as rapid convergence is of utmost concern and highly desired when solving expensive multiobjective optimization problems (MOPs). Computational results for a number of test problems indicate that FastPGA is a promising approach. FastPGA yields similar performance to that of the improved nondominated sorting genetic algorithm (NSGA-II), a widely-accepted benchmark in the MOEA research community. However, FastPGA outperforms NSGA-II when only a small number of solution evaluations are permitted, as would be the case when solving expensive MOPs.     

A metamodel-assisted evolutionary algorithm for expensive optimization

Expensive optimization aims to find the global minimum of a given function within a very limited number of function evaluations. It has drawn much attention in recent years. The present expensive optimization algorithms focus their attention on metamodeling techniques, and call existing global optimization algorithms as subroutines. So it is difficult for them to keep a good balance between model approximation and global search due to their two-part property. To overcome this difficulty, we try to embed a metamodel mechanism into an efficient evolutionary algorithm, low dimensional simplex evolution (LDSE), in this paper. The proposed algorithm is referred to as the low dimensional simplex evolution extension (LDSEE). It is inherently parallel and self-contained. This renders it very easy to use. Numerical results show that our proposed algorithm is a competitive alternative for expensive optimization problems.     

A metamodel-assisted evolutionary algorithm for expensive optimization

Expensive optimization aims to find the global minimum of a given function within a very limited number of function evaluations. It has drawn much attention in recent years. The present expensive optimization algorithms focus their attention on metamodeling techniques, and call existing global optimization algorithms as subroutines. So it is difficult for them to keep a good balance between model approximation and global search due to their two-part property. To overcome this difficulty, we try to embed a metamodel mechanism into an efficient evolutionary algorithm, low dimensional simplex evolution (LDSE), in this paper. The proposed algorithm is referred to as the low dimensional simplex evolution extension (LDSEE). It is inherently parallel and self-contained. This renders it very easy to use. Numerical results show that our proposed algorithm is a competitive alternative for expensive optimization problems.     

A time-space Kriging-based sequential metamodeling approach for multi-objective crashworthiness optimization

A time-space Kriging-based sequential metamodeling approach is proposed for multi-objective crashworthiness optimization (MOCO) in this paper. By defining the novel time-space design criteria, the constructed metamodels for the optimization objectives include the characteristic mechanical responses with respect to both the structural space domain and crash time domain, compared to standard metrics with the extremum of the time history of the entire structure. The adaptive addition of new samples is performed to gradually improve the approximation accuracy during the optimization with the guidance of an adaptive weighted sum method. The effectiveness of the proposed method is demonstrated by investigating a multi-cell thin-walled crashworthiness design problem. Finally, its effectiveness in practical engineering is validated by the crashworthiness design for a vehicle under full-overlap frontal crash loadcase.     

A new approach to constrained function optimization

A new approach to the constrained function optimization problem is presented. It is shown that the ordinary Lagrange multiplier method and the penalty function method may be generalized and combined, and the new concept ofmultiplier function is introduced. The problem may then be converted into an unconstrained well-conditioned optimization problem. Methods for numerical solution are discussed, and new algorithms are derived.The author wishes to express his gratitude to Professor K. J. Åström for his encouragement and assistance and to Professor P. Falb for valuable suggested improvements. This work was supported by the Swedish Board for Technical Development, Contract No. 70-337/U270.     

Particle swarm with radial basis function surrogates for expensive black-box optimization

This paper develops the OPUS (Optimization by Particle swarm Using Surrogates) framework for expensive black-box optimization. In each iteration, OPUS considers multiple trial positions for each particle in the swarm and uses a surrogate model to identify the most promising trial position. Moreover, the current overall best position is refined by finding the global minimum of the surrogate in the neighborhood of that position. OPUS is implemented using an RBF surrogate and the resulting OPUS-RBF algorithm is applied to a 36-D groundwater bioremediation problem, a 14-D watershed calibration problem, and ten mostly 30-D test problems. OPUS-RBF is compared with a standard PSO, CMA-ES, two other surrogate-assisted PSO algorithms, and an RBF-assisted evolution strategy. The numerical results suggest that OPUS-RBF is promising for expensive black-box optimization.     

A sequential auction involving asymmetrically-informed bidders

An example is given of a sequential auction in which, at equilibrium, the expected profit of an informed bidder may be strictly less than the expected profit of an uninformed bidder. This phenomenon is interpreted in terms of the internal game between a player's “types” which arises in a setting of incomplete information.     

Parallel radial basis function methods for the global optimization of expensive functions

We introduce a master–worker framework for parallel global optimization of computationally expensive functions using response surface models. In particular, we parallelize two radial basis function (RBF) methods for global optimization, namely, the RBF method by Gutmann [Gutmann, H.M., 2001a. A radial basis function method for global optimization. Journal of Global Optimization 19(3), 201–227] (Gutmann-RBF) and the RBF method by Regis and Shoemaker [Regis, R.G., Shoemaker, C.A., 2005. Constrained global optimization of expensive black box functions using radial basis functions, Journal of Global Optimization 31, 153–171] (CORS-RBF). We modify these algorithms so that they can generate multiple points for simultaneous evaluation in parallel. We compare the performance of the two parallel RBF methods with a parallel multistart derivative-based algorithm, a parallel multistart derivative-free trust-region algorithm, and a parallel evolutionary algorithm on eleven test problems and on a 6-dimensional groundwater bioremediation application. The results indicate that the two parallel RBF algorithms are generally better than the other three alternatives on most of the test problems. Moreover, the two parallel RBF algorithms have comparable performances on the test problems considered. Finally, we report good speedups for both parallel RBF algorithms when using a small number of processors.     

RBFOpt: an open-source library for black-box optimization with costly function evaluations

We consider the problem of optimizing an unknown function given as an oracle over a mixed-integer box-constrained set. We assume that the oracle is expensive to evaluate, so that estimating partial derivatives by finite differences is impractical. In the literature, this is typically called a black-box optimization problem with costly evaluation. This paper describes the solution methodology implemented in the open-source library RBFOpt, available on COIN-OR. The algorithm is based on the Radial Basis Function method originally proposed by Gutmann (J Glob Optim 19:201–227, 2001.  https://doi.org/10.1023/A:1011255519438), which builds and iteratively refines a surrogate model of the unknown objective function. The two main methodological contributions of this paper are an approach to exploit a noisy but less expensive oracle to accelerate convergence to the optimum of the exact oracle, and the introduction of an automatic model selection phase during the optimization process. Numerical experiments show that RBFOpt is highly competitive on a test set of continuous and mixed-integer nonlinear unconstrained problems taken from the literature: it outperforms the open-source solvers included in our comparison by a large amount, and performs slightly better than a commercial solver. Our empirical evaluation provides insight on which parameterizations of the algorithm are the most effective in practice. The software reviewed as part of this submission was given the Digital Object Identifier (DOI)  https://doi.org/10.5281/zenodo.597767.     

A proximal trust-region method for nonsmooth optimization with inexact function and gradient evaluations

Mathematical Programming - Many applications require minimizing the sum of smooth and nonsmooth functions. For example, basis pursuit denoising problems in data science require minimizing a measure...     

A sequential implicit function theorem for iterative solution of generalized equation involving point-based approximation

In this paper we study the local convergence of the method

$$0\in A(p,x_{k},x_{k+1})+F(x_{k+1}).$$     

Using parallel function evaluations to improve hessian approximation for unconstrained optimization

This paper presents a new class of methods for solving unconstrained optimization problems on parallel computers. The methods are intended to solve small to moderate dimensional problems where function and derivative evaluation is the dominant cost. They utilize multiple processors to evaluate the function, (finite difference) gradient, and a portion of the finite difference Hessian simultaneously at each iterate. We introduce three types of new methods, which all utilize the new finite difference Hessian information in forming the new Hessian approximation at each iteration; they differ in whether and how they utilize the standard secant information from the current step as well. We present theoretical analyses of the rate of convergence of several of these methods. We also present computational results which illustrate their performance on parallel computers when function evaluation is expensive.Research supported by AFOSR grant AFOSR-85-0251, ARO contract DAAG 29-84-K-0140, NSF grant DCR-8403483, and NFS cooperative agreement DCR -8420944.     

一个包含Smarandache函数的复合函数

对任意正整数n,著名的Smarandache函数S(n)定义为最小的正整数m使得n|m!,或者S(n)=min{m∶n|m!,m∈N}.而函数Z(n)定义为最小的正整数k使得n≤k(k 1)/2,即就是Z(n)=min{k:n≤k(k 1)/2}.本文的主要目的是利用初等及解析方法研究复合函数S(Z(n))的均值,并给出一个较强的渐近公式.     

A norandom variational approach to stochastic linear quadratic Gaussian optimization involving fractional noises (FLQG)

It is shown that the problem of minimizing (maximizing) a quadratic cost functional (quadratic gain functional) given the dynamicsdx=(fx+gu)dt+hdb(t,a) whereb(t, a) is a fractional Brownian motion of ordera, 0<2a<1, can be solved completely (and meaningfully!) by using the dynamical equations of the moments ofx(t). The key is to use fractional Taylor's series to obtain a relation between differential and differential of fractional order.     

A quasi-multistart framework for global optimization of expensive functions using response surface models

We present the AQUARS (A QUAsi-multistart Response Surface) framework for finding the global minimum of a computationally expensive black-box function subject to bound constraints. In a traditional multistart approach, the local search method is blind to the trajectories of the previous local searches. Hence, the algorithm might find the same local minima even if the searches are initiated from points that are far apart. In contrast, AQUARS is a novel approach that locates the promising local minima of the objective function by performing local searches near the local minima of a response surface (RS) model of the objective function. It ignores neighborhoods of fully explored local minima of the RS model and it bounces between the best partially explored local minimum and the least explored local minimum of the RS model. We implement two AQUARS algorithms that use a radial basis function model and compare them with alternative global optimization methods on an 8-dimensional watershed model calibration problem and on 18 test problems. The alternatives include EGO, GLOBALm, MLMSRBF (Regis and Shoemaker in INFORMS J Comput 19(4):497–509, 2007), CGRBF-Restart (Regis and Shoemaker in J Global Optim 37(1):113–135 2007), and multi level single linkage (MLSL) coupled with two types of local solvers: SQP and Mesh Adaptive Direct Search (MADS) combined with kriging. The results show that the AQUARS methods generally use fewer function evaluations to identify the global minimum or to reach a target value compared to the alternatives. In particular, they are much better than EGO and MLSL coupled to MADS with kriging on the watershed calibration problem and on 15 of the test problems.     

Fractal interpolation: a sequential approach

Fractal interpolation is a modern technique to fit and analyze scientific data. We develop a new class of fractal interpolation functions which converge to a data generating(original) function for any choice of the scaling factors. Consequently, our method offers an alternative to the existing fractal interpolation functions(FIFs). We construct a sequence of α-FIFs using a suitable sequence of iterated function systems(IFSs). Without imposing any condition on the scaling vector, we establish constrained interpolation by using fractal functions. In particular,the constrained interpolation discussed herein includes a method to obtain fractal functions that preserve positivity inherent in the given data. The existence of Cr-α-FIFs is investigated. We identify suitable conditions on the associated scaling factors so that α-FIFs preserve r-convexity in addition to the C~r-smoothness of original function.     

Bilevel aggregator-prosumers' optimization problem in real-time: A convex optimization approach

This paper proposes a Real-Time Market (RTM) platform for an aggregator and its corresponding prosumers to participate in the electricity wholesale market. The proposed energy market platform is modeled as a bilevel optimization problem where the aggregator and the prosumers are considered as self-interested agents. We present a convex optimization problem which can capture a subset of the set of global optima of the bilevel problem as its optimal solution.     

A free boundary optimization problem involving weighted areas

In the class of all strip-like regions (Fig. 1) satisfying a certain condition involving weighted areas, we determine (in the sense of existence, uniqueness, and characterization) the region with the least capacitance.

---

Zusammenfassung Unter allen streifenartigen Gebieten (Fig. 1) die ein bestimmte Bedingung betreffend gewichteter Flächen erfüllen, bestimmen wir (im Sinne von Existenz, Eindeutigkeit und Charakterisierung) dasjenige mit kleinster Kapazität.

---

---

These results were presented at the conference on the numerical treatment of differential equations with special consideration of free-boundary problems, Mathematisches Forschungsinstitut Oberwolfach, 1.5.1977.