Reduced Functions, Gradients and Hessians from Fixed-Point Iterations for State Equations

In design optimization and parameter identification, the objective, or response function(s) are typically linked to the actually independent variables through equality constraints, which we will refer to as state equations. Our key assumption is that it is impossible to form and factor the corresponding constraint Jacobian, but one has instead some fixed-point algorithm for computing a feasible state, given any reasonable value of the independent variables. Assuming that this iteration is eventually contractive, we will show how reduced gradients (Jacobians) and Hessians (in other words, the total derivatives) of the response(s) with respect to the independent variables can be obtained via algorithmic, or automatic, differentiation (AD). In our approach the actual application of the so-called reverse, or adjoint differentiation mode is kept local to each iteration step. Consequently, the memory requirement is typically not unduly enlarged. The resulting approximating Lagrange multipliers are used to compute estimates of the reduced function values that can be shown to converge twice as fast as the underlying state space iteration. By a combination with the forward mode of AD, one can also obtain extra-accurate directional derivatives of the reduced functions as well as feasible state space directions and the corresponding reduced or projected Hessians of the Lagrangian. Our approach is verified by test calculations on an aircraft wing with two responses, namely, the lift and drag coefficient, and two variables, namely, the angle of attack and the Mach number. The state is a 2-dimensional flow field defined as solution of the discretized Euler equation under transonic conditions.     

On a one-dimensional optimization problem derived from the efficiency analysis of Newton-PCG-like algorithms

The Newton-PCG (preconditioned conjugate gradient) like algorithms are usually very efficient. However, their efficiency is mainly supported by the numerical experiments. Recently, a new kind of Newton-PCG-like algorithms is derived in (J. Optim. Theory Appl. 105 (2000) 97; Superiority analysis on truncated Newton method with preconditioned conjugate gradient technique for optimization, in preparation) by the efficiency analysis. It is proved from the theoretical point of view that their efficiency is superior to that of Newton's method for the special cases where Newton's method converges with precise Q-order 2 and α(⩾2), respectively. In the process of extending such kind of algorithms to the more general case where Newton's method has no fixed convergence order, the first is to get the solutions to the one-dimensional optimization problems with many different parameter values of α. If these problems were solved by numerical method one by one, the computation cost would reduce the efficiency of the Newton-PCG algorithm, and therefore is unacceptable. In this paper, we overcome the difficulty by deriving an analytic expression of the solution to the one-dimensional optimization problem with respect to the parameter α.     

Switching Time and Parameter Optimization in Nonlinear Switched Systems with Multiple Time-Delays

In this paper, we consider a dynamic optimization problem involving a general switched system that evolves by switching between several subsystems of nonlinear delay-differential equations. The optimization variables in this system consist of: (1) the times at which the subsystem switches occur; and (2) a set of system parameters that influence the subsystem dynamics. We first establish the existence of the partial derivatives of the system state with respect to both the switching times and the system parameters. Then, on the basis of this result, we show that the gradient of the cost function can be computed by solving the state system forward in time followed by a costate system backward in time. This gradient computation procedure can be combined with any gradient-based optimization method to determine the optimal switching times and parameters. We propose an effective optimization algorithm based on this idea. Finally, we consider three numerical examples, one involving the 1,3-propanediol fed-batch production process, to illustrate the effectiveness and applicability of the proposed algorithm.     

A new proof of a theorem of J. Moser concerning the restricted problem of three bodies

In a recent paper [Barrar (1965)], we have shown that the result ofR. Arenstorf (1963) on the existence of periodic orbits of the second kind for the restricted problem of three bodies can be very readily obtained with the use of Delaunay or Poincaré variables. In the present paper we will show that the results ofJ. Moser (1953) can also be more readily obtained with the use of Poincaré variables.Moser, dealing with the restricted problem of three bodies, demonstrated the existence of periodic solutions that close after many revolutions and are near periodic solutions of the first kind.     

Integrated design and optimization of a complex nonlinear hybrid electric powertrain including simulations in time domain

This paper deals with the design and optimization of hybrid electric powertrains. Therefore basic relations of the behavior of hybrid electric powertrain systems and the controller design are introduced. Based on models of typical hybrid electric system components principal optimization approaches with respect to performance parameters like efficiency, availability, lifetime, etc. are shown. Hereby an optimization algorithm based on a global optimization technique is applied. Using the example of a fuel cell based hybrid electric powertrain system the approaches are introduced and compared to each using time-domain simulations integrated in optimization algorithms. The results show that both approaches are appropriate to design the system as well as the controllers. (© 2011 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim)     

Systematic occurrence of repeated eigenvalues in structural optimization

Examples of finite-dimensional structural design problems with constraints on natural frequency and buckling are used to demonstrate that repeated eigenvalues may be expected to occur when a structure is optimized. It is shown that a repeated eigenvalue is not generally differentiable with respect to design variables. Directional derivatives are shown to exist, and a method of calculating directional derivatives is given. Necessary conditions of optimality are derived and applied to a vibration optimization problem. Extensions of the theory to distributed-parameter structures and numerical methods are outlined.This research was supported by NSF Grant No. CMS-80-05677     

Upper bound limit analysis using the weak form quadrature element method

Limit analysis is a useful tool for design and safety assessment of structures in civil and geotechnical engineering. In the present study, a newly developed high order algorithm-the weak form quadrature element method is reformulated for upper bound limit analysis. The dual formulations of the kinematic theorem are employed with the nodal stresses chosen as the optimization variables. The weak form equilibrium constraint is numerically integrated by Lobatto integration and then the nodal derivatives are approximated by differential quadrature analogue. The resulting optimization problem is formulated as a standard second-order cone programming problem and solved by the optimization toolbox Mosek. This paper aims to improve the efficiency of the existing numerical limit algorithms especially for problems with singularities such as cracked structures and to overcome the well-known volumetric locking occurred for incompressible materials. Some numerical tests are given to show the accuracy and efficiency of the present method.     

A modular and efficient approach to computational modeling and sensitivity analysis of robot and human motion dynamics

In this paper a new class library for the computation of the forward multi-body-system (MBS) dynamics of robots and biomechanical models of human motion is presented. By the developed modular modeling approach the library can be flexibly extended by specific modeling elements like joints with specific geometry or different muscle models and thus can be applied efficiently for a number of dynamic simulation and optimization problems. The library not only provides several methods for solving the forward dynamics problem (like articulated body or composite rigid body algorithms) which can transparently be exchanged. Moreover, the numerical solution of optimal control problems, like in the forward dynamics optimization of human motion, is significantly facilitated by the computation of the sensitivity matrix with respect to the control variables. Examples are given to demonstrate the efficiency of the approach. (© 2012 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim)     

Shape and topology optimization for elliptic boundary value problems using a piecewise constant level set method

The aim of this paper is to propose a variational piecewise constant level set method for solving elliptic shape and topology optimization problems. The original model is approximated by a two-phase optimal shape design problem by the ersatz material approach. Under the piecewise constant level set framework, we first reformulate the two-phase design problem to be a new constrained optimization problem with respect to the piecewise constant level set function. Then we solve it by the projection Lagrangian method. A gradient-type iterative algorithm is presented. Comparisons between our numerical results and those obtained by level set approaches show the effectiveness, accuracy and efficiency of our algorithm.     

Control of a mechanical aeration process via topological sensitivity analysis

The topological sensitivity analysis method gives the variation of a criterion with respect to the creation of a small hole in the domain. In this paper, we use this method to control the mechanical aeration process in eutrophic lakes. A simplified model based on incompressible Navier–Stokes equations is used, only considering the liquid phase, which is the dominant one. The injected air is taken into account through local boundary conditions for the velocity, on the injector holes. A 3D numerical simulation of the aeration effects is proposed using a mixed finite element method. In order to generate the best motion in the fluid for aeration purposes, the optimization of the injector location is considered. The main idea is to carry out topological sensitivity analysis with respect to the insertion of an injector. Finally, a topological optimization algorithm is proposed and some numerical results, showing the efficiency of our approach, are presented.     

Removal of malaria-infected red blood cells using magnetic cell separators: A computational study

High gradient magnetic field separators have been widely used in a variety of biological applications. Recently, the use of magnetic separators to remove malaria-infected red blood cells (pRBCs) from blood circulation in patients with severe malaria has been proposed in a dialysis-like treatment. The capture efficiency of this process depends on many interrelated design variables and constraints such as magnetic pole array pitch, chamber height, and flow rate. In this paper, we model the malaria-infected RBCs (pRBCs) as paramagnetic particles suspended in a Newtonian fluid. Trajectories of the infected cells are numerically calculated inside a micro-channel exposed to a periodic magnetic field gradient. First-order stiff ordinary differential equations (ODEs) governing the trajectory of particles under periodic magnetic fields due to an array of wires are solved numerically using the 1st to 5th order adaptive step Runge-Kutta solver. The numerical experiments show that in order to achieve a capture efficiency of 99% for the pRBCs it is required to have a longer length than 80 mm; this implies that in principle, using optimization techniques the length could be adjusted, i.e., shortened to achieve 99% capture efficiency of the pRBCs.     

Parameter optimization using algorithmic differentiation in a reduced-form model of the Atlantic thermohaline circulation

Optimizing some model parameters a reduced-form model of the Atlantic thermohaline circulation is fitted to data provided by a comprehensive climate model. Different techniques to compute stationary states of the reduced-form model are discussed. The fitting problem is formulated as weighted least-squares optimization problem with non-linear constraints that enforce a proper representation of the present climate. Possible formulations of the optimization problem are presented and compared with respect to their numerical treatment. The technique of algorithmic or automatic differentiation (AD) is used to provide gradient information that can be used in the optimization. The application of the AD software is described in detail and numerical results are given.     

Extension to Markov processes of a result by A. Wald about the consistency of the maximum likelihood estimate

Summary In this note the proof of the consistency of a maximum likelihood estimate (MLE) obtained by Wald in [7] in the case of independent and identically distributed random variables is extended to the case of Markov processes.There is an extensive literature about the existence of a MLE and its consistency, most of which includes the assumption of the existence of derivatives of the densities with respect to the parameter involved. (See, for example, [2] and other references cited there.) Even under the rather strong assumption of pointwise differentiability of densities, and other additional regularity conditions, the problem of existence and consistency of a MLE has not been solved satisfactorily. (See, for example, [1], [2], [4], [6].) On the other hand, there have appeared papers like [3], where the consistency of a MLE is proved for processes with dependent random variables, and without the usual differentiability assumptions. The conditions used in the present paper are, however, of a different nature from those imposed in [3], and also are slightly different from Wald's assumptions in [7]. To our knowledge, a proof of consistency of a MLE under conditions similar to the ones used here has not appeared in the literature.I would like to take this opportunity to thank Professor L. LeCam for a number of remarks in connection with this paper.Prepared with the partial support of the National Science Foundation, Grant GP-10.     

A comparative study of SQP-type algorithms for nonlinear and nonconvex mixed-integer optimization

We present numerical results of a comparative study of codes for nonlinear and nonconvex mixed-integer optimization. The underlying algorithms are based on sequential quadratic programming (SQP) with stabilization by trust-regions, linear outer approximations, and branch-and-bound techniques. The mixed-integer quadratic programming subproblems are solved by a branch-and-cut algorithm. Second order information is updated by a quasi-Newton update formula (BFGS) applied to the Lagrange function for continuous, but also for integer variables. We do not require that the model functions can be evaluated at fractional values of the integer variables. Thus, partial derivatives with respect to integer variables are replaced by descent directions obtained from function values at neighboring grid points, and the number of simulations or function evaluations, respectively, is our main performance criterion to measure the efficiency of a code. Numerical results are presented for a set of 100 academic mixed-integer test problems. Since not all of our test examples are convex, we reach the best-known solutions in about 90 % of the test runs, but at least feasible solutions in the other cases. The average number of function evaluations of the new mixed-integer SQP code is between 240 and 500 including those needed for one- or two-sided approximations of partial derivatives or descent directions, respectively. In addition, we present numerical results for a set of 55 test problems with some practical background in petroleum engineering.     

Automatic Differentiation for MATLAB Programs

Derivative information is required in numerous applications, including sensitivity analysis and numerical optimization. For simple functions, symbolic differentiation–done either manually or with a computer algebra system–can provide the derivatives, whereas divided differences (DD) have been used traditionally for functions defined by (potentially very complex) computer programs, even if only approximate values can be obtained this way. An alternative approach for such functions is automatic differentiation (AD), yielding exact derivatives at often lower cost than DD, and without restrictions on the program complexity. In this paper we compare the functionality and describe the use of ADMIT/ADMAT and ADiMat. These two AD tools provide derivatives for programs written in the MATLAB language, which is widely used for prototype and production software in scientific and engineering applications. While ADMIT/ADMAT implements a pure operator overloading approach of AD, ADiMat also employes source transformation techniques.     

Convergence analysis of a nonlinear Lagrange algorithm for general nonlinear constrained optimization problems

The convergence analysis of a nonlinear Lagrange algorithm for solving nonlinear constrained optimization problems with both inequality and equality constraints is explored in detail. The estimates for the derivatives of the multiplier mapping and the solution mapping of the proposed algorithm are discussed via the technique of the singular value decomposition of matrix. Based on the estimates, the local convergence results and the rate of convergence of the algorithm are presented when the penalty parameter is less than a threshold under a set of suitable conditions on problem functions. Furthermore, the condition number of the Hessian of the nonlinear Lagrange function with respect to the decision variables is analyzed, which is closely related to efficiency of the algorithm. Finally, the preliminary numericM results for several typical test problems are reported.     

An analytical approach to global optimization

Global optimization problems with a few variables and constraints arise in numerous applications but are seldom solved exactly. Most often only a local optimum is found, or if a global optimum is detected no proof is provided that it is one. We study here the extent to which such global optimization problems can be solved exactly using analytical methods. To this effect, we propose a series of tests, similar to those of combinatorial optimization, organized in a branch-and-bound framework. The first complete solution of two difficult test problems illustrates the efficiency of the resulting algorithm. Computational experience with the programbagop, which uses the computer algebra systemmacsyma, is reported on. Many test problems from the compendiums of Hock and Schittkowski and others sources have been solved.The research of the first and the third authors has been supported by AFOSR grants #0271 and #0066 to Rutgers University. Research of the second author has been supported by NSERC grant #GP0036426 and FCAR grants #89EQ4144 and #90NC0305.     

Hybrid evolutionary algorithm with Hermite radial basis function interpolants for computationally expensive adjoint solvers

In this paper, we present an evolutionary algorithm hybridized with a gradient-based optimization technique in the spirit of Lamarckian learning for efficient design optimization. In order to expedite gradient search, we employ local surrogate models that approximate the outputs of a computationally expensive Euler solver. Our focus is on the case when an adjoint Euler solver is available for efficiently computing the sensitivities of the outputs with respect to the design variables. We propose the idea of using Hermite interpolation to construct gradient-enhanced radial basis function networks that incorporate sensitivity data provided by the adjoint Euler solver. Further, we conduct local search using a trust-region framework that interleaves gradient-enhanced surrogate models with the computationally expensive adjoint Euler solver. This ensures that the present hybrid evolutionary algorithm inherits the convergence properties of the classical trust-region approach. We present numerical results for airfoil aerodynamic design optimization problems to show that the proposed algorithm converges to good designs on a limited computational budget.     

Isogeometric structural shape optimization using a fictitious energy regularization

In isogeometric analysis, NURBS basis functions are used as shape functions in an isoparametric finite-element-type discretization. Among other advantageous features, this approach is able to provide exact and smooth representations of a broad class of computational domains with curved boundaries. Therefore, this discretization method seems to be especially convenient for computational shape optimization, where a smooth and CAD-like parametrization of the optimal geometry is desired. Choosing boundary control point coordinates of an isogeometric discretization as design variables, an additional design model can be avoided. However, for a higher number of design variables, typical drawbacks like oscillating boundaries as known from early node-based shape optimization methods appear. To overcome this problem, we propose to use a fictitious energy regularization: the strain energy of a fictitious deformation, which maps the initial to the optimized domain, is employed as a regularizing term in the optimization problem. Moreover, this deformation is used for efficiently moving the dependent nodes within the domain in each step of the optimization process. (© 2011 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim)     

Finding the optimum domain of a nonlinear wave optimal control system by measures

We will explain a new method for obtaining the nearly optimal domain for optimal shape design problems associated with the solution of a nonlinear wave equation. Taking into account the boundary and terminal conditions of the system, a new approach is applied to determine the optimal domain and its related optimal control function with respect to the integral performance criteria, by use of positive Radon measures. The approach, say shape-measure, consists of two steps; first for a fixed domain, the optimal control will be identified by the use of measures. This function and the optimal value of the objective function depend on the geometrical variables of the domain. In the second step, based on the results of the previous one and by applying some convenient optimization techniques, the optimal domain and its related optimal control function will be identified at the same time. The existence of the optimal solution is considered and a numerical example is also given.