A gradient projection method for solving continuous games

This paper develops a procedure for numerically solving continuous games (and also matrix games) using a gradient projection method in a general Hilbert space setting. First, we analyze the symmetric case. Our approach is to introduce a functional which measures how far a strategy deviates from giving zero value (i.e., how near the strategy is to being optimal). We then incorporate this functional into a nonlinear optimization problem with constraints and solve this problem using the gradient projection algorithm. The convergence is studied via the corresponding steepest-descent differential equation. The differential equation is a nonlinear initial-value problem in a Hilbert space; thus, we include a proof of existence and uniqueness of its solution. Finally, nonsymmetric games are handled using the symmetrization techniques of Ref. 1.     

Vector-valued Lg-splines II. Smoothing splines

In the first paper of this series, Lg-spline theory was extended to the vector-valued interpolating case. Here this work is complemented by giving the extension for smoothing splines. The problem is formulated as a constrained minimum norm problem in a reproducing kernel Hilbert space, and solved recursively using a congruent stochastic estimation model.     

Global optimization of general constrained grey-box models: new method and its application to constrained PDEs for pressure swing adsorption

This paper introduces a novel methodology for the global optimization of general constrained grey-box problems. A grey-box problem may contain a combination of black-box constraints and constraints with a known functional form. The novel features of this work include (i) the selection of initial samples through a subset selection optimization problem from a large number of faster low-fidelity model samples (when a low-fidelity model is available), (ii) the exploration of a diverse set of interpolating and non-interpolating functional forms for representing the objective function and each of the constraints, (iii) the global optimization of the parameter estimation of surrogate functions and the global optimization of the constrained grey-box formulation, and (iv) the updating of variable bounds based on a clustering technique. The performance of the algorithm is presented for a set of case studies representing an expensive non-linear algebraic partial differential equation simulation of a pressure swing adsorption system for \(\hbox {CO}_{2}\). We address three significant sources of variability and their effects on the consistency and reliability of the algorithm: (i) the initial sampling variability, (ii) the type of surrogate function, and (iii) global versus local optimization of the surrogate function parameter estimation and overall surrogate constrained grey-box problem. It is shown that globally optimizing the parameters in the parameter estimation model, and globally optimizing the constrained grey-box formulation has a significant impact on the performance. The effect of sampling variability is mitigated by a two-stage sampling approach which exploits information from reduced-order models. Finally, the proposed global optimization approach is compared to existing constrained derivative-free optimization algorithms.     

OPTIMALITY CONDITIONS AND APPROXIMATE OPTIMALITY CONDITIONS IN LOCALLY LIPSCHITZ VECTOR OPTIMIZATION

Abstract

In this paper, we study constrained locally Lipschitz vector optimization problems in which the objective and constraint spaces are Hilbert spaces, the decision space is a Banach space, the dominating cone and the constraint cone may be with empty interior. Necessary optimality conditions for this type of optimization problems are derived. A sufficient condition for the existence of approximate efficient solutions to a general vector optimization problem is presented. Necessary conditions for approximate efficient solutions to a constrained locally Lipschitz optimization problem is obtained.     

On the algorithm of smoothing by a spline with bilateral constraints

In this paper, the problem of constructing a spline σ in the Hilbert space, which satisfies the bilateral constraints z ^? ≤ Aσ ≤ z ⁺ with a linear operator A and minimizes the squared Hilbert seminorm is studied. A solution to this problem can be obtained with convex programming iterative methods, in particular, with the gradient projection method. A modification of the gradient projection method is proposed, which allows one to find a set of active constraints with a smaller number of iterations. The efficiency of the modification proposed is demonstrated in the problem of approximation with a pseudolinear bivariate spline.     

Robust nonlinear optimization with conic representable uncertainty set

The robust optimization methodology is known as a popular method dealing with optimization problems with uncertain data and hard constraints. This methodology has been applied so far to various convex conic optimization problems where only their inequality constraints are subject to uncertainty. In this paper, the robust optimization methodology is applied to the general nonlinear programming (NLP) problem involving both uncertain inequality and equality constraints. The uncertainty set is defined by conic representable sets, the proposed uncertainty set is general enough to include many uncertainty sets, which have been used in literature, as special cases. The robust counterpart (RC) of the general NLP problem is approximated under this uncertainty set. It is shown that the resulting approximate RC of the general NLP problem is valid in a small neighborhood of the nominal value. Furthermore a rather general class of programming problems is posed that the robust counterparts of its problems can be derived exactly under the proposed uncertainty set. Our results show the applicability of robust optimization to a wider area of real applications and theoretical problems with more general uncertainty sets than those considered so far. The resulting robust counterparts which are traditional optimization problems make it possible to use existing algorithms of mathematical optimization to solve more complicated and general robust optimization problems.     

Fast Fourier optimization

Many interesting and fundamentally practical optimization problems, ranging from optics, to signal processing, to radar and acoustics, involve constraints on the Fourier transform of a function. It is well-known that the fast Fourier transform (fft) is a recursive algorithm that can dramatically improve the efficiency for computing the discrete Fourier transform. However, because it is recursive, it is difficult to embed into a linear optimization problem. In this paper, we explain the main idea behind the fast Fourier transform and show how to adapt it in such a manner as to make it encodable as constraints in an optimization problem. We demonstrate a real-world problem from the field of high-contrast imaging. On this problem, dramatic improvements are translated to an ability to solve problems with a much finer grid of discretized points. As we shall show, in general, the “fast Fourier” version of the optimization constraints produces a larger but sparser constraint matrix and therefore one can think of the fast Fourier transform as a method of sparsifying the constraints in an optimization problem, which is usually a good thing.     

Robust best approximation with interpolation constraints under ellipsoidal uncertainty: Strong duality and nonsmooth Newton methods

In this paper we present a duality approach for finding a robust best approximation from a set involving interpolation constraints and uncertain inequality constraints in a Hilbert space that is immunized against the data uncertainty using a nonsmooth Newton method. Following the framework of robust optimization, we assume that the input data of the inequality constraints are not known exactly while they belong to an ellipsoidal data uncertainty set. We first show that finding a robust best approximation is equivalent to solving a second-order cone complementarity problem by establishing a strong duality theorem under a strict feasibility condition. We then examine a nonsmooth version of Newton’s method and present their convergence analysis in terms of the metric regularity condition.     

On Quantitative Stability in Optimization and Optimal Control

We study two continuity concepts for set-valued maps that play central roles in quantitative stability analysis of optimization problems: Aubin continuity and Lipschitzian localization. We show that various inverse function theorems involving these concepts can be deduced from a single general result on existence of solutions to an inclusion in metric spaces. As applications, we analyze the stability with respect to canonical perturbations of a mathematical program in a Hilbert space and an optimal control problem with inequality control constraints. For stationary points of these problems, Aubin continuity and Lipschitzian localization coincide; moreover, both properties are equivalent to surjectivity of the map of the gradients of the active constraints combined with a strong second-order sufficient optimality condition.     

Indefinite Abstract Splines with a Quadratic Constraint

We study an extension to Krein spaces of the abstract interpolating spline problem in Hilbert spaces, introduced by M. Atteia. This is a quadratically constrained quadratic programming problem, where the objective function is not convex, while the equality constraint is sign indefinite. We characterize the existence of solutions and, if there are any, we describe the set of solutions as the union of a family of affine manifolds parallel to a fixed subspace, which depend on the original data.

     

Optimality conditions for vector optimization problems with non-cone constraints in image space

ABSTRACT

In this paper, we employ the image space analysis method to investigate a vector optimization problem with non-cone constraints. First, we use the linear and nonlinear separation techniques to establish Lagrange-type sufficient and necessary optimality conditions of the given problem under convexity assumptions and generalized Slater condition. Moreover, we give some characterizations of generalized Lagrange saddle points in image space without any convexity assumptions. Finally, we derive the vectorial penalization for the vector optimization problem with non-cone constraints by a general way.     

Stable iterative Lagrange principle in convex programming as a tool for solving unstable problems

A convex programming problem in a Hilbert space with an operator equality constraint and a finite number of functional inequality constraints is considered. All constraints involve parameters. The close relation of the instability of this problem and, hence, the instability of the classical Lagrange principle for it to its regularity properties and the subdifferentiability of the value function in the problem is discussed. An iterative nondifferential Lagrange principle with a stopping rule is proved for the indicated problem. The principle is stable with respect to errors in the initial data and covers the normal, regular, and abnormal cases of the problem and the case where the classical Lagrange principle does not hold. The possibility of using the stable sequential Lagrange principle for directly solving unstable optimization problems is discussed. The capabilities of this principle are illustrated by numerically solving the classical ill-posed problem of finding the normal solution of a Fredholm integral equation of the first kind.     

On Blaschke products with derivatives in Bergman spaces with normal weights

We generalize a well-known sufficient condition for interpolating sequences for the Hilbert Bergman spaces to other Bergman spaces with normal weights (as defined by Shields and Williams) and obtain new results regarding the membership of the derivative of a Blaschke product or a general inner function in such spaces. We also apply duality techniques to obtain further results of this type and obtain new results about interpolating Blaschke products.     

Epsilon-Ritz Method for Solving a Class of Fractional Constrained Optimization Problems

In this paper, epsilon and Ritz methods are applied for solving a general class of fractional constrained optimization problems. The goal is to minimize a functional subject to a number of constraints. The functional and constraints can have multiple dependent variables, multiorder fractional derivatives, and a group of initial and boundary conditions. The fractional derivative in the problem is in the Caputo sense. The constrained optimization problems include isoperimetric fractional variational problems (IFVPs) and fractional optimal control problems (FOCPs). In the presented approach, first by implementing epsilon method, we transform the given constrained optimization problem into an unconstrained problem, then by applying Ritz method and polynomial basis functions, we reduce the optimization problem to the problem of optimizing a real value function. The choice of polynomial basis functions provides the method with such a flexibility that initial and boundary conditions can be easily imposed. The convergence of the method is analytically studied and some illustrative examples including IFVPs and FOCPs are presented to demonstrate validity and applicability of the new technique.     

Convergence Analysis of a Class of Penalty Methods for Vector Optimization Problems with Cone Constraints

In this paper, we consider convergence properties of a class of penalization methods for a general vector optimization problem with cone constraints in infinite dimensional spaces. Under certain assumptions, we show that any efficient point of the cone constrained vector optimization problem can be approached by a sequence of efficient points of the penalty problems. We also show, on the other hand, that any limit point of a sequence of approximate efficient solutions to the penalty problems is a weekly efficient solution of the original cone constrained vector optimization problem. Finally, when the constrained space is of finite dimension, we show that any limit point of a sequence of stationary points of the penalty problems is a KKT stationary point of the original cone constrained vector optimization problem if Mangasarian–Fromovitz constraint qualification holds at the limit point.This work is supported by the Postdoctoral Fellowship of Hong Kong Polytechnic University.     

Solving a gas-lift optimization problem by dynamic programming

Gas lift is a costly, however indispensable means to recover oil from high-depth reservoirs that entails solving the gas-lift optimization problem, GOP, often in response to variations in the dynamics of the reservoir and economic oscillations. GOP can be cast as a mixed integer nonlinear programming problem whose integer variables decide which oil wells should produce, while the continuous variables allocate the gas-compressing capacity to the active ones. This paper extends the GOP formulation to encompass uncertainties in the oil outflow and precedence constraints imposed on the activation of wells. Recursive solutions are suggested for instances devoid of precedence constraints, as well as instances arising from precedence constraints induced by forests and general acyclic graphs. For the first two classes, pseudo-polynomial algorithms are developed to solve a discretized version of GOP, while the most general version is shown to be NP-Hard in the strong sense. Numerical experiments provide evidence that the approximate algorithms obtained by solving the recurrences produce near-optimal solutions. Further, the family of cover inequalities of the knapsack problem is extended to the gas-lift optimization problem.     

Optimal filter design subject to output sidelobe constraints: Theoretical considerations

The design of filters for detection and estimation in radar and communications systems is considered, with inequality constraints on the maximum output sidelobe levels. A constrained optimization problem in Hilbert space is formulated, incorporating the sidelobe constraints via a partial ordering of continuous functions. Generalized versions (in Hilbert space) of the Kuhn-Tucker and duality theorems allow the reduction of this problem to an unconstrained one in the dual space of regular Borel measures.This research was supported by the National Science Foundation under Grant No. GK-2645, by the National Aeronautics and Space Administration under Grant No. NGL-22-009(124), and by the Australian Research Grants Committee. The authors wish to express their gratitude to Dr. Robert McAulay and Professor Ian Rhodes for various comments and suggestions.     

Coupling the Gradient Method with a General Exterior Penalization Scheme for Convex Minimization

In this paper, we propose and analyze an algorithm that couples the gradient method with a general exterior penalization scheme for constrained or hierarchical minimization of convex functions in Hilbert spaces. We prove that a proper but simple choice of the step sizes and penalization parameters guarantees the convergence of the algorithm to solutions for the optimization problem. We also establish robustness and stability results that account for numerical approximation errors, discuss implementation issues and provide examples in finite and infinite dimension.     

General Iterative Methods for Semigroups of Nonexpansive Mappings Related to Optimization Problems

Abstract

In this article, we introduce two general iterative methods for a certain optimization problem of which the constrained set is the set of the solution set of the variational inequality problem for the fixed point set of nonexpansive semigroups in Hilbert spaces. Under some control conditions, we establish the strong convergence of the proposed methods to the fixed point set, which is the unique solution of a certain optimization problem. Applications to solutions of equilibrium problems are also presented.     

A New Conical Regularization for Some Optimization and Optimal Control Problems: Convergence Analysis and Finite Element Discretization

In this article, we study an abstract constrained optimization problem that appears commonly in the optimal control of linear partial differential equations. The main emphasis of the present study is on the case when the ordering cone for the optimization problem has an empty interior. To circumvent this major difficulty, we propose a new conical regularization approach in which the main idea is to replace the ordering cone by a family of dilating cones. We devise a general regularization approach and use it to give a detailed convergence analysis for the conical regularization as well as a related regularization approach. We showed that the conical regularization approach leads to a family of optimization problems that admit regular multipliers. The approach remains valid in the setting of general Hilbert spaces and it does not require any sort of compactness or positivity condition on the operators involved. One of the main advantages of the approach is that it is amenable for numerical computations. We consider four different examples, two of them elliptic control problems with state constraints, and present numerical results that completely support our theoretical results and confirm the numerical feasibility of our approach. The motivation for the conical regularization is to overcome the difficulties associated with the lack of Slater's type constraint qualification, which is a common hurdle in numerous branches of applied mathematics including optimal control, inverse problems, vector optimization, set-valued optimization, sensitivity analysis, variational inequalities, among others.