Adjoint Coexhausters in Nonsmooth Analysis and Extremality Conditions

In the classical (“smooth”) mathematical analysis, a differentiable function is studied by means of the derivative (gradient in the multidimensional space). In the case of nondifferentiable functions, the tools of nonsmooth analysis are to be employed. In convex analysis and minimax theory, the corresponding classes of functions are investigated by means of the subdifferential (it is a convex set in the dual space), quasidifferentiable functions are treated via the notion of quasidifferential (which is a pair of sets). To study an arbitrary directionally differentiable function, the notions of upper and lower exhausters (each of them being a family of convex sets) are used. It turns out that conditions for a minimum are described by an upper exhauster, while conditions for a maximum are stated in terms of a lower exhauster. This is why an upper exhauster is called a proper one for the minimization problem (and an adjoint exhauster for the maximization problem) while a lower exhauster will be referred to as a proper one for the maximization problem (and an adjoint exhauster for the minimization problem). The directional derivatives (and hence, exhausters) provide first-order approximations of the increment of the function under study. These approximations are positively homogeneous as functions of direction. They allow one to formulate optimality conditions, to find steepest ascent and descent directions, to construct numerical methods. However, if, for example, the maximizer of the function is to be found, but one has an upper exhauster (which is not proper for the maximization problem), it is required to use a lower exhauster. Instead, one can try to express conditions for a maximum in terms of upper exhauster (which is an adjoint one for the maximization problem). The first to get such conditions was Roshchina. New optimality conditions in terms of adjoint exhausters were recently obtained by Abbasov. The exhauster mappings are, in general, discontinuous in the Hausdorff metric, therefore, computational problems arise. To overcome these difficulties, the notions of upper and lower coexhausters are used. They provide first-order approximations of the increment of the function which are not positively homogeneous any more. These approximations also allow one to formulate optimality conditions, to find ascent and descent directions (but not the steepest ones), to construct numerical methods possessing good convergence properties. Conditions for a minimum are described in terms of an upper coexhauster (which is, therefore, called a proper coexhauster for the minimization problem) while conditions for a maximum are described in terms of a lower coexhauster (which is called a proper one for the maximization problem). In the present paper, we derive optimality conditions in terms of adjoint coexhausters.     

Proper and adjoint exhausters in nonsmooth analysis: optimality conditions

The notions of upper and lower exhausters represent generalizations of the notions of exhaustive families of upper convex and lower concave approximations (u.c.a., l.c.a.). The notions of u.c.a.’s and l.c.a.’s were introduced by Pshenichnyi (Convex Analysis and Extremal Problems, Series in Nonlinear Analysis and its Applications, 1980), while the notions of exhaustive families of u.c.a.’s and l.c.a.’s were described by Demyanov and Rubinov in Nonsmooth Problems of Optimization Theory and Control, Leningrad University Press, Leningrad, 1982. These notions allow one to solve the problem of optimization of an arbitrary function by means of Convex Analysis thus essentially extending the area of application of Convex Analysis. In terms of exhausters it is possible to describe extremality conditions, and it turns out that conditions for a minimum are expressed via an upper exhauster while conditions for a maximum are formulated in terms of a lower exhauster (Abbasov and Demyanov (2010), Demyanov and Roshchina (Appl Comput Math 4(2): 114–124, 2005), Demyanov and Roshchina (2007), Demyanov and Roshchina (Optimization 55(5–6): 525–540, 2006)). This is why an upper exhauster is called a proper exhauster for minimization problems while a lower exhauster is called a proper one for maximization problems. The results obtained provide a simple geometric interpretation and allow one to construct steepest descent and ascent directions. Until recently, the problem of expressing extremality conditions in terms of adjoint exhausters remained open. Demyanov and Roshchina (Appl Comput Math 4(2): 114–124, 2005), Demyanov and Roshchina (Optimization 55(5–6): 525–540, 2006) was the first to derive such conditions. However, using the conditions obtained (unlike the conditions expressed in terms of proper exhausters) it was not possible to find directions of descent and ascent. In Abbasov (2011) new extremality conditions in terms of adjoint exhausters were discovered. In the present paper, a different proof of these conditions is given and it is shown how to find steepest descent and ascent conditions in terms of adjoint exhausters. The results obtained open the way to constructing numerical methods based on the usage of adjoint exhausters thus avoiding the necessity of converting the adjoint exhauster into a proper one.     

Extremum conditions for a nonsmooth function in terms of exhausters and coexhausters

The notions of upper and lower exhausters and coexhausters are discussed and necessary conditions for an unconstrained extremum of a nonsmooth function are derived. The necessary conditions for a minimum are formulated in terms of an upper exhauster (coexhauster) and the necessary conditions for a maximum are formulated in terms of a lower exhauster (coexhauster). This involves the problem of transforming an upper exhauster (coexhauster) into a lower exhauster (coexhauster) and vice versa. The transformation is carried out by means of a conversion operation (converter). Second-order approximations obtained with the help of second-order (upper and lower) coexhausters are considered. It is shown how a secondorder upper coexhauster can be converted into a lower coexhauster and vice versa. This problem is reduced to using a first-order conversion operator but in a space of a higher dimension. The obtained result allows one to construct second-order methods for the optimization of nonsmooth functions (Newton-type methods).     

Exhausters, optimality conditions and related problems

The notions of exhausters were introduced in (Demyanov, Exhauster of a positively homogeneous function, Optimization 45, 13–29 (1999)). These dual tools (upper and lower exhausters) can be employed to describe optimality conditions and to find directions of steepest ascent and descent for a very wide range of nonsmooth functions. What is also important, exhausters enjoy a very good calculus (in the form of equalities). In the present paper we review the constrained and unconstrained optimality conditions in terms of exhausters, introduce necessary and sufficient conditions for the Lipschitzivity and Quasidifferentiability, and also present some new results on relationships between exhausters and other nonsmooth tools (such as the Clarke, Michel-Penot and Fréchet subdifferentials).     

Exhausters af a positively homogeneous function *

Notions of upper exhauster and lower exhauster of a positively homogeneous (of the first degree) function h: ? ⁿ →? are introduced. They are linked to exhaustive families of upper convex and lower concave approximations of the function h. The pair of an upper exhauster and a lower exhauster is called a biexhauster of h. A calculus for biexhausters is described (in particular, a composition theorem is formulated). The problem of minimality of exhausters is stated. Necessary and sufficient conditions for a constrained minimum and a constrained maximum of a directionally differentiable function f: ? ⁿ →? are formulated in terms of exhausters of the directional derivative of f. In general, they are described by means of exhausters of the Hadamard upper and lower directional derivatives of the function f. To formulate conditions for a minimum, an upper exhauster is employed while conditions for a maximum are formulated via a lower exhauster of the respective directional derivative (the Hadamard lower derivative for a minimum and the Hadamard upper derivative for a maximum).

If a point x _o is not stationary then directions of steepest ascent and descent can also be calculated by means of exhausters.     

Comparison Between Quasidifferentials and Exhausters

Directional derivatives play one of the major roles in optimization. Optimality conditions can be described in terms of these objects. These conditions, however, are not constructive. To overcome this problem, one has to represent the directional derivative in special forms. Two such forms are quasidifferentials and exhausters proposed by V.F. Demyanov. Quasidifferentials were introduced in 1980s. Optimality conditions in terms of these objects were developed by L.N. Polyakova and V.F. Demyanov. It was described how to find directions of steepest descent and ascent when these conditions are not satisfied. This paved a way for constructing new optimization algorithms. Quasidifferentials allow one to treat a wide class of functions. V.F. Demyanov introduced the notion of exhausters in 2000s to expand the class of functions that can be treated. It should be noted that a great contribution to the emergence of this notion was made by B.N. Pshenichny and A.M. Rubinov. In this work it is shown that exhausters not only allow one to treat a wider class of functions than quasidifferentials (since every quasidifferentiable function has exhausters) but is also preferable even for quasidifferentiable functions when solving nonsmooth optimization problems.     

Reduction and Minimality of Coexhausters

V.F. Demyanov introduced exhausters for the study of nonsmooth functions. These are families of convex compact sets that enable one to represent the main part of the increment of a considered function in a neighborhood of the studied point as MaxMin or MinMax of linear functions. Optimality conditions were described in terms of these objects. This provided a way for constructing new algorithms for solving nondifferentiable optimization problems. Exhausters are defined not uniquely. It is obvious that the smaller an exhauster, the less are the computational expenses when working with it. Thus, the problem of reduction of an available family arises. For the first time, this problem was considered by V.A. Roshchina. She proposed conditions for minimality and described some methods of reduction in the case when these conditions are not satisfied. However, it turned out that the exhauster mapping is not continuous in the Hausdorff metrics, which leads to the problems with convergence of numerical methods. To overcome this difficulty, Demyanov proposed the notion of coexhausters. These objects enable one to represent the main part of the increment of the considered function in a neighborhood of the studied point in the form of MaxMin or MinMax of affine functions. One can define a class of functions with the continuous coexhauster mapping. Optimality conditions can be stated in terms of these objects too. But coexhausters are also defined not uniquely. The problem of reduction of coexhausters is considered in this paper for the first time. Definitions of minimality proposed by Roshchina are used. In contrast to ideas proposed in the works of Roshchina, the minimality conditions and the technique of reduction developed in this paper have a clear and transparent geometric interpretation.     

Reducing Exhausters

The notions of upper and lower exhausters were introduced by Demyanov (Optimization 45:13–29, 1999). Upper and lower exhausters can be employed to study a very wide range of positively homogeneous functions, for example, various directional derivatives of nonsmooth functions. Exhausters are not uniquely defined; hence, the problem of minimality arises naturally. This paper describes some techniques for reducing exhausters, both in size and amount of sets. We define also a modified convertor which provides much more flexibility in converting upper exhausters to lower ones and vice versa, and allows us to obtain much smaller sets.     

Relationships between upper exhausters and the basic subdifferential in variational analysis

In this paper we establish a relationship between the basic subdifferential and upper exhausters of positively homogeneous and polyhedral functions. In the case of a finite exhauster this relationship is represented in a form of an equality, and in the case of a Lipschitz function an inclusion formula is obtained.     

Inhomogeneous convex approximations of nonsmooth functions

In this paper we introduce notions of inhomogeneous upper convex and lower concave approximations of an increment of a nonsmooth function defined on a normed space and study exhaustive families of these approximations. In terms of the introduced notions we establish optimality conditions for various constrained and unconstrained extremum problems.     

Quasidifferentiability of nonsmooth quasiconvex functions

The paper deals with nonsmooth quasiconvex functions and develops a quasidifferential analysis for this class of functions. Therefore, in terms of sub and superdifferentials, first order approximations of the functions are derived, optimality conditions are stated and directions of descent (either simple feasible or of steepest descent) are determined. Moreover, a relation among positively homogeneous convex and quasiconvex functions is established     

Finding the distance between ellipsoids

Under study is the problem of finding the nearest points from one ellipsoid to the other. Some new algorithms for solving this problem are constructed, using the theory of exact penalty functions and nonsmooth analysis. We propose two iterative methods of (steepest and hypodifferential) descent. The new algorithms (as compared with those previously known) have specific advantages; in particular, they are universal and less labor-consuming. Software for implementing these algorithms is developed inMATLAB and Maple.     

On Optimal Control of a Free Surface Flow

We discuss an optimal control approach for a 2D Stokes flow with a free surface. The aim is to optimize the shape of a polymer film by adjusting the ambient pressure in a casting process. The resulting minimization problem is solved by the method of steepest descent. Numerical results will be presented. Furthermore we state the adjoint system for the Lagrangian formalism. (© 2009 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim)     

Adjoint Based Optimization of a Free Surface Flow for Film Casting

Motivated by the industrial process of film casting, we discuss an optimal control approach for a viscous free surface flow that is asymptotically described by 2d Stokes equations. The aim is to optimize the shape of the free boundaries by adjusting the ambient pressure. A PDE-constrained optimization problem is set up, where the free surfaces are described as graphs. The first-order optimality system is derived via the Lagrangian formalism. The resulting minimization problem is solved by a steepest descent method, where the gradient is expressed in terms of the adjoint variables. Numerical results are presented. (© 2010 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim)     

Abstract convex approximations of nonsmooth functions

In the article we use abstract convexity theory in order to unify and generalize many different concepts of nonsmooth analysis. We introduce the concepts of abstract codifferentiability, abstract quasidifferentiability and abstract convex (concave) approximations of a nonsmooth function mapping a topological vector space to an order complete topological vector lattice. We study basic properties of these notions, construct elaborate calculus of abstract codifferentiable functions and discuss continuity of abstract codifferential. We demonstrate that many classical concepts of nonsmooth analysis, such as subdifferentiability and quasidifferentiability, are particular cases of the concepts of abstract codifferentiability and abstract quasidifferentiability. We also show that abstract convex and abstract concave approximations are a very convenient tool for the study of nonsmooth extremum problems. We use these approximations in order to obtain various necessary optimality conditions for nonsmooth nonconvex optimization problems with the abstract codifferentiable or abstract quasidifferentiable objective function and constraints. Then, we demonstrate how these conditions can be transformed into simpler and more constructive conditions in some particular cases.     

Nonsmooth Steepest Descent Method by Proximal Subdifferentials in Hilbert Spaces

In this paper, we first study a nonsmooth steepest descent method for nonsmooth functions defined on a Hilbert space and establish the corresponding algorithm by proximal subgradients. Then, we use this algorithm to find stationary points for those functions satisfying prox-regularity and Lipschitz continuity. As an application, the established algorithm is used to search for the minimizer of a lower semicontinuous and convex function on a finite-dimensional space. A convergence theorem, as an extension and improvement of the existing converging result for twice continuously differentiable convex functions, is also presented therein.     

On necessary and sufficient optimality conditions for multicriterial optimization problems

In this paper we derive first order necessary and sufficient optimality conditions for nonsmooth optimization problems with multiple criteria. These conditions are given for different optimality notions (i.e. weak, Pareto- and proper minimality) and for different types of derivatives of nonsmooth objective functions (locally Lipschitz continuous and quasidifferentiable) mappings. The conditions are given, if possible, in terms of a derivative and a subdifferential of those mappings.     

Analysis of Nonlinear Quadratic Control in Infinite Time

This study refers to minimization of quadratic functionals in infinite time. The coefficients of the quadratic form are quadratic matrices, function of the state variable. Dynamic constraints are represented by a bilinear differential systems of the form. The necessary extremum conditions determine the adjoint variables λ and the control variables u as functions of state variable, respectively the adjoint system corresponding to those functions. Thus it will be obtained a matrix differential equation where the solution representing the positive defined symmetric matrix P ( x ), verifies the Riccati algebraic equation. (© 2008 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim)     

On the control of time discretized dynamic contact problems

We consider optimal control problems with distributed control that involve a time-stepping formulation of dynamic one body contact problems as constraints. We link the continuous and the time-stepping formulation by a nonconforming finite element discretization and derive existence of optimal solutions and strong stationarity conditions. We use this information for a steepest descent type optimization scheme based on the resulting adjoint scheme and implement its numerical application.     

On the Local Convergence Analysis of the Gradient Sampling Method for Finite Max-Functions

The gradient sampling method is a recently developed tool for solving unconstrained nonsmooth optimization problems. Using just first-order information about the objective function, it generalizes the steepest descent method, one of the most classical methods for minimizing a smooth function. This study aims at determining under which circumstances one can expect the same local convergence result of the Cauchy method for the gradient sampling algorithm under the assumption that the problem is stated by a finite max-function around the optimal point. Additionally, at the end, we show how to practically accomplish the required hypotheses during the execution of the algorithm.