Reduction and Minimality of Coexhausters

V.F. Demyanov introduced exhausters for the study of nonsmooth functions. These are families of convex compact sets that enable one to represent the main part of the increment of a considered function in a neighborhood of the studied point as MaxMin or MinMax of linear functions. Optimality conditions were described in terms of these objects. This provided a way for constructing new algorithms for solving nondifferentiable optimization problems. Exhausters are defined not uniquely. It is obvious that the smaller an exhauster, the less are the computational expenses when working with it. Thus, the problem of reduction of an available family arises. For the first time, this problem was considered by V.A. Roshchina. She proposed conditions for minimality and described some methods of reduction in the case when these conditions are not satisfied. However, it turned out that the exhauster mapping is not continuous in the Hausdorff metrics, which leads to the problems with convergence of numerical methods. To overcome this difficulty, Demyanov proposed the notion of coexhausters. These objects enable one to represent the main part of the increment of the considered function in a neighborhood of the studied point in the form of MaxMin or MinMax of affine functions. One can define a class of functions with the continuous coexhauster mapping. Optimality conditions can be stated in terms of these objects too. But coexhausters are also defined not uniquely. The problem of reduction of coexhausters is considered in this paper for the first time. Definitions of minimality proposed by Roshchina are used. In contrast to ideas proposed in the works of Roshchina, the minimality conditions and the technique of reduction developed in this paper have a clear and transparent geometric interpretation.     

Proper exhausters and coexhausters in nonsmooth analysis

There exist many tools to analyze nonsmooth functions. For convex and max-type functions, the notion of subdifferential is used, for quasidifferentiable functions – that of quasidifferential. By means of these tools, one is able to solve, e.g. the following problems: to get an approximation of the increment of a functional, to formulate conditions for an extremum, to find steepest descent and ascent directions and to construct numerical methods. For arbitrary directionally differentiable functions, these problems are solved by employing the notions of upper and lower exhausters and coexhausters, which are generalizations of such notions of nonsmooth analysis as sub- and superdifferentials, quasidifferentials and codifferentials. Exhausters allow one to construct homogeneous approximations of the increment of a functional while coexhausters provide nonhomogeneous approximations. It became possible to formulate conditions for an extremum in terms of exhausters and coexhausters. It turns out that conditions for a minimum are expressed by an upper exhauster, and conditions for a maximum are formulated via a lower one. This is why an upper exhauster is called a proper one for the minimization problem (and adjoint for the maximization problem) while a lower exhauster is called a proper one for the maximization problem (and adjoint for the minimization problem). The conditions obtained provide a simple geometric interpretation and allow one to find steepest descent and ascent directions. In this article, optimization problems are treated by means of proper exhausters and coexhausters.     

Proper and adjoint exhausters in nonsmooth analysis: optimality conditions

The notions of upper and lower exhausters represent generalizations of the notions of exhaustive families of upper convex and lower concave approximations (u.c.a., l.c.a.). The notions of u.c.a.’s and l.c.a.’s were introduced by Pshenichnyi (Convex Analysis and Extremal Problems, Series in Nonlinear Analysis and its Applications, 1980), while the notions of exhaustive families of u.c.a.’s and l.c.a.’s were described by Demyanov and Rubinov in Nonsmooth Problems of Optimization Theory and Control, Leningrad University Press, Leningrad, 1982. These notions allow one to solve the problem of optimization of an arbitrary function by means of Convex Analysis thus essentially extending the area of application of Convex Analysis. In terms of exhausters it is possible to describe extremality conditions, and it turns out that conditions for a minimum are expressed via an upper exhauster while conditions for a maximum are formulated in terms of a lower exhauster (Abbasov and Demyanov (2010), Demyanov and Roshchina (Appl Comput Math 4(2): 114–124, 2005), Demyanov and Roshchina (2007), Demyanov and Roshchina (Optimization 55(5–6): 525–540, 2006)). This is why an upper exhauster is called a proper exhauster for minimization problems while a lower exhauster is called a proper one for maximization problems. The results obtained provide a simple geometric interpretation and allow one to construct steepest descent and ascent directions. Until recently, the problem of expressing extremality conditions in terms of adjoint exhausters remained open. Demyanov and Roshchina (Appl Comput Math 4(2): 114–124, 2005), Demyanov and Roshchina (Optimization 55(5–6): 525–540, 2006) was the first to derive such conditions. However, using the conditions obtained (unlike the conditions expressed in terms of proper exhausters) it was not possible to find directions of descent and ascent. In Abbasov (2011) new extremality conditions in terms of adjoint exhausters were discovered. In the present paper, a different proof of these conditions is given and it is shown how to find steepest descent and ascent conditions in terms of adjoint exhausters. The results obtained open the way to constructing numerical methods based on the usage of adjoint exhausters thus avoiding the necessity of converting the adjoint exhauster into a proper one.     

Representative of Quasidifferentials and Its Formula for a Quasidifferentiable Function

The quasidifferential of a quasidifferentiable function in the sense of Demyanov and Rubinov is not uniquely defined. Xia proposed the notion of the kernelled quasidifferential, which is expected to be a representative for the equivalent class of quasidifferentials. In the 2-dimensional case, the existence of the kernelled quasidifferential was shown. In this paper, the existence of the kernelled quasidifferential in the n-dimensional space (n>2) is proved under the assumption that the Minkowski difference and the Demyanov difference of subdifferential and minus superdifferential coincide. In particular, given a quasidifferential, the kernelled quasidifferential can be formulated. Applications to two classes of generalized separable quasidifferentiable functions are developed. Mathematics Subject Classifications (2000) 49J52, 54C60, 90C26. This work was supported by Shanghai Education Committee (04EA01).     

具有等式与不等式约束条件拟可微优化的Lagrange乘子型最优性条件

讨论了不等式约束优化问题中拟微分形式下Ｆｒｉｔｚ Ｊｏｈｎ必要条件与 Ｃｌａｒｋｅ广义梯度形式下Ｆｒｉｔｚ Ｊｏｈｎ必要条件的关系．在较弱条件下给出了具有等式与不等式约束条件的两个Ｌａｇｒａｎｇｅ乘子形式的最优性必要条件,在这两个条件中等式约束函数的拟微分和Ｃｌａｒｋｅ广义梯度分别被使用。     

Exhausters, optimality conditions and related problems

The notions of exhausters were introduced in (Demyanov, Exhauster of a positively homogeneous function, Optimization 45, 13–29 (1999)). These dual tools (upper and lower exhausters) can be employed to describe optimality conditions and to find directions of steepest ascent and descent for a very wide range of nonsmooth functions. What is also important, exhausters enjoy a very good calculus (in the form of equalities). In the present paper we review the constrained and unconstrained optimality conditions in terms of exhausters, introduce necessary and sufficient conditions for the Lipschitzivity and Quasidifferentiability, and also present some new results on relationships between exhausters and other nonsmooth tools (such as the Clarke, Michel-Penot and Fréchet subdifferentials).     

Adjoint Coexhausters in Nonsmooth Analysis and Extremality Conditions

In the classical (“smooth”) mathematical analysis, a differentiable function is studied by means of the derivative (gradient in the multidimensional space). In the case of nondifferentiable functions, the tools of nonsmooth analysis are to be employed. In convex analysis and minimax theory, the corresponding classes of functions are investigated by means of the subdifferential (it is a convex set in the dual space), quasidifferentiable functions are treated via the notion of quasidifferential (which is a pair of sets). To study an arbitrary directionally differentiable function, the notions of upper and lower exhausters (each of them being a family of convex sets) are used. It turns out that conditions for a minimum are described by an upper exhauster, while conditions for a maximum are stated in terms of a lower exhauster. This is why an upper exhauster is called a proper one for the minimization problem (and an adjoint exhauster for the maximization problem) while a lower exhauster will be referred to as a proper one for the maximization problem (and an adjoint exhauster for the minimization problem). The directional derivatives (and hence, exhausters) provide first-order approximations of the increment of the function under study. These approximations are positively homogeneous as functions of direction. They allow one to formulate optimality conditions, to find steepest ascent and descent directions, to construct numerical methods. However, if, for example, the maximizer of the function is to be found, but one has an upper exhauster (which is not proper for the maximization problem), it is required to use a lower exhauster. Instead, one can try to express conditions for a maximum in terms of upper exhauster (which is an adjoint one for the maximization problem). The first to get such conditions was Roshchina. New optimality conditions in terms of adjoint exhausters were recently obtained by Abbasov. The exhauster mappings are, in general, discontinuous in the Hausdorff metric, therefore, computational problems arise. To overcome these difficulties, the notions of upper and lower coexhausters are used. They provide first-order approximations of the increment of the function which are not positively homogeneous any more. These approximations also allow one to formulate optimality conditions, to find ascent and descent directions (but not the steepest ones), to construct numerical methods possessing good convergence properties. Conditions for a minimum are described in terms of an upper coexhauster (which is, therefore, called a proper coexhauster for the minimization problem) while conditions for a maximum are described in terms of a lower coexhauster (which is called a proper one for the maximization problem). In the present paper, we derive optimality conditions in terms of adjoint coexhausters.     

Weak subdifferential/superdifferential,weak exhausters and optimality conditions

In this work, the notion of weak superdifferential is presented. Some calculation rules are given to evaluate weak subdifferential and weak superdifferential of some classes of functions represented by support functions. Moreover, some methods are obtained to calculate weak subdifferential of convex functions. In addition, the concept of weak lower and weak upper exhausters of positively homogeneous functions are introduced by using weak subdifferential and weak superdifferential, respectively. In terms of weak exhausters, some optimality conditions are given to find local or global minimizers/maximizers of some classes of functions.     

Higher order optimality conditions with an arbitrary non-differentiable function

In this paper, we introduce a higher order directional derivative and higher order subdifferential of Hadamard type of a given proper extended real function. We obtain necessary and sufficient optimality conditions of order n (n is a positive integer) for unconstrained problems in terms of them. We do not require any restrictions on the function in our results. In contrast to the most known directional derivatives, our derivative is harmonized with the classical higher order Fréchet directional derivative of the same order in the sense that both of them coincide, provided that the last one exists. A notion of a higher order critical direction is introduced. It is applied in the characterizations of the isolated local minimum of order n. Higher order invex functions are defined. They are the largest class such that the necessary conditions for a local minimum are sufficient for global one. We compare our results with some previous ones. As an application, we improve a result due to V. F. Demyanov, showing that the condition introduced by this author is a complete characterization of isolated local minimizers of order n.     

Reducing Exhausters

The notions of upper and lower exhausters were introduced by Demyanov (Optimization 45:13–29, 1999). Upper and lower exhausters can be employed to study a very wide range of positively homogeneous functions, for example, various directional derivatives of nonsmooth functions. Exhausters are not uniquely defined; hence, the problem of minimality arises naturally. This paper describes some techniques for reducing exhausters, both in size and amount of sets. We define also a modified convertor which provides much more flexibility in converting upper exhausters to lower ones and vice versa, and allows us to obtain much smaller sets.     

Exact calculus of Fréchet subdifferentials for Hadamard directionally differentiable functions

We continue the study of the calculus of the generalized subdifferentials started in [V.F. Demyanov, V. Roshchina, Exhausters and subdifferentials in nonsmooth analysis, Optimization (2006) (in press)] and [V. Roshchina, Relationships between upper exhausters and the basic subdifferential in Variational Analysis, Journal of Mathematical Analysis and Applications 334 (2007) 261–272] and provide some basic calculus rules for the Fréchet subdifferentials via collections of compact convex sets associated with Hadamard directional derivative. The main result of this paper is the sum rule for the Fréchet subdifferential in the form of an equality, which holds for Hadamard directionally differentiable functions, and is of significant interest from the points of view of both theory and applications.     

Convex Difference Criteria for the Quantitative Stability of Parametric Quasidifferentiable Systems

In this paper solvability and Lipschitzian stability properties for a special class of nonsmooth parametric generalized systems defined in Banach are studied via a variational analysis approach. Verifiable sufficient conditions for such properties to hold under scalar quasidifferentiability assumptions are formulated by combining *-difference and Demyanov difference of convex compact subsets of the dual space with classic quasidifferential calculus constructions. Applications to the formulation of sufficient conditions for metric regularity/open covering of nonsmooth maps, along with their employment in deriving optimality conditions for quasidifferentiable extremum problems, as well as an application to the study of semicontinuity of the optimal value function in parametric optimization are discussed. In memory of Aleksandr Moiseevich Rubinov (1940–2006).     

On second-order directional derivatives of value functions

Directional derivatives of value functions play an essential role in the sensitivity and stability analysis of parametric optimization problems, in studying bi-level and min–max problems, in quasi-differentiable calculus. Their calculation is studied in numerous works by A.V. Fiacco, V.F. Demyanov and A.M. Rubinov, R.T. Rockafellar, A. Shapiro, J.F. Bonnans, A.D. Ioffe, A. Auslender and R. Cominetti, and many other authors. This article is devoted to the existence of the second order directional derivatives of value functions in parametric problems with non-single-valued solutions. The main idea of the investigation approach is based on the development of the method of the first-order approximations by V.F. Demyanov and A.M. Rubinov.     

Multidirectional Mean Value Inequalities in Quasidifferential Calculus

In the current paper, a Clarke–Ledyaev type mean value inequality is proved for semicontinuous functions defined in a Banach space that are quasidifferentiable in the sense of Demyanov–Rubinov. A stronger variant valid under compactness assumption in separable spaces and extensions for functions with semicontinuous Dini derivatives in locally uniformly convex Banach spaces and with merely bounded Dini derivatives are then established. Subsequently, applications of these mean value inequalities to solvability of nonsmooth parametric equations and to the estimation of local and global Hoffman error bound for inequalities are investigated via a decrease principle.     

Some relationships among quasidifferential,weak subdifferential and exhausters

In this study, some relationships between quasidifferentiability and weakly subdifferentiability of a function are investigated, and some results on calculating the quasidifferential in terms of weak exhausters are presented. Moreover, under some assumptions weak subdifferentiability of a quasidifferentiable function is proved and the weak subdifferential is obtained by using the quasidifferential. Under some assumptions, it is proved that a positively homogeneous and lower semicontinuous function is quasidifferentiable. It is also shown that directional differentiability and weak subdifferentiability imply quasidifferentiability. Furthermore, a method to evaluate the quasidifferential is given by using reduced weak lower exhausters.     

Quasi-Multiplier Rules for Quasidifferentiable Extremum Problems

The purpose of the present article is to contribute to clarify the role of the Lagrange multipliers within the theory of the first order necessary optimality conditions for nonsmooth constrained optimization, when the directional derivatives of functions involved in the extremum problems are not sublinear. This task is accomplished in the particular case of quasidifferentiable problems with side constraints. In such setting, making use of the image-space approach, it is possible to establish a generalized (nonlinear) separation result by means of which a new Lagrange principle is obtained. According to this principle, which seems to fit better quasidifferentiable extremum problems than the classic one, the concept of linear multiplier is to be replaced with that of quasi-multiplier, a sublinear and continuous functional whose existence can be guaranteed under mild assumptions, even when classic multipliers fail to exist. Such as extension allows to formulate in terms of Lagrange function the known optimality necessary condition for unconstrained quasidifferentiable optimization expressed in form of quasidifferential inclusion. Along with this, other multiplier rules are established.     

Fenchel-Lagrange Duality Versus Geometric Duality in Convex Optimization

We present a new duality theory to treat convex optimization problems and we prove that the geometric duality used by Scott and Jefferson in different papers during the last quarter of century is a special case of it. Moreover, weaker sufficient conditions to achieve strong duality are considered and optimality conditions are derived. Next, we apply our approach to some problems considered by Scott and Jefferson, determining their duals. We give weaker sufficient conditions to achieve strong duality and the corresponding optimality conditions. Finally, posynomial geometric programming is viewed also as a particular case of the duality approach that we present. Communicated by V. F. Demyanov The first author was supported in part by Gottlieb Daimler and Karl Benz Stiftung 02-48/99. The second author was supported in part by Karl und Ruth Mayer Stiftung.     

A Solution Method for a Special Class of Nondifferentiable Unconstrained Optimization Problems

We consider quasidifferentiable functions in the sense of Demyanov and Rubinov, i. e. functions, which are directionally differentiable and whose directional derivative can be expressed as a difference of two sublinear functions, so that its subdifferential, called the quasidifferential, consists of a pair of sets. For these functions a generalized gradient algorithm is proposed. Its behaviour is studied in detail for the special class of continuously subdifferentiable functions. Numerical test results are given. Finally, the general quasidifferentiable case is simulated by means of perturbed subdifferentials, where we make use of the non-uniqueness in the quasidifferential representation.     

Representation of the Clarke subdifferential for a regular quasidifferentiable function

A class of Lipschitz quasidifferentiable functions is described for which the exact representation of the Clarke subdifferential in terms of a quasidifferential holds. The sufficient conditions formulated are different from those previously established by Rubinov and Akhundov.     

An algorithm for the estimation of a regression function by continuous piecewise linear functions

The problem of the estimation of a regression function by continuous piecewise linear functions is formulated as a nonconvex, nonsmooth optimization problem. Estimates are defined by minimization of the empirical L ₂ risk over a class of functions, which are defined as maxima of minima of linear functions. An algorithm for finding continuous piecewise linear functions is presented. We observe that the objective function in the optimization problem is semismooth, quasidifferentiable and piecewise partially separable. The use of these properties allow us to design an efficient algorithm for approximation of subgradients of the objective function and to apply the discrete gradient method for its minimization. We present computational results with some simulated data and compare the new estimator with a number of existing ones.