Proper and adjoint exhausters in nonsmooth analysis: optimality conditions

The notions of upper and lower exhausters represent generalizations of the notions of exhaustive families of upper convex and lower concave approximations (u.c.a., l.c.a.). The notions of u.c.a.’s and l.c.a.’s were introduced by Pshenichnyi (Convex Analysis and Extremal Problems, Series in Nonlinear Analysis and its Applications, 1980), while the notions of exhaustive families of u.c.a.’s and l.c.a.’s were described by Demyanov and Rubinov in Nonsmooth Problems of Optimization Theory and Control, Leningrad University Press, Leningrad, 1982. These notions allow one to solve the problem of optimization of an arbitrary function by means of Convex Analysis thus essentially extending the area of application of Convex Analysis. In terms of exhausters it is possible to describe extremality conditions, and it turns out that conditions for a minimum are expressed via an upper exhauster while conditions for a maximum are formulated in terms of a lower exhauster (Abbasov and Demyanov (2010), Demyanov and Roshchina (Appl Comput Math 4(2): 114–124, 2005), Demyanov and Roshchina (2007), Demyanov and Roshchina (Optimization 55(5–6): 525–540, 2006)). This is why an upper exhauster is called a proper exhauster for minimization problems while a lower exhauster is called a proper one for maximization problems. The results obtained provide a simple geometric interpretation and allow one to construct steepest descent and ascent directions. Until recently, the problem of expressing extremality conditions in terms of adjoint exhausters remained open. Demyanov and Roshchina (Appl Comput Math 4(2): 114–124, 2005), Demyanov and Roshchina (Optimization 55(5–6): 525–540, 2006) was the first to derive such conditions. However, using the conditions obtained (unlike the conditions expressed in terms of proper exhausters) it was not possible to find directions of descent and ascent. In Abbasov (2011) new extremality conditions in terms of adjoint exhausters were discovered. In the present paper, a different proof of these conditions is given and it is shown how to find steepest descent and ascent conditions in terms of adjoint exhausters. The results obtained open the way to constructing numerical methods based on the usage of adjoint exhausters thus avoiding the necessity of converting the adjoint exhauster into a proper one.     

Adjoint Coexhausters in Nonsmooth Analysis and Extremality Conditions

In the classical (“smooth”) mathematical analysis, a differentiable function is studied by means of the derivative (gradient in the multidimensional space). In the case of nondifferentiable functions, the tools of nonsmooth analysis are to be employed. In convex analysis and minimax theory, the corresponding classes of functions are investigated by means of the subdifferential (it is a convex set in the dual space), quasidifferentiable functions are treated via the notion of quasidifferential (which is a pair of sets). To study an arbitrary directionally differentiable function, the notions of upper and lower exhausters (each of them being a family of convex sets) are used. It turns out that conditions for a minimum are described by an upper exhauster, while conditions for a maximum are stated in terms of a lower exhauster. This is why an upper exhauster is called a proper one for the minimization problem (and an adjoint exhauster for the maximization problem) while a lower exhauster will be referred to as a proper one for the maximization problem (and an adjoint exhauster for the minimization problem). The directional derivatives (and hence, exhausters) provide first-order approximations of the increment of the function under study. These approximations are positively homogeneous as functions of direction. They allow one to formulate optimality conditions, to find steepest ascent and descent directions, to construct numerical methods. However, if, for example, the maximizer of the function is to be found, but one has an upper exhauster (which is not proper for the maximization problem), it is required to use a lower exhauster. Instead, one can try to express conditions for a maximum in terms of upper exhauster (which is an adjoint one for the maximization problem). The first to get such conditions was Roshchina. New optimality conditions in terms of adjoint exhausters were recently obtained by Abbasov. The exhauster mappings are, in general, discontinuous in the Hausdorff metric, therefore, computational problems arise. To overcome these difficulties, the notions of upper and lower coexhausters are used. They provide first-order approximations of the increment of the function which are not positively homogeneous any more. These approximations also allow one to formulate optimality conditions, to find ascent and descent directions (but not the steepest ones), to construct numerical methods possessing good convergence properties. Conditions for a minimum are described in terms of an upper coexhauster (which is, therefore, called a proper coexhauster for the minimization problem) while conditions for a maximum are described in terms of a lower coexhauster (which is called a proper one for the maximization problem). In the present paper, we derive optimality conditions in terms of adjoint coexhausters.     

Proper exhausters and coexhausters in nonsmooth analysis

There exist many tools to analyze nonsmooth functions. For convex and max-type functions, the notion of subdifferential is used, for quasidifferentiable functions – that of quasidifferential. By means of these tools, one is able to solve, e.g. the following problems: to get an approximation of the increment of a functional, to formulate conditions for an extremum, to find steepest descent and ascent directions and to construct numerical methods. For arbitrary directionally differentiable functions, these problems are solved by employing the notions of upper and lower exhausters and coexhausters, which are generalizations of such notions of nonsmooth analysis as sub- and superdifferentials, quasidifferentials and codifferentials. Exhausters allow one to construct homogeneous approximations of the increment of a functional while coexhausters provide nonhomogeneous approximations. It became possible to formulate conditions for an extremum in terms of exhausters and coexhausters. It turns out that conditions for a minimum are expressed by an upper exhauster, and conditions for a maximum are formulated via a lower one. This is why an upper exhauster is called a proper one for the minimization problem (and adjoint for the maximization problem) while a lower exhauster is called a proper one for the maximization problem (and adjoint for the minimization problem). The conditions obtained provide a simple geometric interpretation and allow one to find steepest descent and ascent directions. In this article, optimization problems are treated by means of proper exhausters and coexhausters.     

Extremum conditions for a nonsmooth function in terms of exhausters and coexhausters

The notions of upper and lower exhausters and coexhausters are discussed and necessary conditions for an unconstrained extremum of a nonsmooth function are derived. The necessary conditions for a minimum are formulated in terms of an upper exhauster (coexhauster) and the necessary conditions for a maximum are formulated in terms of a lower exhauster (coexhauster). This involves the problem of transforming an upper exhauster (coexhauster) into a lower exhauster (coexhauster) and vice versa. The transformation is carried out by means of a conversion operation (converter). Second-order approximations obtained with the help of second-order (upper and lower) coexhausters are considered. It is shown how a secondorder upper coexhauster can be converted into a lower coexhauster and vice versa. This problem is reduced to using a first-order conversion operator but in a space of a higher dimension. The obtained result allows one to construct second-order methods for the optimization of nonsmooth functions (Newton-type methods).     

Comparison Between Quasidifferentials and Exhausters

Directional derivatives play one of the major roles in optimization. Optimality conditions can be described in terms of these objects. These conditions, however, are not constructive. To overcome this problem, one has to represent the directional derivative in special forms. Two such forms are quasidifferentials and exhausters proposed by V.F. Demyanov. Quasidifferentials were introduced in 1980s. Optimality conditions in terms of these objects were developed by L.N. Polyakova and V.F. Demyanov. It was described how to find directions of steepest descent and ascent when these conditions are not satisfied. This paved a way for constructing new optimization algorithms. Quasidifferentials allow one to treat a wide class of functions. V.F. Demyanov introduced the notion of exhausters in 2000s to expand the class of functions that can be treated. It should be noted that a great contribution to the emergence of this notion was made by B.N. Pshenichny and A.M. Rubinov. In this work it is shown that exhausters not only allow one to treat a wider class of functions than quasidifferentials (since every quasidifferentiable function has exhausters) but is also preferable even for quasidifferentiable functions when solving nonsmooth optimization problems.     

The tricriterion shortest path problem with at least two bottleneck objective functions

The focus of this paper is on the tricriterion shortest path problem where two objective functions are of the bottleneck type, for example MinMax or MaxMin. The third objective function may be of the same kind or we may consider, for example, MinSum or MaxProd. Let p(n) be the complexity of a classical single objective algorithm responsible for this third function, where n is the number of nodes and m be the number of arcs of the graph. An O(m²p(n)) algorithm is presented that can generate the minimal complete set of Pareto-optimal solutions. Finding the maximal complete set is also possible. Optimality proofs are given and extensions for several special cases are presented. Computational experience for a set of randomly generated problems is reported.     

Reducing Exhausters

The notions of upper and lower exhausters were introduced by Demyanov (Optimization 45:13–29, 1999). Upper and lower exhausters can be employed to study a very wide range of positively homogeneous functions, for example, various directional derivatives of nonsmooth functions. Exhausters are not uniquely defined; hence, the problem of minimality arises naturally. This paper describes some techniques for reducing exhausters, both in size and amount of sets. We define also a modified convertor which provides much more flexibility in converting upper exhausters to lower ones and vice versa, and allows us to obtain much smaller sets.     

Exact calculus of Fréchet subdifferentials for Hadamard directionally differentiable functions

We continue the study of the calculus of the generalized subdifferentials started in [V.F. Demyanov, V. Roshchina, Exhausters and subdifferentials in nonsmooth analysis, Optimization (2006) (in press)] and [V. Roshchina, Relationships between upper exhausters and the basic subdifferential in Variational Analysis, Journal of Mathematical Analysis and Applications 334 (2007) 261–272] and provide some basic calculus rules for the Fréchet subdifferentials via collections of compact convex sets associated with Hadamard directional derivative. The main result of this paper is the sum rule for the Fréchet subdifferential in the form of an equality, which holds for Hadamard directionally differentiable functions, and is of significant interest from the points of view of both theory and applications.     

Exhausters af a positively homogeneous function *

Notions of upper exhauster and lower exhauster of a positively homogeneous (of the first degree) function h: ? ⁿ →? are introduced. They are linked to exhaustive families of upper convex and lower concave approximations of the function h. The pair of an upper exhauster and a lower exhauster is called a biexhauster of h. A calculus for biexhausters is described (in particular, a composition theorem is formulated). The problem of minimality of exhausters is stated. Necessary and sufficient conditions for a constrained minimum and a constrained maximum of a directionally differentiable function f: ? ⁿ →? are formulated in terms of exhausters of the directional derivative of f. In general, they are described by means of exhausters of the Hadamard upper and lower directional derivatives of the function f. To formulate conditions for a minimum, an upper exhauster is employed while conditions for a maximum are formulated via a lower exhauster of the respective directional derivative (the Hadamard lower derivative for a minimum and the Hadamard upper derivative for a maximum).

If a point x _o is not stationary then directions of steepest ascent and descent can also be calculated by means of exhausters.     

Reduction of finite exhausters

In this paper we introduce the notation of shadowing sets which is a generalization of the notion of separating sets to the family of more than two sets. We prove that \({\bigcap_{i\in I}A_{i}}\) is a shadowing set of the family \({\{A_{i}\}_{i\in I}}\) if and only if \({\sum_{i\in I}A_{i}=\bigvee_{i\in I}\sum_{k\in I\setminus \{i\}}A_{i} + \bigcap_{i\in I}A_{i}}\). It generalizes the theorem stating that \({A\cap B}\) is separating set for A and B if and only if \({A+B=A\cap B+A\vee B}\). In terms of shadowing sets, we give a criterion for an arbitrary upper exhauster to be an exhauster of sublinear function and a criterion for the minimality of finite upper exhausters. Finally we give an example of two different minimal upper exhausters of the same function, which answers a question posed by Vera Roshchina (J Convex Anal, to appear).     

Representative of Quasidifferentials and Its Formula for a Quasidifferentiable Function

The quasidifferential of a quasidifferentiable function in the sense of Demyanov and Rubinov is not uniquely defined. Xia proposed the notion of the kernelled quasidifferential, which is expected to be a representative for the equivalent class of quasidifferentials. In the 2-dimensional case, the existence of the kernelled quasidifferential was shown. In this paper, the existence of the kernelled quasidifferential in the n-dimensional space (n>2) is proved under the assumption that the Minkowski difference and the Demyanov difference of subdifferential and minus superdifferential coincide. In particular, given a quasidifferential, the kernelled quasidifferential can be formulated. Applications to two classes of generalized separable quasidifferentiable functions are developed. Mathematics Subject Classifications (2000) 49J52, 54C60, 90C26. This work was supported by Shanghai Education Committee (04EA01).     

Demyanov Difference of Two Sets and Optimality Conditions of Lagrange Multiplier Type for Constrained Quasidifferentiable Optimization

In the first part of this paper, the Demyanov difference of two sets is considered. An expression for the Demyanov difference of two sets, which are the convex hulls of a finite number of points, is presented. In the second part, first-order necessary optimality conditions of the Lagrange multiplier type, for quasidifferentiable optimization with equality and inequality constraints, are given by means of the Demyanov difference of subdifferential and negative superdifferential.     

On Optimal Defuzzification and Learning Algorithms: Theory and Applications

In this paper, the issue of optimal defuzzification which is advocated in the Optimality Principle of Defuzzification (Song and Leland (1996)) is addressed. It was shown that defuzzification can be treated as a mapping from a high dimensional space to the real line. When system performance indices are considered, the defuzzification mapping which optimizes the performance indices for the given fuzzy sets is known as the optimal defuzzification mapping. Thus, finding this optimal defuzzification mapping becomes the essence of defuzzification. The problem with this idea, however, is that the space formed by all possible continuous defuzzification mappings is so large to search that the only recourse is an approximation to the optimal defuzzification mapping. With this, learning algorithms can be devised to find the optimal parameters of defuzzifiers with fixed structures. The proposed method is rigorously examined and compared with some well-known defuzzification methods. To overcome the resultant enormous computational load problem with this algorithm, the concept of defuzzification filter is additionally proposed. An application of the method to the power system stabilization problem is presented.     

Extremal problems on the set of nonnegative definite matrices

An optimization problem of minimizing a real-valued function of certain elements of a symmetric matrix subject to this matrix being nonnegative definite is considered. Optimality conditions are proposed. The duality result of Olkin and Pukelsheim (1982) is extended to a wide class of such problems. Applications are discussed.     

Arcwise Connected Cone-Convex Functions and Mathematical Programming

The concept of arcwise connected cone-convex functions in topological vector spaces is introduced. Optimality conditions and duality theorems for a vector-valued nonlinear programming problem involving arcwise connected cone-convex functions are discussed.     

Algorithms for optimal area triangulations of a convex polygon

Given a convex polygon with n vertices in the plane, we are interested in triangulations of its interior, i.e., maximal sets of non-intersecting diagonals that subdivide the interior of the polygon into triangles. The MaxMin area triangulation is the triangulation of the polygon that maximizes the area of the smallest triangle in the triangulation. Similarly, the MinMax area triangulation is the triangulation that minimizes the area of the largest area triangle in the triangulation. We present algorithms that construct MaxMin and MinMax area triangulations of a convex polygon in O(n²logn) time and O(n²) space. The algorithms use dynamic programming and a number of geometric properties that are established within the paper.     

Max-min resource allocation

A problem is considered of the allocation of resources so as to maximize the minimum return from several activities. Optimality conditions are given for the case of a single resource, and are used to derive a solution algorithm. Problems with several resources cannot be solved by resourcewise optimization. Concave return functions are treated approximately by linear programming, and optimality or almost optimality of any feasible solution to such a problem can be evaluated by the solution of a linear programming problem. The evaluation measure is extended to certain feasible solutions of problems which have continuous, but not necessarily concave, return functions. A numerical example is given.     

Proof of covering minimality by generalizing the notion of independence

A method is proposed for obtaining lower bounds for the length of the shortest cover and complexity of the minimal cover based on the notion of independent family of sets. For the problem of minimization of Boolean functions, we provide the functions and construct coverings by faces of the set of unit vertices for which the suggested lower bounds can be achieved in the case of five or more variables. The lower bounds, based on independent sets, are unreachable and cannot be used as sufficient conditions for minimality of such functions.     

Bounded knapsack sharing

A bounded knapsack sharing problem is a maximin or minimax mathematical programming problem with one or more linear inequality constraints, an objective function composed of single variable continuous functions called tradeoff functions, and lower and upper bounds on the variables. A single constraint problem which can have negative or positive constraint coefficients and any type of continuous tradeoff functions (including multi-modal, multiple-valued and staircase functions) is considered first. Limiting conditions where the optimal value of a variable may be plus or minus infinity are explicitly considered. A preprocessor procedure to transform any single constraint problem to a finite form problem (an optimal feasible solution exists with finite variable values) is developed. Optimality conditions and three algorithms are then developed for the finite form problem. For piecewise linear tradeoff functions, the preprocessor and algorithms are polynomially bounded. The preprocessor is then modified to handle bounded knapsack sharing problems with multiple constraints. An optimality condition and algorithm is developed for the multiple constraint finite form problem. For multiple constraints, the time needed for the multiple constraint finite form algorithm is the time needed to solve a single constraint finite form problem multiplied by the number of constraints. Some multiple constraint problems cannot be transformed to multiple constraint finite form problems.     

A metaheuristic approach to the dominating tree problem

This paper considers a recently introduced NP-hard problem on graphs, called the dominating tree problem. In order to solve this problem, we develop a variable neighborhood search (VNS) based heuristic. Feasible solutions are obtained by using the set of vertex permutations that allow us to implement standard neighborhood structures and the appropriate local search procedure. Computational experiments include two classes of randomly generated test instances and benchmark test instances from the literature. Optimality of VNS solutions on small size instances is verified with CPLEX.