Forward-Backward and Tseng’s Type Penalty Schemes for Monotone Inclusion Problems

We deal with monotone inclusion problems of the form 0 ∈ A x + D x + N _C (x) in real Hilbert spaces, where A is a maximally monotone operator, D a cocoercive operator and C the nonempty set of zeros of another cocoercive operator. We propose a forward-backward penalty algorithm for solving this problem which extends the one proposed by Attouch et al. (SIAM J. Optim. 21(4): 1251-1274, 2011). The condition which guarantees the weak ergodic convergence of the sequence of iterates generated by the proposed scheme is formulated by means of the Fitzpatrick function associated to the maximally monotone operator that describes the set C. In the second part we introduce a forward-backward-forward algorithm for monotone inclusion problems having the same structure, but this time by replacing the cocoercivity hypotheses with Lipschitz continuity conditions. The latter penalty type algorithm opens the gate to handle monotone inclusion problems with more complicated structures, for instance, involving compositions of maximally monotone operators with linear continuous ones.     

Solving integer programs over monotone inequalities in three variables: A framework for half integrality and good approximations

We define a class of monotone integer programs with constraints that involve up to three variables each. A generic constraint in such integer program is of the form ax−by⩽z+c, where a and b are nonnegative and the variable z appears only in that constraint. We devise an algorithm solving such problems in time polynomial in the length of the input and the range of variables U. The solution is obtained from a minimum cut on a graph with O(nU) nodes and O(mU) arcs where n is the number of variables of the types x and y and m is the number of constraints. Our algorithm is also valid for nonlinear objective functions.Nonmonotone integer programs are optimization problems with constraints of the type ax+by⩽z+c without restriction on the signs of a and b. Such problems are in general NP-hard. We devise here an algorithm, relying on a transformation to the monotone case, that delivers half integral superoptimal solutions in polynomial time. Such solutions provide bounds on the optimum value that can only be superior to bounds provided by linear programming relaxation. When the half integral solution can be rounded to an integer feasible solution, this is a 2-approximate solution. In that the technique is a unified 2-approximation technique for a large class of problems. The results apply also for general integer programming problems with worse approximation factors that depend on a quantifier measuring how far the problem is from the class of problems we describe.The algorithm described here has a wide array of problem applications. An additional important consequence of our results is that nonmonotone problems in the framework are MAX SNP-hard and at least as hard to approximate as vertex cover.Problems that are amenable to the analysis provided here are easily recognized. The analysis itself is entirely technical and involves manipulating the constraints and transforming them to a totally unimodular system while losing no more than a factor of 2 in the integrality.     

A distributed Douglas-Rachford splitting method for multi-block convex minimization problems

A customized Douglas-Rachford splitting method (DRSM) was recently proposed to solve two-block separable convex optimization problems with linear constraints and simple abstract constraints. The algorithm has advantage over the well-known alternating direction method of multipliers (ADMM), the dual application of DRSM to the two-block convex minimization problem, in the sense that the subproblems can have larger opportunity of possessing closed-form solutions since they are unconstrained. In this paper, we further study along this way by considering the primal application of DRSM for the general case m≥3, i.e., we consider the multi-block separable convex minimization problem with linear constraints where the objective function is separable into m individual convex functions without coupled variables. The resulting method fully exploits the separable structure and enjoys decoupled subproblems which can be solved simultaneously. Both the exact and inexact versions of the new method are presented in a unified framework. Under mild conditions, we manage to prove the global convergence of the algorithm. Preliminary numerical experiments for extracting the background from corrupted surveillance video verify the encouraging efficiency of the new algorithm.     

Decomposable norm minimization with proximal-gradient homotopy algorithm

We study the convergence rate of the proximal-gradient homotopy algorithm applied to norm-regularized linear least squares problems, for a general class of norms. The homotopy algorithm reduces the regularization parameter in a series of steps, and uses a proximal-gradient algorithm to solve the problem at each step. Proximal-gradient algorithm has a linear rate of convergence given that the objective function is strongly convex, and the gradient of the smooth component of the objective function is Lipschitz continuous. In many applications, the objective function in this type of problem is not strongly convex, especially when the problem is high-dimensional and regularizers are chosen that induce sparsity or low-dimensionality. We show that if the linear sampling matrix satisfies certain assumptions and the regularizing norm is decomposable, proximal-gradient homotopy algorithm converges with a linear rate even though the objective function is not strongly convex. Our result generalizes results on the linear convergence of homotopy algorithm for \(\ell _1\)-regularized least squares problems. Numerical experiments are presented that support the theoretical convergence rate analysis.     

Optimal scheduling on parallel machines for a new order class

This paper addresses the problem of scheduling n unit length tasks on m identical machines under certain precedence constraints. The aim is to compute minimal length nonpreemptive schedules. We introduce a new order class which contains properly two rich families of precedence graphs: interval orders and a subclass of the class of series parallel orders. We present a linear time algorithm to find an optimal schedule for this new order class on any number of machines.     

General Class of Implicit Variational Inclusions and Graph Convergence on <Emphasis Type="Italic">A</Emphasis>-Maximal Relaxed Monotonicity

Based on the generalized graph convergence, first a general framework for an implicit algorithm involving a sequence of generalized resolvents (or generalized resolvent operators) of set-valued A-maximal monotone (also referred to as A-maximal (m)-relaxed monotone, and A-monotone) mappings, and H-maximal monotone mappings is developed, and then the convergence analysis to the context of solving a general class of nonlinear implicit variational inclusion problems in a Hilbert space setting is examined. The obtained results generalize the work of Huang, Fang and Cho (in J. Nonlinear Convex Anal. 4:301–308, 2003) involving the classical resolvents to the case of the generalized resolvents based on A-maximal monotone (and H-maximal monotone) mappings, while the work of Huang, Fang and Cho (in J. Nonlinear Convex Anal. 4:301–308, 2003) added a new dimension to the classical resolvent technique based on the graph convergence introduced by Attouch (in Variational Convergence for Functions and Operators, Applied Mathematics Series, Pitman, London 1984). In general, the notion of the graph convergence has potential applications to several other fields, including models of phenomena with rapidly oscillating states as well as to probability theory, especially to the convergence of distribution functions on ℜ. The obtained results not only generalize the existing results in literature, but also provide a certain new approach to proofs in the sense that our approach starts in a standard manner and then differs significantly to achieving a linear convergence in a smooth manner.     

General implicit variational inclusion problems based on A-maximal (m)-relaxed monotonicity (AMRM) frameworks

A general framework for an algorithmic procedure based on the variational convergence of operator sequences involving A-maximal (m)-relaxed monotone (AMRM) mappings in a Hilbert space setting is developed, and then it is applied to approximating the solution of a general class of nonlinear implicit inclusion problems involving A-maximal (m)-relaxed monotone mappings. Furthermore, some specializations of interest on existence theorems and corresponding approximation solvability theorems on H-maximal monotone mappings are included that may include several other results for general variational inclusion problems on general maximal monotonicity in the literature.     

An O(n) algorithm for discrete n-point convex approximation with applications to continuous case

Given a bounded real function ? defined on a closed bounded real interval I, the problem is to find a convex function g so as to minimize the supremum of $\text{¦f(t) ? g(t)¦}$ for all t in I, over the class of all convex functions on I. The usual approach is to consider a discrete version of the problem on a grid of (n + 1) points in I, apply a conventional linear program to obtain an optimal solution, and let the grid size go to zero. This paper presents an alternative algorithm of complexity O(n), which is based on the concept of the greatest convex minorant of a function, for computation of a special “maximal” optimal solution to the discrete problem. It establishes the rate of convergence of this optimal solution to a solution of the original problem as the grid size goes to zero. It presents an alternative efficient linear program that generates the maximal optimal solution to the discrete problem. It also gives an O(n) algorithm for the discrete n-point monotone approximation problem.     

Decomposition Techniques for Bilinear Saddle Point Problems and Variational Inequalities with Affine Monotone Operators

The majority of first-order methods for large-scale convex–concave saddle point problems and variational inequalities with monotone operators are proximal algorithms. To make such an algorithm practical, the problem’s domain should be proximal-friendly—admit a strongly convex function with easy to minimize linear perturbations. As a by-product, this domain admits a computationally cheap linear minimization oracle (LMO) capable to minimize linear forms. There are, however, important situations where a cheap LMO indeed is available, but the problem domain is not proximal-friendly, which motivates search for algorithms based solely on LMO. For smooth convex minimization, there exists a classical algorithm using LMO—conditional gradient. In contrast, known to us similar techniques for other problems with convex structure (nonsmooth convex minimization, convex–concave saddle point problems, even as simple as bilinear ones, and variational inequalities with monotone operators, even as simple as affine) are quite recent and utilize common approach based on Fenchel-type representations of the associated objectives/vector fields. The goal of this paper was to develop alternative (and seemingly much simpler) decomposition techniques based on LMO for bilinear saddle point problems and for variational inequalities with affine monotone operators.     

A simple averaging process for approximating the solutions of certain optimal control problems

It is shown that the extremal solutions of fixed duration Mayer control problems with implicit terminal constraints can be interpreted as fixed points of certain function-valued operators F constructed by solving pairs of initial value problems in tandem. A class of simple recursive averaging processes is proposed for approximating the fixed points of F. Results from the theory of monotone Hilbert space operators are used to establish the convergence of the averaging processes for a general linear-quadratic curve follower problem with unbounded control inputs, and for a simple second order bounded control input problem.     

Convergence analysis of the direct extension of ADMM for multiple-block separable convex minimization

Recently, the alternating direction method of multipliers (ADMM) has found many efficient applications in various areas; and it has been shown that the convergence is not guaranteed when it is directly extended to the multiple-block case of separable convex minimization problems where there are m ≥ 3 functions without coupled variables in the objective. This fact has given great impetus to investigate various conditions on both the model and the algorithm’s parameter that can ensure the convergence of the direct extension of ADMM (abbreviated as “e-ADMM”). Despite some results under very strong conditions (e.g., at least (m ? 1) functions should be strongly convex) that are applicable to the generic case with a general m, some others concentrate on the special case of m = 3 under the relatively milder condition that only one function is assumed to be strongly convex. We focus on extending the convergence analysis from the case of m = 3 to the more general case of m ≥ 3. That is, we show the convergence of e-ADMM for the case of m ≥ 3 with the assumption of only (m ? 2) functions being strongly convex; and establish its convergence rates in different scenarios such as the worst-case convergence rates measured by iteration complexity and the globally linear convergence rate under stronger assumptions. Thus the convergence of e-ADMM for the general case of m ≥ 4 is proved; this result seems to be still unknown even though it is intuitive given the known result of the case of m = 3. Even for the special case of m = 3, our convergence results turn out to be more general than the existing results that are derived specifically for the case of m = 3.     

Hybrid inexact proximal point algorithms based on RMM frameworks with applications to variational inclusion problems

A new application-oriented notion of relatively A-maximal monotonicity (RMM) framework is introduced, and then it is applied to the approximation solvability of a general class of inclusion problems, while generalizing other existing results on linear convergence, including Rockafellar’s theorem (1976) on linear convergence using the proximal point algorithm in a real Hilbert space setting. The obtained results not only generalize most of the existing investigations, but also reduce smoothly to the case of the results on maximal monotone mappings and corresponding classical resolvent operators. Furthermore, our proof approach differs significantly to that of Rockafellar’s celebrated work, where the Lipschitz continuity of M ^?1, the inverse of M:X→2 ^X , at zero is assumed to achieve a linear convergence of the proximal point algorithm. Note that the notion of relatively A-maximal monotonicity framework seems to be used to generalize the classical Yosida approximation (which is applied and studied mostly based on the classical resolvent operator in the literature) that in turn can be applied to first-order evolution equations as well as evolution inclusions.     

Stable Optimizationless Recovery from Phaseless Linear Measurements

We address the problem of recovering an n-vector from m linear measurements lacking sign or phase information. We show that lifting and semidefinite relaxation suffice by themselves for stable recovery in the setting of m=O(nlogn) random sensing vectors, with high probability. The recovery method is optimizationless in the sense that trace minimization in the PhaseLift procedure is unnecessary. That is, PhaseLift reduces to a feasibility problem. The optimizationless perspective allows for a Douglas-Rachford numerical algorithm that is unavailable for PhaseLift. This method exhibits linear convergence with a favorable convergence rate and without any parameter tuning.     

Inertial proximal algorithm for difference of two maximal monotone operators

In this note, a new algorithm is presented for finding a zero of difference of two maximal monotone operators T and S, i.e., T — S in finite dimensional real Hilbert space H in which operator S has local boundedness property. This condition is weaker than Moudafi’s condition on operator S in [13]. Moreover, applying some conditions on inertia term in new algorithm, one can improve speed of convergence of sequence.     

Local convergence in Fermat's problem

The general Fermat problem is to find the minimum of the weighted sum of distances fromm destination points in Euclideann-space. Kuhn recently proved that a classical iterative algorithm converges to the unique minimizing point , for any choice of the initial point except for a denumerable set. In this note, it is shown that although convergence is global, the rapidity of convergence depends strongly upon whether or not  is a destination. If  is not a destination, then locally convergence is always linear with upper and lower asymptotic convergence boundsλ andλ′ (λ ≥ 1/2, whenn=2). If  is a destination, then convergence can be either linear, quadratic or sublinear. Three numerical examples which illustrate the different possibilities are given and comparisons are made with the use of Steffensen's scheme to accelerate convergence.     

Inexact <Emphasis Type="Italic">A</Emphasis>-Proximal Point Algorithm and Applications to Nonlinear Variational Inclusion Problems

A generalization to the Rockafellar theorem (1976) on the linear convergence in the context of approximating a solution to a general class of inclusion problems involving set-valued A-maximal relaxed monotone mappings using the proximal point algorithm in a real Hilbert space setting is given. There exists a vast literature on this theorem, but most of the investigations are focused on relaxing the proximal point algorithm and applying it to the inclusion problems. The general framework for A-maximal relaxed monotonicity generalizes the theory of set-valued maximal monotone mappings, including H-maximal monotone mappings. The obtained results are general in nature, while application-oriented as well.     

Convexification, Concavification and Monotonization in Global Optimization

We show in this paper that via certain convexification, concavification and monotonization schemes a nonconvex optimization problem over a simplex can be always converted into an equivalent better-structured nonconvex optimization problem, e.g., a concave optimization problem or a D.C. programming problem, thus facilitating the search of a global optimum by using the existing methods in concave minimization and D.C. programming. We first prove that a monotone optimization problem (with a monotone objective function and monotone constraints) can be transformed into a concave minimization problem over a convex set or a D.C. programming problem via pth power transformation. We then prove that a class of nonconvex minimization problems can be always reduced to a monotone optimization problem, thus a concave minimization problem or a D.C. programming problem.     

A note on the optimization of constrained design problems

This paper describes a new algorithm for solving constrained optimization problems, based on a method proposed by Chattopadhyay. The proposed algorithm replaces the original problem withm constraints,m>1, by a sequence of optimization problems, with one constraint. Here, we modify the algorithm given by Chattopadhyay in order to make it applicable for a larger class of optimization problems and to improve its convergence characteristics.     

Algebraic and topological closure conditions for classes of pseudo-Boolean functions

We examine classes of real-valued functions of 0-1 variables closed under algebraic operations as well as topological convergence, and having a certain local characteristic (requiring that any function not in the class should have a k-variable minor not belonging to this class). It is shown that for k=2, the only 4 maximal classes with these properties are those of submodular, supermodular, monotone increasing and monotone decreasing functions. All the 13 locally defined closed classes are determined and shown to be intersections of the 4 maximal ones. All maximal classes for k≥3 are determined and characterized by the sign of higher order derivatives of the functions in the class.     

Generating all minimal integral solutions to AND-OR systems of monotone inequalities: Conjunctions are simpler than disjunctions

We consider monotone ∨,∧-formulae φ of m atoms, each of which is a monotone inequality of the form f_i(x)?t_i over the integers, where for i=1,…,m, f_i:Zⁿ?R is a given monotone function and t_i is a given threshold. We show that if the ∨-degree of φ is bounded by a constant, then for linear, transversal and polymatroid monotone inequalities all minimal integer vectors satisfying φ can be generated in incremental quasi-polynomial time. In contrast, the enumeration problem for the disjunction of m inequalities is NP-hard when m is part of the input. We also discuss some applications of the above results in disjunctive programming, data mining, matroid and reliability theory.