Convergence results for a self-dual regularization of convex problems

We study a one-parameter regularization technique for convex optimization problems whose main feature is self-duality with respect to the Legendre–Fenchel conjugation. The self-dual technique, introduced by Goebel, can be defined for both convex and saddle functions. When applied to the latter, we show that if a saddle function has at least one saddle point, then the sequence of saddle points of the regularized saddle functions converges to the saddle point of minimal norm of the original one. For convex problems with inequality and state constraints, we apply the regularization directly on the objective and constraint functions, and show that, under suitable conditions, the associated Lagrangians of the regularized problem hypo/epi-converge to the original Lagrangian, and that the associated value functions also epi-converge to the original one. Finally, we find explicit conditions ensuring that the regularized sequence satisfies Slater's condition.     

A minimization method with approximation of feasible set and epigraph of objective function

For a convex programming problem we propose a solution method which belongs to the class of cutting-plane methods. When constructing approximate solutions to the problem, this technique concurrently approximates its feasible set and the epigraph of the objective function. Planes for cutting the iteration points are being constructed with the help of subgradients of the objective function and left-hand sides of constraints. In this connection, one can find each iteration point by solving a linear programming problem. As distinct from most other well-known cuttingplane methods, the proposed technique allows the possibility to periodically update approximating sets by dropping accumulated constraints. We substantiate the convergence of the proposed method and discuss its numerical realization.     

Numerical differentiation for the second order derivatives of functions of two variables

A regularized optimization problem for computing numerical differentiation for the second order derivatives of functions with two variables from noisy values at scattered points is discussed in this article. We prove the existence and uniqueness of the solution to this problem, provide a constructive scheme for the solution which is based on bi-harmonic Green's function and give a convergence estimate of the regularized solution to the exact solution for the problem under a simple choice of regularization parameter. The efficiency of the constructive scheme is shown by some numerical examples.     

Minimizing nonsmooth DC functions via successive DC piecewise-affine approximations

We introduce a proximal bundle method for the numerical minimization of a nonsmooth difference-of-convex (DC) function. Exploiting some classic ideas coming from cutting-plane approaches for the convex case, we iteratively build two separate piecewise-affine approximations of the component functions, grouping the corresponding information in two separate bundles. In the bundle of the first component, only information related to points close to the current iterate are maintained, while the second bundle only refers to a global model of the corresponding component function. We combine the two convex piecewise-affine approximations, and generate a DC piecewise-affine model, which can also be seen as the pointwise maximum of several concave piecewise-affine functions. Such a nonconvex model is locally approximated by means of an auxiliary quadratic program, whose solution is used to certify approximate criticality or to generate a descent search-direction, along with a predicted reduction, that is next explored in a line-search setting. To improve the approximation properties at points that are far from the current iterate a supplementary quadratic program is also introduced to generate an alternative more promising search-direction. We discuss the main convergence issues of the line-search based proximal bundle method, and provide computational results on a set of academic benchmark test problems.     

An approximate bundle method for solving nonsmooth equilibrium problems

We present an approximate bundle method for solving nonsmooth equilibrium problems. An inexact cutting-plane linearization of the objective function is established at each iteration, which is actually an approximation produced by an oracle that gives inaccurate values for the functions and subgradients. The errors in function and subgradient evaluations are bounded and they need not vanish in the limit. A descent criterion adapting the setting of inexact oracles is put forward to measure the current descent behavior. The sequence generated by the algorithm converges to the approximately critical points of the equilibrium problem under proper assumptions. As a special illustration, the proposed algorithm is utilized to solve generalized variational inequality problems. The numerical experiments show that the algorithm is effective in solving nonsmooth equilibrium problems.     

Regularization of a two-dimensional singularly perturbed parabolic problem

A regularized asymptotic expansion of the solution to a singularly perturbed two-dimensional parabolic problem in domains with boundaries containing corner points is constructed. The asymptotics of solutions to such problems contain ordinary boundary-layer functions, parabolic boundary-layer functions, and their products, which describe a corner boundary layer.     

Dimension-splitting data points redistribution for meshless approximation

To better approximate nearly singular functions with meshless methods, we propose a data points redistribution method extended from the well-known one-dimensional equidistribution principle. With properly distributed data points, nearly singular functions can be well approximated by linear combinations of global radial basis functions. The proposed method is coupled with an adaptive trial subspace selection algorithm in order to reduce computational cost. In our numerical examples, clear exponential convergence (with respect to the numbers of data points) can be observed.     

Gap Functions and Error Bounds for Weak Generalized Ky Fan Inequalities

In this paper, we study a weak generalized Ky Fan inequality with cone constraints through image space analysis. First, we characterize the separation for the weak generalized Ky Fan inequality with cone constraints using the saddle points of generalized Lagrangian function. Then, we use regular weak separation functions to construct gap functions and regularized gap functions for the weak generalized Ky Fan inequality with cone constraints in a general way, and establish its error bounds in terms of these gap functions.     

Algorithms for the Best Simultaneous Uniform Approximation of a Family of Functions Continuous on a Compact Set by a Chebyshev Subspace

We generalize the cutting-plane method and the Remez method to the case of the problem of the best simultaneous uniform approximation of a family of functions continuous on a compact set.     

Generalizations of Tikhonov’s regularized method of least squares to non-Euclidean vector norms

Tikhonov’s regularized method of least squares and its generalizations to non-Euclidean norms, including polyhedral, are considered. The regularized method of least squares is reduced to mathematical programming problems obtained by “instrumental” generalizations of the Tikhonov lemma on the minimal (in a certain norm) solution of a system of linear algebraic equations with respect to an unknown matrix. Further studies are needed for problems concerning the development of methods and algorithms for solving reduced mathematical programming problems in which the objective functions and admissible domains are constructed using polyhedral vector norms.     

On a generalized proximal point method for solving equilibrium problems in Banach spaces

We introduce a regularized equilibrium problem in Banach spaces, involving generalized Bregman functions. For this regularized problem, we establish the existence and uniqueness of solutions. These regularizations yield a proximal-like method for solving equilibrium problems in Banach spaces. We prove that the proximal sequence is an asymptotically solving sequence when the dual space is uniformly convex. Moreover, we prove that all weak accumulation points are solutions if the equilibrium function is lower semicontinuous in its first variable. We prove, under additional assumptions, that the proximal sequence converges weakly to a solution.     

Differential quadrature method for space-fractional diffusion equations on 2D irregular domains

In mathematical physics, the space-fractional diffusion equations are of particular interest in the studies of physical phenomena modelled by Lévy processes, which are sometimes called super-diffusion equations. In this article, we develop the differential quadrature (DQ) methods for solving the 2D space-fractional diffusion equations on irregular domains. The methods in presence reduce the original equation into a set of ordinary differential equations (ODEs) by introducing valid DQ formulations to fractional directional derivatives based on the functional values at scattered nodal points on problem domain. The required weighted coefficients are calculated by using radial basis functions (RBFs) as trial functions, and the resultant ODEs are discretized by the Crank-Nicolson scheme. The main advantages of our methods lie in their flexibility and applicability to arbitrary domains. A series of illustrated examples are finally provided to support these points.     

Regularized Kernel-Based Reconstruction in Generalized Besov Spaces

We present a theoretical framework for reproducing kernel-based reconstruction methods in certain generalized Besov spaces based on positive, essentially self-adjoint operators. An explicit representation of the reproducing kernel is given in terms of an infinite series. We provide stability estimates for the kernel, including inverse Bernstein-type estimates for kernel-based trial spaces, and we give condition estimates for the interpolation matrix. Then, a deterministic error analysis for regularized reconstruction schemes is presented by means of sampling inequalities. In particular, we provide error bounds for a regularized reconstruction scheme based on a numerically feasible approximation of the kernel. This allows us to derive explicit coupling relations between the series truncation, the regularization parameters and the data set.     

Comparison of bundle and classical column generation

When a column generation approach is applied to decomposable mixed integer programming problems, it is standard to formulate and solve the master problem as a linear program. Seen in the dual space, this results in the algorithm known in the nonlinear programming community as the cutting-plane algorithm of Kelley and Cheney-Goldstein. However, more stable methods with better theoretical convergence rates are known and have been used as alternatives to this standard. One of them is the bundle method; our aim is to illustrate its differences with Kelley’s method. In the process we review alternative stabilization techniques used in column generation, comparing them from both primal and dual points of view. Numerical comparisons are presented for five applications: cutting stock (which includes bin packing), vertex coloring, capacitated vehicle routing, multi-item lot sizing, and traveling salesman. We also give a sketchy comparison with the volume algorithm. This research has been supported by Inria New Investigation Grant “Convex Optimization and Dantzig-Wolfe Decomposition”.     

Merit functions for semi-definite complemetarity problems

Merit functions such as the gap function, the regularized gap function, the implicit Lagrangian, and the norm squared of the Fischer-Burmeister function have played an important role in the solution of complementarity problems defined over the cone of nonnegative real vectors. We study the extension of these merit functions to complementarity problems defined over the cone of block-diagonal symmetric positive semi-definite real matrices. The extension suggests new solution methods for the latter problems. This research is supported by National Science Foundation Grant CCR-9311621.     

Fractional programming by lower subdifferentiability techniques

The notion of lower subdifferentiability is applied to the analysis of convex fractional programming problems. In particular, duality results and optimality conditions are presented, and the applicability of a cutting-plane algorithm using lower subgradients is discussed. These methods are useful also in generalized fractional programming, where, in the linear case, the performance of the cutting-plane algorithm is compared with that of the most efficient version of the Dinkelbach method, which is based on the solution of a parametric linear programming problem.The authors wish to thank Mr. Jaume Timoneda for his help in the implementation of the numerical methods on the computer and the referees for valuable comments and suggestions; the present improved statement and proof of Proposition 2.1 is due to one of them. Financial support from the Dirección General de Investigación Científica y Técnica (DGICYT), under project PS89-0058, is gratefully acknowledged.     

Gap functions and global error bounds for differential variational–hemivariational inequalities

This paper concerns with the study of a differential variational–hemivariational inequality (DVHVI, for short) in infinite-dimensional Banach spaces. We first introduce the new concept of gap functions for the variational control system of (DVHVI). Then, we consider two kinds of gap functions which are regularized gap function and Moreau–Yosida regularized gap function, respectively, and examine the relevant properties of the gap functions. Moreover, two global error bounds which depend implicitly on the regularized gap function and the Moreau–Yosida regularized gap function, accordingly, are obtained. Finally, in order to illustrate the applicability of the theoretical results, we investigate a coupled dynamic system which is formulated by a nonlinear reaction–diffusion equation described by a time-dependent nonsmooth semipermeability problem.     

Design and verify: a new scheme for generating cutting-planes

A cutting-plane procedure for integer programming (IP) problems usually involves invoking a black-box procedure (such as the Gomory–Chvátal procedure) to compute a cutting-plane. In this paper, we describe an alternative paradigm of using the same cutting-plane black-box. This involves two steps. In the first step, we design an inequality $cx \le d$ where $c$ and $d$ are integral, independent of the cutting-plane black-box. In the second step, we verify that the designed inequality is a valid inequality by verifying that the set $P \cap \{x\in \mathbb R ^n \mid cx \ge d + 1\} \cap \mathbb Z ^n$ is empty using cutting-planes from the black-box. Here $P$ is the feasible region of the linear-programming relaxation of the IP. We refer to the closure of all cutting-planes that can be verified to be valid using a specific cutting-plane black-box as the verification closure of the considered cutting-plane black-box. This paper undertakes a systematic study of properties of verification closures of various cutting-plane black-box procedures.     

Reformulation versus cutting-planes for robust optimization

Robust optimization (RO) is a tractable method to address uncertainty in optimization problems where uncertain parameters are modeled as belonging to uncertainty sets that are commonly polyhedral or ellipsoidal. The two most frequently described methods in the literature for solving RO problems are reformulation to a deterministic optimization problem or an iterative cutting-plane method. There has been limited comparison of the two methods in the literature, and there is no guidance for when one method should be selected over the other. In this paper we perform a comprehensive computational study on a variety of problem instances for both robust linear optimization (RLO) and robust mixed-integer optimization (RMIO) problems using both methods and both polyhedral and ellipsoidal uncertainty sets. We consider multiple variants of the methods and characterize the various implementation decisions that must be made. We measure performance with multiple metrics and use statistical techniques to quantify certainty in the results. We find for polyhedral uncertainty sets that neither method dominates the other, in contrast to previous results in the literature. For ellipsoidal uncertainty sets we find that the reformulation is better for RLO problems, but there is no dominant method for RMIO problems. Given that there is no clearly dominant method, we describe a hybrid method that solves, in parallel, an instance with both the reformulation method and the cutting-plane method. We find that this hybrid approach can reduce runtimes to 50–75 % of the runtime for any one method and suggest ways that this result can be achieved and further improved on.     

A cutting-plane approach for large-scale capacitated multi-period facility location using a specialized interior-point method

We propose a cutting-plane approach (namely, Benders decomposition) for a class of capacitated multi-period facility location problems. The novelty of this approach lies on the use of a specialized interior-point method for solving the Benders subproblems. The primal block-angular structure of the resulting linear optimization problems is exploited by the interior-point method, allowing the (either exact or inexact) efficient solution of large instances. The consequences of different modeling conditions and problem specifications on the computational performance are also investigated both theoretically and empirically, providing a deeper understanding of the significant factors influencing the overall efficiency of the cutting-plane method. The methodology proposed allowed the solution of instances of up to 200 potential locations, one million customers and three periods, resulting in mixed integer linear optimization problems of up to 600 binary and 600 millions of continuous variables. Those problems were solved by the specialized approach in less than one hour and a half, outperforming other state-of-the-art methods, which exhausted the (144 GB of) available memory in the largest instances.