A bundle-filter method for nonsmooth convex constrained optimization

For solving nonsmooth convex constrained optimization problems, we propose an algorithm which combines the ideas of the proximal bundle methods with the filter strategy for evaluating candidate points. The resulting algorithm inherits some attractive features from both approaches. On the one hand, it allows effective control of the size of quadratic programming subproblems via the compression and aggregation techniques of proximal bundle methods. On the other hand, the filter criterion for accepting a candidate point as the new iterate is sometimes easier to satisfy than the usual descent condition in bundle methods. Some encouraging preliminary computational results are also reported.     

A hybrid descent method for solving a convex constrained optimization problem with applications

The purpose of this paper is to introduce a hybrid descent algorithm for finding a common point in fixed‐point sets of quasi‐nonexpansive mappings and solution sets of variational inequality problems. In the framework of Hilbert spaces, the strong convergence of the hybrid descent algorithm is established. Numerical experiments for the bandwidth allocation, which demonstrate the effectiveness, performance, and convergence of the proposed algorithm, are provided.     

A hybrid convex variational model for image restoration

We propose a new hybrid model for variational image restoration using an alternative diffusion switching non-quadratic function with a parameter. The parameter is chosen adaptively so as to minimize the smoothing near the edges and allow the diffusion to smooth away from the edges. This model belongs to a class of edge-preserving regularization methods proposed in the past, the ?-function formulation. This involves a minimizer to the associated energy functional. We study the existence and uniqueness of the energy functional of the model. Using real and synthetic images we show that the model is effective in image restoration.     

An algorithm for constrained convex optimization

This paper presents a feasible direction algorithm for the minimization of a pseudoconvex function over a smooth, compact, convex set. We establish that each cluster point of the generated sequence is an optimal solution of the problem without introducing anti-jamming procedures. Each iteration of the algorithm involves as subproblems only one line search for a zero of a continuously differentiable convex function and one univariate function minimization on a compact interval.     

Infinite-dimensional convex programming with applications to constrained approximation

In this paper, existence and characterization of solutions and duality aspects of infinite-dimensional convex programming problems are examined. Applications of the results to constrained approximation problems are considered. Various duality properties for constrained interpolation problems over convex sets are established under general regularity conditions. The regularity conditions are shown to hold for many constrained interpolation problems. Characterizations of local proximinal sets and the set of best approximations are also given in normed linear spaces.The author is grateful to the referee for helpful suggestions which have contributed to the final preparation of this paper. This research was partially supported by Grant A68930162 from the Australian Research Council.     

Kronecker product approximations for image restoration with new mean boundary conditions

As an alternative to classic boundary conditions, new mean boundary conditions for image restoration problem have been recently introduced by Shi and Chang [Y. Shi, Q. Chang, Acceleration methods for image restoration problem with different boundary conditions, Applied Numerical Mathematics 58 (2008) 602-614]. In this paper, we propose an efficient scheme for computing the Kronecker product approximations of blurring matrices with the new mean boundary conditions. Our scheme does not require the symmetry condition of point spread functions. Detailed experiments in image restoration are given to demonstrate the efficiency of our scheme.     

Gradient-type penalty method with inertial effects for solving constrained convex optimization problems with smooth data

We consider the problem of minimizing a smooth convex objective function subject to the set of minima of another differentiable convex function. In order to solve this problem, we propose an algorithm which combines the gradient method with a penalization technique. Moreover, we insert in our algorithm an inertial term, which is able to take advantage of the history of the iterates. We show weak convergence of the generated sequence of iterates to an optimal solution of the optimization problem, provided a condition expressed via the Fenchel conjugate of the constraint function is fulfilled. We also prove convergence for the objective function values to the optimal objective value. The convergence analysis carried out in this paper relies on the celebrated Opial Lemma and generalized Fejér monotonicity techniques. We illustrate the functionality of the method via a numerical experiment addressing image classification via support vector machines.     

A limiting Lagrangian for infinitely constrained convex optimization inR n

For convex optimization inR ⁿ,we show how a minor modification of the usual Lagrangian function (unlike that of the augmented Lagrangians), plus a limiting operation, allows one to close duality gaps even in the absence of a Kuhn-Tucker vector [see the introductory discussion, and see the discussion in Section 4 regarding Eq. (2)]. The cardinality of the convex constraining functions can be arbitrary (finite, countable, or uncountable).In fact, our main result (Theorem 4.3) reveals much finer detail concerning our limiting Lagrangian. There are affine minorants (for any value 0<1 of the limiting parameter ) of the given convex functions, plus an affine form nonpositive onK, for which a general linear inequality holds onR ⁿAfter substantial weakening, this inequality leads to the conclusions of the previous paragraph.This work is motivated by, and is a direct outgrowth of, research carried out jointly with R. J. Duffin.This research was supported by NSF Grant No. GP-37510X1 and ONR Contract No. N00014-75-C0621, NR-047-048. This paper was presented at Constructive Approaches to Mathematical Models, a symposium in honor of R. J. Duffin, Pittsburgh, Pennsylvania, 1978. The author is grateful to Professor Duffin for discussions relating to the work reported here.The author wishes to thank R. J. Duffin for reading an earlier version of this paper and making numerous suggestions for improving it, which are incorporated here. Our exposition and proofs have profited from comments of C. E. Blair and J. Borwein.     

Level bundle methods for constrained convex optimization with various oracles

We propose restricted memory level bundle methods for minimizing constrained convex nonsmooth optimization problems whose objective and constraint functions are known through oracles (black-boxes) that might provide inexact information. Our approach is general and covers many instances of inexact oracles, such as upper, lower and on-demand accuracy oracles. We show that the proposed level bundle methods are convergent as long as the memory is restricted to at least four well chosen linearizations: two linearizations for the objective function, and two linearizations for the constraints. The proposed methods are particularly suitable for both joint chance-constrained problems and two-stage stochastic programs with risk measure constraints. The approach is assessed on realistic joint constrained energy problems, arising when dealing with robust cascaded-reservoir management.     

Ideal formulations for constrained convex optimization problems with indicator variables

Mathematical Programming - Motivated by modern regression applications, in this paper, we study the convexification of a class of convex optimization problems with indicator variables and...     

Robust optimization approximation for joint chance constrained optimization problem

Chance constraint is widely used for modeling solution reliability in optimization problems with uncertainty. Due to the difficulties in checking the feasibility of the probabilistic constraint and the non-convexity of the feasible region, chance constrained problems are generally solved through approximations. Joint chance constrained problem enforces that several constraints are satisfied simultaneously and it is more complicated than individual chance constrained problem. This work investigates the tractable robust optimization approximation framework for solving the joint chance constrained problem. Various robust counterpart optimization formulations are derived based on different types of uncertainty set. To improve the quality of robust optimization approximation, a two-layer algorithm is proposed. The inner layer optimizes over the size of the uncertainty set, and the outer layer optimizes over the parameter t which is used for the indicator function upper bounding. Numerical studies demonstrate that the proposed method can lead to solutions close to the true solution of a joint chance constrained problem.     

Kronecker product approximations for image restoration with anti‐reflective boundary conditions

The anti‐reflective boundary condition for image restoration was recently introduced as a mathematically desirable alternative to other boundary conditions presently represented in the literature. It has been shown that, given a centrally symmetric point spread function (PSF), this boundary condition gives rise to a structured blurring matrix, a submatrix of which can be diagonalized by the discrete sine transform (DST), leading to an O(n² log n) solution algorithm for an image of size n × n. In this paper, we obtain a Kronecker product approximation of the general structured blurring matrix that arises under this boundary condition, regardless of symmetry properties of the PSF. We then demonstrate the usefulness and efficiency of our approximation in an SVD‐based restoration algorithm, the computational cost of which would otherwise be prohibitive. Copyright © 2005 John Wiley & Sons, Ltd.     

Regularized dual gradient distributed method for constrained convex optimization over unbalanced directed graphs

Numerical Algorithms - This paper investigates a distributed optimization problem over a cooperative multi-agent time–varying network, where each agent has its own decision variables that...     

Combination of trust region method and simplicial decomposition for convex constrained nonlinear optimization

The algorithm presented in this article incorporates the trust region method (TR) into the restricted decomposition algorithm for convex-constrained nonlinear problems (RSDCC) to solve the master problem of RSDCC. The global convergence is proved. The computational comparison between the presented algorithm and RSDCC is given. The results show that the former is much better than the latter.     

A reduced Hessian method for constrained optimization

Reduced Hessian methods have been shown to be successful for equality constrained problems. However there are few results on reduced Hessian methods for general constrained problems. In this paper we propose a method for general constrained problems, based on Byrd and Schnabel's basis-independent algorithm. It can be regarded as a smooth extension of the standard reduced Hessian Method.Research supported in part by NSF, AFORS and ONR through NSF grant DMS-8920550.     

A feasible-direction method for nonlinear constrained optimization

In this paper we describe and analyse a numerical method presented by Herskovits for the solution of the nonlinear programming problem with inequality constraints. We propose some new updating rules for the parameters of the algorithm and show that these rules are valid and efficient.

---

Sunto In questo lavoro viene descritto e analizzato un metodo numerico proposto da Herskovits per la soluzione di problemi di minimo non lineari con vincoli di disuguaglianza. In particolare vengono proposte nuove regole per l’aggiornamento dei parametri dell’algoritmo e viene mostrato che tali regole sono valide ed efficienti.

     

A TRUST REGION METHOD WITH A CONIC MODEL FOR NONLINEARLY CONSTRAINED OPTIMIZATION

Trust region methods are powerful and effective optimization methods.The conic model method is a new type of method with more information available at each iteration than standard quadratic-based methods.The advantages of the above two methods can be combined to form a more powerful method for constrained optimization.The trust region subproblem of our method is to minimize a conic function subject to the linearized constraints and trust region bound.At the same time,the new algorithm still possesses robust global properties.The global convergence of the new algorithm under standard conditions is established.     

A SUCCESSIVE APPROXIMATION METHOD FOR SOLVING PROBABILISTIC CONSTRAINED PROGRAMS

ＡＳＵＣＣＥＳＳＩＶＥＡＰＰＲＯＸＩＭＡＴＩＯＮＭＥＴＨＯＤＦＯＲＳＯＬＶＩＮＧＰＲＯＢＡＢＩＬＩＳＴＩＣＣＯＮＳＴＲＡＩＮＥＤＰＲＯＧＲＡＭＳＷＡＮＧＪＩＮＤＥ（王金德）（ＤｅｐａｒｔｍｅｎｔｏｆＭａｔｈｅｍａｔｉｃｓ，ＮａｎｊｉｎｇＵｎｉｖｅｒｓ...     

Polyhedral approximation in mixed-integer convex optimization

Generalizing both mixed-integer linear optimization and convex optimization, mixed-integer convex optimization possesses broad modeling power but has seen relatively few advances in general-purpose solvers in recent years. In this paper, we intend to provide a broadly accessible introduction to our recent work in developing algorithms and software for this problem class. Our approach is based on constructing polyhedral outer approximations of the convex constraints, resulting in a global solution by solving a finite number of mixed-integer linear and continuous convex subproblems. The key advance we present is to strengthen the polyhedral approximations by constructing them in a higher-dimensional space. In order to automate this extended formulation we rely on the algebraic modeling technique of disciplined convex programming (DCP), and for generality and ease of implementation we use conic representations of the convex constraints. Although our framework requires a manual translation of existing models into DCP form, after performing this transformation on the MINLPLIB2 benchmark library we were able to solve a number of unsolved instances and on many other instances achieve superior performance compared with state-of-the-art solvers like Bonmin, SCIP, and Artelys Knitro.     

Inverse optimization for linearly constrained convex separable programming problems

In this paper, we study inverse optimization for linearly constrained convex separable programming problems that have wide applications in industrial and managerial areas. For a given feasible point of a convex separable program, the inverse optimization is to determine whether the feasible point can be made optimal by adjusting the parameter values in the problem, and when the answer is positive, find the parameter values that have the smallest adjustments. A sufficient and necessary condition is given for a feasible point to be able to become optimal by adjusting parameter values. Inverse optimization formulations are presented with ℓ₁ and ℓ₂ norms. These inverse optimization problems are either linear programming when ℓ₁ norm is used in the formulation, or convex quadratic separable programming when ℓ₂ norm is used.