Efficient implementation of pattern recognition algorithms

The model of estimation algorithms designed for pattern recognition is considered. A partial order for a class of systems of support sets that admit an efficient implementation is defined and investigated. It is proved that the problem of finding a counterexample for the efficiency property with additional constraints is NP-complete.     

Convergence proof of minimization algorithms for nonconvex functions

In this paper, we prove a theorem of convergence to a point for descent minimization methods. When the objective function is differentiable, the convergence point is a stationary point. The theorem, however, is applicable also to nondifferentiable functions. This theorem is then applied to prove convergence of some nongradient algorithms.     

Random algorithms for convex minimization problems

This paper deals with iterative gradient and subgradient methods with random feasibility steps for solving constrained convex minimization problems, where the constraint set is specified as the intersection of possibly infinitely many constraint sets. Each constraint set is assumed to be given as a level set of a convex but not necessarily differentiable function. The proposed algorithms are applicable to the situation where the whole constraint set of the problem is not known in advance, but it is rather learned in time through observations. Also, the algorithms are of interest for constrained optimization problems where the constraints are known but the number of constraints is either large or not finite. We analyze the proposed algorithm for the case when the objective function is differentiable with Lipschitz gradients and the case when the objective function is not necessarily differentiable. The behavior of the algorithm is investigated both for diminishing and non-diminishing stepsize values. The almost sure convergence to an optimal solution is established for diminishing stepsize. For non-diminishing stepsize, the error bounds are established for the expected distances of the weighted averages of the iterates from the constraint set, as well as for the expected sub-optimality of the function values along the weighted averages.     

Convergence of some algorithms for convex minimization

We present a simple and unified technique to establish convergence of various minimization methods. These contain the (conceptual) proximal point method, as well as implementable forms such as bundle algorithms, including the classical subgradient relaxation algorithm with divergent series.An important research work of Phil Wolfe's concerned convex minimization. This paper is dedicated to him, on the occasion of his 65th birthday, in appreciation of his creative and pioneering work.     

Adaptive smoothing algorithms for nonsmooth composite convex minimization

We propose an adaptive smoothing algorithm based on Nesterov’s smoothing technique in Nesterov (Math Prog 103(1):127–152, 2005) for solving “fully” nonsmooth composite convex optimization problems. Our method combines both Nesterov’s accelerated proximal gradient scheme and a new homotopy strategy for smoothness parameter. By an appropriate choice of smoothing functions, we develop a new algorithm that has the \(\mathcal {O}\left( \frac{1}{\varepsilon }\right) \)-worst-case iteration-complexity while preserves the same complexity-per-iteration as in Nesterov’s method and allows one to automatically update the smoothness parameter at each iteration. Then, we customize our algorithm to solve four special cases that cover various applications. We also specify our algorithm to solve constrained convex optimization problems and show its convergence guarantee on a primal sequence of iterates. We demonstrate our algorithm through three numerical examples and compare it with other related algorithms.     

Empty convex polygons in almost convex sets

A finite set of points, in general position in the plane, is almost convex if every triple determines a triangle with at most one point in its interior. For every ℓ ≥ 3, we determine the maximum size of an almost convex set that does not contain the vertex set of an empty convex ℓ-gon. Partially supported by grants T043631 and NK67867 of the Hungarian NFSR (OTKA).     

Separating plane algorithms for convex optimization

The equivalent formulation of a convex optimization problem is the computation of a value of a conjugate function at the origin. The latter can be achieved by approximation of the epigraph of the conjugate function around the origin and gradual refinement of the approximation. This yields a generic algorithm of convex optimization which transforms into some well-known techniques when certain strategies of approximation are employed. It also suggests new algorithmic approaches with promising computational experience and provides a uniform treatment of constrained and unconstrained optimization.     

Nonmonotone bundle-type scheme for convex nonsmooth minimization

In this paper, we present a general scheme for bundle-type algorithms which includes a nonmonotone line search procedure and for which global convergence can be proved. Some numerical examples are reported, showing that the nonmonotonicity can be beneficial from a computational point of view.This work was partially supported by the National Research Program on Metodi di ottimizzazione per le decisioni, Ministero dell' Universitá e della Ricerca Scientifica e Tecnologica and by ASI: Agenzia Spaziale Italiana.     

Algorithmic construction of low-discrepancy point sets via dependent randomized rounding

We provide a deterministic algorithm that constructs small point sets exhibiting a low star discrepancy. The algorithm is based on recent results on randomized roundings respecting hard constraints and their derandomization. It is structurally much simpler than a previous algorithm presented for this problem in [B. Doerr, M. Gnewuch, A. Srivastav, Bounds and constructions for the star discrepancy via δ$\delta$-covers, J. Complexity, 21 (2005) 691–709]. Besides leading to better theoretical running time bounds, our approach also can be implemented with reasonable effort. We implemented this algorithm and performed numerical comparisons with other known low-discrepancy constructions. The experiments take place in dimensions ranging from 5 to 21 and indicate that our algorithm leads to superior results if the dimension is relatively high and the number of points that have to be constructed is rather small.     

Approximation algorithms for dynamic resource allocation

We consider a problem of allocating limited quantities of M types of resources among N independent activities that evolve over T epochs. In each epoch, we assign to each activity a task which consumes resources, generates utility, and determines the subsequent state of the activity. We study the complexity of, and approximation algorithms for, maximizing average utility.     

Proximal-type algorithms for split minimization problem in P-uniformly convex metric spaces

Numerical Algorithms - In this paper, we study strong convergence of some proximal-type algorithms to a solution of split minimization problem in complete p-uniformly convex metric spaces. We also...     

Global convergence rates for descent algorithms: Improved results for strictly convex problems

Convergence to the minimal value is studied for the important type of descent algorithm which, at each interation, uses a search direction making an angle with the negative gradient which is smaller than a prespecified angle. Improvements on existing convergence rate results are obtained.Paper received on 4 October, 1977; in revised form, April 3, 1978     

Some investigations about a unified approach to quadratically convergent algorithms for function minimization

In 1969, Huang (Ref. 1) proposed a unified approach to quadratically convergent algorithms for function minimization without constraints; if we slightly modify a few points of his development, some other algorithms can be generated, among which several were published after 1969. It is also possible to generate a class of algorithms which provide quadratic convergence without linear search at each step.     

Non-monotone trust-region algorithms for nonlinear optimization subject to convex constraints

This paper presents two new trust-region methods for solving nonlinear optimization problems over convex feasible domains. These methods are distinguished by the fact that they do not enforce strict monotonicity of the objective function values at successive iterates. The algorithms are proved to be convergent to critical points of the problem from any starting point. Extensive numerical experiments show that this approach is competitive with the LANCELOT package.     

Improved bounds for minimal feedback vertex sets in tournaments

We study feedback vertex sets (FVS) in tournaments, which are orientations of complete graphs. As our main result, we show that any tournament on n nodes has at most 1.5949ⁿ minimal FVS. This significantly improves the previously best upper bound of 1.6667ⁿ by Fomin et al. [STOC 2016] and 1.6740ⁿ by Gaspers and Mnich [J. Graph Theory 72 (1):72–89, 2013]. Our new upper bound almost matches the best‐known lower bound of , due to Gaspers and Mnich. Our proof is algorithmic, and shows that all minimal FVS of tournaments can be enumerated in time .     

Additive and multiplicative Sidon sets

Summary Inspired by a paper of Sárk?zy [4] we study sets of integers and sets of residues with the property that all sums and all products are distinct.     

A -algorithm for convex minimization

For convex minimization we introduce an algorithm based on -space decomposition. The method uses a bundle subroutine to generate a sequence of approximate proximal points. When a primal-dual track leading to a solution and zero subgradient pair exists, these points approximate the primal track points and give the algorithm's , or corrector, steps. The subroutine also approximates dual track points that are -gradients needed for the method's -Newton predictor steps. With the inclusion of a simple line search the resulting algorithm is proved to be globally convergent. The convergence is superlinear if the primal-dual track points and the objective's -Hessian are approximated well enough. Dedicated to Terry Rockafellar who has had a great influence on our work via strong support for proximal points and for structural definitions that involve tangential convergence. On leave from INRIA Rocquencourt Research of the first author supported by the National Science Foundation under Grant No. DMS-0071459 and by CNPq (Brazil) under Grant No. 452966/2003-5. Research of the second author supported by FAPERJ (Brazil) under Grant No.E26/150.581/00 and by CNPq (Brazil) under Grant No. 383066/2004-2.     

Bracketing numbers for axis-parallel boxes and applications to geometric discrepancy

In the first part of this paper we derive lower bounds and constructive upper bounds for the bracketing numbers of anchored and unanchored axis-parallel boxes in the d$d$-dimensional unit cube.