Quadratic Convergence of Newton's Method for Convex Interpolation and Smoothing

   Abstract. In this paper, we prove that Newton's method for convex best interpolation is locally quadratically convergent, giving an answer to a question of Irvine, Marin, and Smith [7] and strengthening a result of Andersson and Elfving [1] and our previous work [5]. A damped Newton-type method is presented which has global quadratic convergence. Analogous results are obtained for the convex smoothing problem. Numerical examples are presented.     

Some Randomized Algorithms for Convex Quadratic Programming

We adapt some randomized algorithms of Clarkson [3] for linear programming to the framework of so-called LP-type problems, which was introduced by Sharir and Welzl [10]. This framework is quite general and allows a unified and elegant presentation and analysis. We also show that LP-type problems include minimization of a convex quadratic function subject to convex quadratic constraints as a special case, for which the algorithms can be implemented efficiently, if only linear constraints are present. We show that the expected running times depend only linearly on the number of constraints, and illustrate this by some numerical results. Even though the framework of LP-type problems may appear rather abstract at first, application of the methods considered in this paper to a given problem of that type is easy and efficient. Moreover, our proofs are in fact rather simple, since many technical details of more explicit problem representations are handled in a uniform manner by our approach. In particular, we do not assume boundedness of the feasible set as required in related methods. Accepted 7 May 1997     

A General Approach to Convergence Properties of Some Methods for Nonsmooth Convex Optimization

Based on the notion of the ε -subgradient, we present a unified technique to establish convergence properties of several methods for nonsmooth convex minimization problems. Starting from the technical results, we obtain the global convergence of: (i) the variable metric proximal methods presented by Bonnans, Gilbert, Lemaréchal, and Sagastizábal, (ii) some algorithms proposed by Correa and Lemaréchal, and (iii) the proximal point algorithm given by Rockafellar. In particular, we prove that the Rockafellar—Todd phenomenon does not occur for each of the above mentioned methods. Moreover, we explore the convergence rate of {||x _k || } and {f(x _k ) } when {x _k } is unbounded and {f(x _k ) } is bounded for the non\-smooth minimization methods (i), (ii), and (iii). Accepted 15 October 1996     

Relaxation Methods for Strictly Convex Regularizations of Piecewise Linear Programs

We give an algorithm for minimizing the sum of a strictly convex function and a convex piecewise linear function. It extends several dual coordinate ascent methods for large-scale linearly constrained problems that occur in entropy maximization, quadratic programming, and network flows. In particular, it may solve exact penalty versions of such (possibly inconsistent) problems, and subproblems of bundle methods for nondifferentiable optimization. It is simple, can exploit sparsity, and in certain cases is highly parallelizable. Its global convergence is established in the recent framework of B -functions (generalized Bregman functions). Accepted 29 October 1996     

A Modified Alternating Direction Method for Variational Inequality Problems

The alternating direction method is an attractive method for solving large-scale variational inequality problems whenever the subproblems can be solved efficiently. However, the subproblems are still variational inequality problems, which are as structurally difficult to solve as the original one. To overcome this disadvantage, in this paper we propose a new alternating direction method for solving a class of nonlinear monotone variational inequality problems. In each iteration the method just makes an orthogonal projection to a simple set and some function evaluations. We report some preliminary computational results to illustrate the efficiency of the method. Accepted 4 May 2001. Online publication 19 October, 2001.     

A Regularization Newton Method for Solving Nonlinear Complementarity Problems

In this paper we construct a regularization Newton method for solving the nonlinear complementarity problem (NCP(F )) and analyze its convergence properties under the assumption that F is a P ₀ -function. We prove that every accumulation point of the sequence of iterates is a solution of NCP(F ) and that the sequence of iterates is bounded if the solution set of NCP(F ) is nonempty and bounded. Moreover, if F is a monotone and Lipschitz continuous function, we prove that the sequence of iterates is bounded if and only if the solution set of NCP(F ) is nonempty by setting , where is a parameter. If NCP(F) has a locally unique solution and satisfies a nonsingularity condition, then the convergence rate is superlinear (quadratic) without strict complementarity conditions. At each step, we only solve a linear system of equations. Numerical results are provided and further applications to other problems are discussed. Accepted 25 March 1998     

A Nyström method for boundary integral equations in domains with corners

Summary We give a convergence and error analysis for a Nyström method on a graded mesh for the numerical solution of boundary integral equations for the harmonic Dirichlet problem in plane domains with corners.

Dedicated to Professor L. Collatz on the occassion of his 80th birthday     

Unicity Results for General Linear Semi-Infinite Optimization Problems Using a New Concept of Active Constraints

We consider parametric semi-infinite optimization problems without the usual asssumptions on the continuity of the involved mappings and on the compactness of the index set counting the inequalities. We establish a characterization of those optimization problems which have a unique or strongly unique solution and which are stable under small pertubations. This result generalizes a well-known theorem of Nürnberger. The crucial roles in our investigations are a new concept of active constraints, a generalized Slater's condition, and a Kuhn—Tucker-type theorem. Finally, we give some applications in vector optimization, for approximation problems in normed spaces, and in the stability of the minimal value. Accepted 5 August 1996     

A Nonmonotone Trust Region Method for Nonlinear Programming with Simple Bound Constraints

In this paper we propose a nonmonotone trust region algorithm for optimization with simple bound constraints. Under mild conditions, we prove the global convergence of the algorithm. For the monotone case it is also proved that the correct active set can be identified in a finite number of iterations if the strict complementarity slackness condition holds, and so the proposed algorithm reduces finally to an unconstrained minimization method in a finite number of iterations, allowing a fast asymptotic rate of convergence. Numerical experiments show that the method is efficient. Accepted 5 September 2000. Online publication 4 December 2000.     

Estimating L ∞ Norms by L 2k Norms for Functions on Orbits

Abstract. Let G be a compact group acting in a real vector space V . We obtain a number of inequalities relating the L ^∞ norm of a matrix element of the representation of G with its L ^2k norm for a positive integer k . As an application, we obtain approximation algorithms to find the maximum absolute value of a given multivariate polynomial over the unit sphere (in which case G is the orthogonal group) and for the assignment problem of degree d , a hard problem of combinatorial optimization generalizing the quadratic assignment problem (in which case G is the symmetric group).     

Contracted Zero Distributions of Extremal Polynomials Associated with Slowly Decaying Weights

We consider the asymptotic zero behavior of polynomials that are extremal with respect to slowly decaying weights on [0, ∈fty) , such as the log-normal weight \exp(-γ ² log  ² x) . The zeros are contracted by taking the appropriate d _n th roots with d _n →∈fty . The limiting distribution of the contracted zeros is described in terms of the solution of an extremal problem in logarithmic potential theory with a circular symmetric external field. November 23, 1998. Date revised: February 8, 1999. Date accepted: March 2, 1999.     

Generalization of Primal—Dual Interior-Point Methods to Convex Optimization Problems in Conic Form

We generalize primal—dual interior-point methods for linear programming (LP) problems to the convex optimization problems in conic form. Previously, the most comprehensive theory of symmetric primal—dual interior-point algorithms was given by Nesterov and Todd for feasible regions expressed as the intersection of a symmetric cone with an affine subspace. In our setting, we allow an arbitrary convex cone in place of the symmetric cone. Even though some of the impressive properties attained by Nesterov—Todd algorithms are impossible in this general setting of convex optimization problems, we show that essentially all primal—dual interior-point algorithms for LP can be extended easily to the general setting. We provide three frameworks for primal—dual algorithms, each framework corresponding to a different level of sophistication in the algorithms. As the level of sophistication increases, we demand better formulations of the feasible solution sets. Our algorithms, in return, attain provably better theoretical properties. We also make a very strong connection to quasi-Newton methods by expressing the square of the symmetric primal—dual linear transformation (the so-called scaling) as a quasi-Newton update in the case of the least sophisticated framework. August 25, 1999. Final version received: March 7, 2001.     

Second-Order Analysis for Control Constrained Optimal Control Problems of Semilinear Elliptic Systems

This paper presents a second-order analysis for a simple model optimal control problem of a partial differential equation, namely, a well-posed semilinear elliptic system with constraints on the control variable only. The cost to be minimized is a standard quadratic functional. Assuming the feasible set to be polyhedric, we state necessary and sufficient second-order optimality conditions, including a characterization of the quadratic growth condition. Assuming that the second-order sufficient condition holds, we give a formula for the second-order expansion of the value of the problem as well as the directional derivative of the optimal control, when the cost function is perturbed. Then we extend the theory of second-order optimality conditions to the case of vector-valued controls when the feasible set is defined by local and smooth convex constraints. When the space dimension n is greater than 3, the results are based on a two norms approach, involving spaces L ² and L ^s , with s>n/2 . Accepted 27 January 1997     

A Class of Hamilton—Jacobi Equations with Unbounded Coefficients in Hilbert Spaces

We establish existence and comparison theorems for a class of Hamilton—Jacobi equations. The class of Hamilton—Jacobi equations includes and is broader than those studied in [8] We apply the existence and uniqueness results to characterizing the value functions associated with the optimal control of systems governed by partial differential equations of parabolic type. Accepted 11 May 2001. Online publication 5 October 2001.