Conditions for variable-metric algorithms to be conjugate-gradient algorithms

Variable-metric algorithms have played an important role in unconstrained optimization theory. This paper presents a sufficiency condition on the sequence of metrics in a variable-metric algorithm that will make it a conjugate-gradient algorithm. The Huang class of algorithms (Ref. 1) and the class of self-scaling variable-metric algorithms by Oren (Ref. 2) all satisfy the condition. This paper also includes a discussion of the behavior of algorithms that meet the condition on nonquadratic functions.     

Probabilistic analysis of a new class of strip packing algorithms

A new class of algorithms for online packing of rectangles into a strip is proposed and studied. It is proved that the expectation of the unfilled area for this class of algorithms is O(N ^2/3) in the standard (for this type of problems) probabilistic model for N random rectangles.     

Iterative algorithms for a new class of extended general nonconvex set-valued variational inequalities

Some new classes of extended general nonconvex set-valued variational inequalities and the extended general Wiener-Hopf inclusions are introduced. By the projection technique, equivalence between the extended general nonconvex set-valued variational inequalities and the fixed point problems as well as the extended general nonconvex Wiener-Hopf inclusions is proved. Then by using this equivalent formulation, we discuss the existence of solutions of the extended general nonconvex set-valued variational inequalities and construct some new perturbed finite step projection iterative algorithms with mixed errors for approximating the solutions of the extended general nonconvex set-valued variational inequalities. We also verify that the approximate solutions obtained by our algorithms converge to the solutions of the extended general nonconvex set-valued variational inequalities. The results presented in this paper extend and improve some known results from the literature.     

A new class of asynchronous iterative algorithms with order intervals

This paper deals with a new class of parallel asynchronous iterative algorithms for the solution of nonlinear systems of equations. The main feature of the new class of methods presented here is the possibility of flexible communication between processors. In particular partial updates can be exchanged. Approximation of the associated fixed point mapping is also considered. A detailed convergence study is presented. A connection with the Schwarz alternating method is made for the solution of nonlinear boundary value problems. Computational results on a shared memory multiprocessor IBM 3090 are briefly presented.

     

A new class of exact penalty functions and penalty algorithms

For nonlinear programming problems, we propose a new class of smooth exact penalty functions, which includes both barrier-type and exterior-type penalty functions as special cases. We develop necessary and sufficient conditions for exact penalty property and inverse proposition of exact penalization, respectively. Furthermore, we establish the equivalent relationship between these penalty functions and classical simple exact penalty functions in the sense of exactness property. In addition, a feasible penalty function algorithm is proposed. The convergence analysis of the algorithm is presented, including the global convergence property and finite termination property. Finally, numerical results are reported.     

基于一类新方向的宽邻域路径跟踪内点算法

基于一类带有参数theta的新方向, 提出了求解单调线性互补问题的宽邻 域路径跟踪内点算法, 且当theta=1时即为经典牛顿方向. 当取theta为与问题规模 n无关的常数时, 算法具有O(nL)迭代复杂性, 其中L是输入数据的长度, 这与经典宽邻 域算法的复杂性相同; 当取theta=\sqrt{n/\beta\tau}时, 算法具有O(\sqrt{n}L)迭代复杂性, 这里的\beta, \tau是邻域参数, 这与窄邻域算法的复杂性相同. 这是首次研究包括经典宽邻域路径跟踪算法的一类内点算法, 给出了统一的算法框架和收敛性分析方法.     

On a class of interior point algorithms

A family of interior point algorithms for solving linear programs is examined. Under the assumption on the nondegeneracy of the problem, a theoretical justification of these algorithms is given. The sets of the algorithms converging to relatively interior optimal solutions and having linear or superlinear convergence rate are identified.     

A new class of hybrid extragradient algorithms for solving quasi-equilibrium problems

Generalized Nash equilibrium problems are important examples of quasi-equilibrium problems. The aim of this paper is to study a general class of algorithms for solving such problems. The method is a hybrid extragradient method whose second step consists in finding a descent direction for the distance function to the solution set. This is done thanks to a linesearch. Two descent directions are studied and for each one several steplengths are proposed to obtain the next iterate. A general convergence theorem applicable to each algorithm of the class is presented. It is obtained under weak assumptions: the pseudomonotonicity of the equilibrium function and the continuity of the multivalued mapping defining the constraint set of the quasi-equilibrium problem. Finally some preliminary numerical results are displayed to show the behavior of each algorithm of the class on generalized Nash equilibrium problems.     

Polynomial algorithms for a class of linear programs

This paper describes several algorithms for solution of linear programs. The algorithms are polynomial when the problem data satisfy certain conditions.     

Interior-point algorithms for P_{*}(\kappa )-LCP based on a new class of kernel functions

In this paper, we propose interior-point algorithms for $P_* (\kappa )$ -linear complementarity problem based on a new class of kernel functions. New search directions and proximity measures are defined based on these functions. We show that if a strictly feasible starting point is available, then the new algorithm has $\mathcal{O }\bigl ((1+2\kappa )\sqrt{n}\log n \log \frac{n\mu ^0}{\epsilon }\bigr )$ and $\mathcal{O }\bigl ((1+2\kappa )\sqrt{n} \log \frac{n\mu ^0}{\epsilon }\bigr )$ iteration complexity for large- and small-update methods, respectively. These are the best known complexity results for such methods.     

Some new analysis results for a class of interface problems

Interface problems modeled by differential equations have many applications in mathematical biology, fluid mechanics, material sciences, and many other areas. Typically, interface problems are characterized by discontinuities in the coefficients and/or the Dirac delta function singularities in the source term. Because of these irregularities, solutions to the differential equations are not smooth or discontinuous. In this paper, some new results on the jump conditions of the solution across the interface are derived using the distribution theory and the theory of weak solutions. Some theoretical results on the boundary singularity in which the singular delta function is at the boundary are obtained. Finally, the proof of the convergency of the immersed boundary (IB) method is presented. The IB method is shown to be first‐order convergent in L ^∞ norm. Copyright © 2013 John Wiley & Sons, Ltd.     

A unified analysis for a class of long-step primal-dual path-following interior-point algorithms for semidefinite programming

We present a unified analysis for a class of long-step primal-dual path-following algorithms for semidefinite programming whose search directions are obtained through linearization of the symmetrized equation of the central pathH _P (XS) [PXSP ^–1 + (PXSP ^–1) ^T ]/2 = I, introduced by Zhang. At an iterate (X,S), we choose a scaling matrixP from the class of nonsingular matricesP such thatPXSP ^–1 is symmetric. This class of matrices includes the three well-known choices, namely:P = S ^1/2 andP = X ^–1/2 proposed by Monteiro, and the matrixP corresponding to the Nesterov—Todd direction. We show that within the class of algorithms studied in this paper, the one based on the Nesterov—Todd direction has the lowest possible iteration-complexity bound that can provably be derived from our analysis. More specifically, its iteration-complexity bound is of the same order as that of the corresponding long-step primal-dual path-following algorithm for linear programming introduced by Kojima, Mizuno and Yoshise. © 1998 The Mathematical Programming Society, Inc. Published by Elsevier Science B.V.Corresponding author.This author's research is supported in part by the National Science Foundation under grants INT-9600343 and CCR-9700448 and the Office of Naval Research under grant N00014-94-1-0340.This author's research was supported in part by DOE DE-FG02-93ER25171-A001.     

On a class of algorithms from experimental design theory

The theory of optimal (approximate) linear regression design has produced several iterative methods to solve a special type of convex minimization problems. The present paper gives a unified and extended theoretical treatment of the methods. The emphasis is on the mathematical structures relevant for the optimization process, rather than on the statistical background of experimental design. So the main body of the paper can be read independently from the experimental design context. Applications are given to a special class of extremum problems arising in statistics. The numerical results obtained indicate that the methods are of practical interest     

Construction algorithms for a class of monotone variational inequalities

This paper is devoted to solve the following monotone variational inequality of finding \(x^*\in \mathrm{Fix}(T)\) such that

$$\begin{aligned} \langle Ax^*,x-x^*\rangle \ge 0,\quad \forall x\in \mathrm{Fix}(T), \end{aligned}$$

where A is a monotone operator and \(\mathrm{Fix}(T)\) is the set of fixed points of nonexpansive operator T. For this purpose, we construct an implicit algorithm and prove its convergence hierarchical to the solution of above monotone variational inequality.

     

Graph coloring in a class of parallel local algorithms

A way of improving the performance of a distributed algorithm is rendering a coloring strategy into an algorithm known as efficient in the nondistributed case. In this paper it is shown that certain sequential coloring algorithm heuristics like largest-first (LF), smallest-last (SL), and saturation largest-first (SLF), as applied to some classes of graphs and to special cases of vertex coloring in distributed algorithms, produce an optimal or near-optimal coloring.     

On the convergence of a class of derivative-free minimization algorithms

A convergence analysis is presented for a general class of derivative-free algorithms for minimizing a functionf(x) for which the analytic form of the gradient and the Hessian is impractical to obtain. The class of algorithms accepts finite-difference approximation to the gradient, with stepsizes chosen in such a way that the length of the stepsize must meet two conditions involving the previous stepsize and the distance from the last estimate of the solution to the current estimate. The algorithms also maintain an approximation to the second-derivative matrix and require that the change inx made at each iteration be subject to a bound that is also revised automatically. The convergence theorems have the features that the starting pointx ¹ need not be close to the true solution andf(x) need not be convex. Furthermore, despite the fact that the second-derivative approximation may not converge to the true Hessian at the solution, the rate of convergence is still Q-superlinear. The theorry is also shown to be applicable to a modification of Powell's dog-leg algorithm.     

Properties of the conjugate-gradient and Davidon methods

Two quadratically convergent gradient methods for minimizing an unconstrained function of several variables are examined. The heart of the Fletcher and Powell reformulation of Davidon's method is a variableH-matrix. The eigenvalues and eigenvectors of this matrix for a quadratic function are explored, leading to a proof that the gradient vectors at each step are mutually orthogonal. From this, a geometric interpretation of theH-matrix in terms of the projection of the gradient into a solution subspace is derived. These properties are then used to arrive at the main result, which states that, for a quadratic function, the direction vectors generated by the Davidon algorithm and the conjugate-gradient algorithm of Hestenes and Stiefel are scalar multiples of each other, provided the initial step each takes is in the direction of steepest descent. If one assumes no round-off error and a perfect one-dimensional search, the methods generate identical steps leading to the minimum.It is also shown that, for a quadratic function, the Davidon algorithm has a simplified version which searches in the same directions. However, the unique advantage of the original scheme, that it yields the curvature of the function at the minimum, is sacrificed for simplicity.Although these results apply to the case of a quadratic function, a comparative study of the same algorithms for a general function can be found in a companion paper.This research was carried out under Contract No. NAS 9-4036, NASA-Manned Spacecraft Center, Houston, Texas. The author is indebted to Dr. H. J. Kelley, whose suggestions and encouragement provided the inspiration for this paper.     

Simplex-inspired algorithms for solving a class of convex programming problems

We consider a class of convex programming problems whose objective function is given as a linear function plus a convex function whose arguments are linear functions of the decision variables and whose feasible region is a polytope. We show that there exists an optimal solution to this class of problems on a face of the constraint polytope of dimension not more than the number of arguments of the convex function. Based on this result, we develop a method to solve this problem that is inspired by the simplex method for linear programming. It is shown that this method terminates in a finite number of iterations in the special case that the convex function has only a single argument. We then use this insight to develop a second algorithm that solves the problem in a finite number of iterations for an arbitrary number of arguments in the convex function. A computational study illustrates the efficiency of the algorithm and suggests that the average-case performance of these algorithms is a polynomial of low order in the number of decision variables. The work of T. C. Sharkey was supported by a National Science Foundation Graduate Research Fellowship. The work of H. E. Romeijn was supported by the National Science Foundation under Grant No. DMI-0355533.