QZ algorithm with two-sided generalized Rayleigh quotient shifts

We generalize the recently proposed two-sided Rayleigh quotient single-shift and the two-sided Grassmann–Rayleigh quotient double-shift used in the QR algorithm and apply the generalized versions to the QZ algorithm. With such shift strategies the QZ algorithm normally has a cubic local convergence rate. Our main focus is on the modified shift strategies and their corresponding truncated versions. Numerical examples are provided to demonstrate the convergence properties and the efficiency of the QZ algorithm equipped with the proposed shifts. For the truncated versions, local convergence analysis is not provided. Numerical examples show they outperform the modified shifts and the standard Rayleigh quotient single-shift and Francis double-shift.     

A new,globally convergent Riemannian conjugate gradient method

This article deals with the conjugate gradient method on a Riemannian manifold with interest in global convergence analysis. The existing conjugate gradient algorithms on a manifold endowed with a vector transport need the assumption that the vector transport does not increase the norm of tangent vectors, in order to confirm that generated sequences have a global convergence property. In this article, the notion of a scaled vector transport is introduced to improve the algorithm so that the generated sequences may have a global convergence property under a relaxed assumption. In the proposed algorithm, the transported vector is rescaled in case its norm has increased during the transport. The global convergence is theoretically proved and numerically observed with examples. In fact, numerical experiments show that there exist minimization problems for which the existing algorithm generates divergent sequences, but the proposed algorithm generates convergent sequences.     

A Quasi-Newton Penalty Barrier Method for Convex Minimization Problems

We describe an infeasible interior point algorithm for convex minimization problems. The method uses quasi-Newton techniques for approximating the second derivatives and providing superlinear convergence. We propose a new feasibility control of the iterates by introducing shift variables and by penalizing them in the barrier problem. We prove global convergence under standard conditions on the problem data, without any assumption on the behavior of the algorithm.     

A multishift QR iteration without computation of the shifts

Each iteration of the multishift QR algorithm of Bai and Demmel requires the computation of a shift vector defined bym shifts of the origin of the spectrum that control the convergence of the process. A common choice of shifts consists of the eigenvalues of the trailing principal submatrix of orderm, and current practice includes the computation of these eigenvalues in the determination of the shift vector. In this paper, we describe an algorithm based on the evaluation of the characteristic polynomial of a Hessenberg matrix, which directly produces the shift vector without computing eigenvalues. This algorithm is stable, more accurate, faster, and simpler than the current alternative. It also allows for a consistent shift strategy with dynamic adjustment of the number of shifts.The work of this author was in part supported by the National Science Foundation, grant number ESC-9003107, and the Army Research Office, grant number DAAL-03-91-G-0038.     

A partition-based global optimization algorithm

This paper is devoted to the study of partition-based deterministic algorithms for global optimization of Lipschitz-continuous functions without requiring knowledge of the Lipschitz constant. First we introduce a general scheme of a partition-based algorithm. Then, we focus on the selection strategy in such a way to exploit the information on the objective function. We propose two strategies. The first one is based on the knowledge of the global optimum value of the objective function. In this case the selection strategy is able to guarantee convergence of every infinite sequence of trial points to global minimum points. The second one does not require any a priori knowledge on the objective function and tries to exploit information on the objective function gathered during progress of the algorithm. In this case, from a theoretical point of view, we can guarantee the so-called every-where dense convergence of the algorithm.     

Global optimization of fractional programs

Dinkelbach's global optimization approach for finding the global maximum of the fractional programming problem is discussed. Based on this idea, a modified algorithm is presented which provides both upper and lower bounds at each iteration. The convergence of the lower and upper bounds to the global maximum function value is shown to be superlinear. In addition, the special case of fractional programming when the ratio involves only linear or quadratic terms is considered. In this case, the algorithm is guaranteed to find the global maximum to within any specified tolerance, regardless of the definiteness of the quadratic form.     

A CLASS OF TRUST REGION METHODS FOR LINEAR INEQUALITY CONSTRAINED OPTIMIZATION AND ITS THEORY ANALYSIS：I．ALGORITHM AND GLOBAL CONVERGENCE

ＡＣＬＡＳＳＯＦＴＲＵＳＴＲＥＧＩＯＮＭＥＴＨＯＤＳＦＯＲＬＩＮＥＡＲＩＮＥＱＵＡＬＩＴＹＣＯＮＳＴＲＡＩＮＥＤＯＰＴＩＭＩＺＡＴＩＯＮＡＮＤＩＴＳＴＨＥＯＲＹＡＮＡＬＹＳＩＳ：Ｉ．ＡＬＧＯＲＩＴＨＭＡＮＤＧＬＯＢＡＬＣＯＮＶＥＲＧＥＮＣＥＸＩＵＮＡ...     

A CLASS OF TRUST REGION METHODS FOR LINEAR INEQUALITY CONSTRAINED OPTIMIZATION AND ITS THEORY ANALYSIS：Ⅰ. ALGORITHM AND GLOBAL CONVERGENCE

A class of trust region methods tor solving linear inequality constrained problems is propo6ed in this paper. It is shown that the algorithm is of global convergence. The algorithm uses a version of the two-slded projection and the strategy of the unconstrained trust region methods. It keeps the good convergence properties of the unconstrained case and has the merits of the projection method. In some sense, our algorithm can be regarded as an extension and improvement of the projected type algorithm.     

Convergence Analysis of the Inexact Infeasible Interior-Point Method for Linear Optimization

We present the convergence analysis of the inexact infeasible path-following (IIPF) interior-point algorithm. In this algorithm, the preconditioned conjugate gradient method is used to solve the reduced KKT system (the augmented system). The augmented system is preconditioned by using a block triangular matrix. The KKT system is solved approximately. Therefore, it becomes necessary to study the convergence of the interior-point method for this specific inexact case. We present the convergence analysis of the inexact infeasible path-following (IIPF) algorithm, prove the global convergence of this method and provide complexity analysis. Communicated by Y. Zhang.     

The classic differential evolution algorithm and its convergence properties

Differential evolution algorithms represent an up to date and efficient way of solving complicated optimization tasks. In this article we concentrate on the ability of the differential evolution algorithms to attain the global minimum of the cost function. We demonstrate that although often declared as a global optimizer the classic differential evolution algorithm does not in general guarantee the convergence to the global minimum. To improve this weakness we design a simple modification of the classic differential evolution algorithm. This modification limits the possible premature convergence to local minima and ensures the asymptotic global convergence. We also introduce concepts that are necessary for the subsequent proof of the asymptotic global convergence of the modified algorithm. We test the classic and modified algorithm by numerical experiments and compare the efficiency of finding the global minimum for both algorithms. The tests confirm that the modified algorithm is significantly more efficient with respect to the global convergence than the classic algorithm.     

Convergence of the shifted algorithm for unitary Hessenberg matrices

This paper shows that for unitary Hessenberg matrices the algorithm, with (an exceptional initial-value modification of) the Wilkinson shift, gives global convergence; moreover, the asymptotic rate of convergence is at least cubic, higher than that which can be shown to be quadratic only for Hermitian tridiagonal matrices, under no further assumption. A general mixed shift strategy with global convergence and cubic rates is also presented.

     

Long-Step Primal Path-Following Algorithm for Monotone Variational Inequality Problems

In this paper, we present a long-step primal path-following algorithm and prove its global convergence under usual assumptions. It is seen that the short-step algorithm is a special case of the long-step algorithm for a specific selection of the parameters and the initial solution. Our theoretical result indicates that the long-step algorithm is more flexible. Numerical results indicate that the long-step algorithm converges faster than the short-step algorithm.     

A Nonmonotone Trust Region Method for Nonlinear Programming with Simple Bound Constraints

In this paper we propose a nonmonotone trust region algorithm for optimization with simple bound constraints. Under mild conditions, we prove the global convergence of the algorithm. For the monotone case it is also proved that the correct active set can be identified in a finite number of iterations if the strict complementarity slackness condition holds, and so the proposed algorithm reduces finally to an unconstrained minimization method in a finite number of iterations, allowing a fast asymptotic rate of convergence. Numerical experiments show that the method is efficient. Accepted 5 September 2000. Online publication 4 December 2000.     

Exactness and algorithm of an objective penalty function

Penalty function is an important tool in solving many constrained optimization problems in areas such as industrial design and management. In this paper, we study exactness and algorithm of an objective penalty function for inequality constrained optimization. In terms of exactness, this objective penalty function is at least as good as traditional exact penalty functions. Especially, in the case of a global solution, the exactness of the proposed objective penalty function shows a significant advantage. The sufficient and necessary stability condition used to determine whether the objective penalty function is exact for a global solution is proved. Based on the objective penalty function, an algorithm is developed for finding a global solution to an inequality constrained optimization problem and its global convergence is also proved under some conditions. Furthermore, the sufficient and necessary calmness condition on the exactness of the objective penalty function is proved for a local solution. An algorithm is presented in the paper in finding a local solution, with its convergence proved under some conditions. Finally, numerical experiments show that a satisfactory approximate optimal solution can be obtained by the proposed algorithm.     

An Extended Sequential Quadratically Constrained Quadratic Programming Algorithm for Nonlinear, Semidefinite, and Second-Order Cone Programming

This paper is concerned with nonlinear, semidefinite, and second-order cone programs. A general algorithm, which includes sequential quadratic programming and sequential quadratically constrained quadratic programming methods, is presented for solving these problems. In the particular case of standard nonlinear programs, the algorithm can be interpreted as a prox-regularization of the Solodov sequential quadratically constrained quadratic programming method presented in Mathematics of Operations Research (2004). For such type of methods, the main cost of computation amounts to solve a linear cone program for which efficient solvers are available. Usually, “global convergence results” for these methods require, as for the Solodov method, the boundedness of the primal sequence generated by the algorithm. The other purpose of this paper is to establish global convergence results without boundedness assumptions on any of the iterative sequences built by the algorithm.     

Convergence guarantees for generalized adaptive stochastic search methods for continuous global optimization

This paper presents some simple technical conditions that guarantee the convergence of a general class of adaptive stochastic global optimization algorithms. By imposing some conditions on the probability distributions that generate the iterates, these stochastic algorithms can be shown to converge to the global optimum in a probabilistic sense. These results also apply to global optimization algorithms that combine local and global stochastic search strategies and also those algorithms that combine deterministic and stochastic search strategies. This makes the results applicable to a wide range of global optimization algorithms that are useful in practice. Moreover, this paper provides convergence conditions involving the conditional densities of the random vector iterates that are easy to verify in practice. It also provides some convergence conditions in the special case when the iterates are generated by elliptical distributions such as the multivariate Normal and Cauchy distributions. These results are then used to prove the convergence of some practical stochastic global optimization algorithms, including an evolutionary programming algorithm. In addition, this paper introduces the notion of a stochastic algorithm being probabilistically dense in the domain of the function and shows that, under simple assumptions, this is equivalent to seeing any point in the domain with probability 1. This, in turn, is equivalent to almost sure convergence to the global minimum. Finally, some simple results on convergence rates are also proved.     

An extragradient method for equilibrium problems on Hadamard manifolds

In this paper, we propose an extragradient algorithm for solving equilibrium problems on Hadamard manifolds to the case where the equilibrium bifunction is not necessarily pseudomonotone. Under mild assumptions, we establish global convergence results. We show that the multiobjective optimization problem satisfies all the hypotheses of our result of convergence, when formulated as an equilibrium problem.     

Extremal Eigenvalues of the Sturm-Liouville Problems with Discontinuous Coefficients

In this paper, an extremal eigenvalue problem to the Sturm-Liouville equations with discontinuous coefficients and volume constraint is investigated. Liouville transformation is applied to change the problem into an equivalent minimization problem. Finite element method is proposed and the convergence for the finite element solution is established. A monotonic decreasing algorithm is presented to solve the extremal eigenvalue problem. A global convergence for the algorithm in the continuous case is proved. A few numerical results are given to depict the efficiency of the method.     

Nonmonotone Levenberg–Marquardt Algorithms and Their Convergence Analysis

In this paper, two nonmonotone Levenberg–Marquardt algorithms for unconstrained nonlinear least-square problems with zero or small residual are presented. These algorithms allow the sequence of objective function values to be nonmonotone, which accelerates the iteration progress, especially in the case where the objective function is ill-conditioned. Some global convergence properties of the proposed algorithms are proved under mild conditions which exclude the requirement for the positive definiteness of the approximate Hessian T(x). Some stronger global convergence properties and the local superlinear convergence of the first algorithm are also proved. Finally, a set of numerical results is reported which shows that the proposed algorithms are promising and superior to the monotone Levenberg–Marquardt algorithm according to the numbers of gradient and function evaluations.