An ODE-based trust region method for unconstrained optimization problems

In this paper, a new trust region algorithm is proposed for solving unconstrained optimization problems. This method can be regarded as a combination of trust region technique, fixed step-length and ODE-based methods. A feature of this proposed method is that at each iteration, only a system of linear equations is solved to obtain a trial step. Another is that when a trial step is not accepted, the method generates an iterative point whose step-length is defined by a formula. Under some standard assumptions, it is proven that the algorithm is globally convergent and locally superlinear convergent. Preliminary numerical results are reported.     

A reflection on the implicitly restarted Arnoldi method for computing eigenvalues near a vertical line

In this article, we will study the link between a method for computing eigenvalues closest to the imaginary axis and the implicitly restarted Arnoldi method. The extension to computing eigenvalues closest to a vertical line is straightforward, by incorporating a shift. Without loss of generality we will restrict ourselves here to computing eigenvalues closest to the imaginary axis.In a recent publication, Meerbergen and Spence discussed a new approach for detecting purely imaginary eigenvalues corresponding to Hopf bifurcations, which is of interest for the stability of dynamical systems. The novel method is based on inverse iteration (inverse power method) applied on a Lyapunov-like eigenvalue problem. To reduce the computational overhead significantly a projection was added.This method can also be used for computing eigenvalues of a matrix pencil near a vertical line in the complex plane. We will prove in this paper that the combination of inverse iteration with the projection step is equivalent to Sorensen’s implicitly restarted Arnoldi method utilizing well-chosen shifts.     

A hybrid ODE-based method for unconstrained optimization problems

This paper presents a hybrid ODE-based method for unconstrained optimization problems, which combines the idea of IMPBOT with the subspace technique and a fixed step-length. The main characteristic of this method is that at each iteration, a lower dimensional system of linear equations is solved only once to obtain a trial step. Another is that when a trial step is not accepted, this proposed method uses minimization of a convex overestimation, thus avoiding performing a line search to compute a step-length. Under some reasonable assumptions, the method is proven to be globally convergent. Numerical results show the efficiency of this proposed method in practical computations, especially for solving small scale unconstrained optimization problems.     

A block Arnoldi-Chebyshev method for computing the leading eigenpairs of large sparse unsymmetric matrices

Summary In this paper we describe a block version of Arnoldi's method for computing a few eigenvalues with largest or smallest real parts. The method is accelerated via Chebyshev iteration and a procedure is developed to identify the optimal ellipse which encloses the spectrum. A parallel implementation of this method is investigated on the eight processor Alliant FX/80. Numerical results and comparisons with simultaneous iteration on some Harwell-Boeing matrices are reported.     

A new supermemory gradient method for unconstrained optimization problems

This paper presents a new supermemory gradient method for unconstrained optimization problems. It can be regarded as a combination of ODE-based methods, line search and subspace techniques. The main characteristic of this method is that, at each iteration, a lower dimensional system of linear equations is solved only once to obtain a trial step, thus avoiding solving a quadratic trust region subproblem. Another is that when a trial step is not accepted, this proposed method generates an iterative point whose step-length satisfies Armijo line search rule, thus avoiding resolving linear system of equations. Under some reasonable assumptions, the method is proven to be globally convergent. Numerical results show the efficiency of this proposed method in practical computation.     

Generalized consistent orderings and the Accelerated Overrelaxation method

In this paper, the behavior of the block Accelerated Overrelaxation (AOR) iterative method, when applied to linear systems with a generalized consistently ordered coefficient matrix, is investigated. An equation, relating the eigenvalues of the block Jacobi iteration matrix to the eigenvalues of its associated block AOR iteration matrix, as well as sufficient conditions for the convergence of the block AOR method, are obtained.     

大型不正定陀螺系统本征值问题

陀螺动力系统可以导入哈密顿辛几何体系,在哈密顿陀螺系统的辛子空间迭代法的基础上提出了一种能够有效计算大型不正定哈密顿函数的陀螺系统本征值问题的算法．利用陀螺矩阵既为哈密顿矩阵而本征值又是纯虚数或零的特点,将对应哈密顿函数为负的本征值分离开来,构造出对应哈密顿函数全为正的本征值问题,利用陀螺系统的辛子空间迭代法计算出正定哈密顿矩阵的本征值,从而解决了大型不正定陀螺系统的本征值问题,算例证明,本征解收敛得很快．     

Broyden method for inverse non-symmetric Sturm-Liouville problems

In this paper, we propose a derivative-free method for recovering symmetric and non-symmetric potential functions of inverse Sturm-Liouville problems from the knowledge of eigenvalues. A class of boundary value methods obtained as an extension of Numerov’s method is the major tool for approximating the eigenvalues in each Broyden iteration step. Numerical examples demonstrate that the method is able to reduce the number of iteration steps, in particular for non-symmetric potentials, without accuracy loss.     

A PROCESS FOR SOLVING A FEW EXTREME EIGENPAIRS OF LARGE SPARSE POSITIVE DEFINITE GENERALIZED EIGENVALUE PROBLEM

1. IntroductionWe are concerned in this work with finding a few extreme eigenvalues and theircorresponding eigenvectors of a generalized large scale eigenvalue problem in which thematrices are sparse and symmetric positive definite.Although finding a few extreme eigenpairs is of interest both in theory and practice,there are only few usable and efficient methods up to now. Reinsch and Baner ([12]),suggested a oR algorithm with Newton shift for the standard eigenproblem which included an ingen…     

一种新的无约束优化线搜索算法

在对各种有效的线搜索算法分析的基础上,给出了一种求解光滑无约束优化问题的新的线搜索算法.对于目标函数是二次连续可微且下有界的无约束优化问题,算法具有与Wolfe-Powell线搜索算法相同的理论性质.在每一步迭代中算法至多需要计算两次梯度,对于计算目标函数梯度花费较大的情形可以节省一定的计算量.数值试验表明本文算法是可行的和有效的.     

Preconditioned AHSS iteration method for singular saddle point problems

In this paper, for solving the singular saddle point problems, we present a new preconditioned accelerated Hermitian and skew-Hermitian splitting (AHSS) iteration method. The semi-convergence of this method and the eigenvalue distribution of the preconditioned iteration matrix are studied. In addition, we prove that all eigenvalues of the iteration matrix are clustered for any positive iteration parameters α and β. Numerical experiments illustrate the theoretical results and examine the numerical effectiveness of the AHSS iteration method served either as a preconditioner or as a solver.     

Sparse Steinian Covariance Estimation

We consider a new method for sparse covariance matrix estimation which is motivated by previous results for the so-called Stein-type estimators. Stein proposed a method for regularizing the sample covariance matrix by shrinking together the eigenvalues; the amount of shrinkage is chosen to minimize an unbiased estimate of the risk (UBEOR) under the entropy loss function. The resulting estimator has been shown in simulations to yield significant risk reductions over the maximum likelihood estimator. Our method extends the UBEOR minimization problem by adding an ?₁ penalty on the entries of the estimated covariance matrix, which encourages a sparse estimate. For a multivariate Gaussian distribution, zeros in the covariance matrix correspond to marginal independences between variables. Unlike the ?₁-penalized Gaussian likelihood function, our penalized UBEOR objective is convex and can be minimized via a simple block coordinate descent procedure. We demonstrate via numerical simulations and an analysis of microarray data from breast cancer patients that our proposed method generally outperforms other methods for sparse covariance matrix estimation and can be computed efficiently even in high dimensions.     

连续时间LQ控制主要本征对的算法

本文首先提出了离散时间LQ控制的本征值方程当△t→0时怎样退化成为连续时间LQ控制的本征值方程.在建立了分离出的n阶连续时间的本征值方程,并保证了其本征值必定都在左半平面后,本文提出计算其最靠近于虚轴的若干个本征对,可以通过A_e=eA的矩阵变换.A_e的本征值全在单位圆之内.本征向量不变,至于本征值则只要做一次对数运算就可以求得原阵的本征值.A_e阵的最接近于单位圆的若干个本征对的算法,可以通过共轭子空间迭代解解决之.     

A criterion for the convergence of the Gauss-Seidel method

The solution of linear equations by iterative methods requires for convergence that the absolute magnitudes of all the eigenvalues of the iteration matrix should be less than unity. The test for convergence however is often difficult to apply because of the computation required. In this paper a method for determining the convergence of the Gauss-Seidel iteration is proposed. The method involves the numerical integration of initial value differential equations in the complex plane around the unit circle. The Gauss-Seidel method converges if the number of roots inside the unit circle is equal to the order of the iteration matrix.     

An implicit QR algorithm for symmetric semiseparable matrices

The QR algorithm is one of the classical methods to compute the eigendecomposition of a matrix. If it is applied on a dense n × n matrix, this algorithm requires O(n³) operations per iteration step. To reduce this complexity for a symmetric matrix to O(n), the original matrix is first reduced to tridiagonal form using orthogonal similarity transformations. In the report (Report TW360, May 2003) a reduction from a symmetric matrix into a similar semiseparable one is described. In this paper a QR algorithm to compute the eigenvalues of semiseparable matrices is designed where each iteration step requires O(n) operations. Hence, combined with the reduction to semiseparable form, the eigenvalues of symmetric matrices can be computed via intermediate semiseparable matrices, instead of tridiagonal ones. The eigenvectors of the intermediate semiseparable matrix will be computed by applying inverse iteration to this matrix. This will be achieved by using an O(n) system solver, for semiseparable matrices. A combination of the previous steps leads to an algorithm for computing the eigenvalue decompositions of semiseparable matrices. Combined with the reduction of a symmetric matrix towards semiseparable form, this algorithm can also be used to calculate the eigenvalue decomposition of symmetric matrices. The presented algorithm has the same order of complexity as the tridiagonal approach, but has larger lower order terms. Numerical experiments illustrate the complexity and the numerical accuracy of the proposed method. Copyright © 2005 John Wiley & Sons, Ltd.     

A MODIFIED INVERSE ITERATION FOR A LARGE SPARSE SPD GENERALIZED EIGENPROBLEM

In this paper, an algorithm based on a shifted inverse power iteration for computing generalized eigenvalues with corresponding eigenvectors of a large scale sparse symmetric positive definite matrix pencil is presented. It converges globally with a cubic asymptotic convergence rate, preserves sparsity of the original matrices and is fully parallelizable. The algebraic multilevel itera-tion method (AMLI) is used to improve the efficiency when symmetric positive definite linear equa-tions need to be solved.     

Solution of finite systems of equations by interval iteration

In actual practice, iteration methods applied to the solution of finite systems of equations yield inconclusive results as to the existence or nonexistence of solutions and the accuracy of any approximate solutions obtained. On the other hand, construction of interval extensions of ordinary iteration operators permits one to carry out interval iteration computationally, with results which can give rigorous guarantees of existence or nonexistence of solutions, and error bounds for approximate solutions. Examples are given of the solution of a nonlinear system of equations and the calculation of eigenvalues and eigenvectors of a matrix by interval iteration. Several ways to obtain lower and upper bounds for eigenvalues are given.Sponsored by the United States Army under Contract No. DAAG29-80-C-0041.     

An adaptive block Lanczos algorithm

A generalization of the block Lanczos algorithm will be given, which allows the block size to be increased during the iteration process. In particular, the algorithm can be implemented with the block size chosen adaptively according to clustering of Ritz values. In this way, multiple and clustered eigenvalues can be found and the difficulty of choosing the block size is eased. Residual bounds for clustered eigenvalues are given. Numerical examples are presented to illustrate the adaptive algorithm.Research supported by a grant from Natural Sciences and Engineering Research Council of Canada.     

Eigenvalues of Hermitian Toeplitz matrices with smooth simple-loop symbols

The paper presents higher-order asymptotic formulas for the eigenvalues of large Hermitian Toeplitz matrices with moderately smooth symbols which trace out a simple loop on the real line. The formulas are established not only for the extreme eigenvalues, but also for the inner eigenvalues. The results extend and make more precise existing results, which so far pertain to banded matrices or to matrices with infinitely differentiable symbols. Also given is a fixed-point equation for the eigenvalues which may be solved numerically by an iteration method.