Curvature integrals and iteration complexities in SDP and symmetric cone programs

In this paper, we study iteration complexities of Mizuno-Todd-Ye predictor-corrector (MTY-PC) algorithms in SDP and symmetric cone programs by way of curvature integrals. The curvature integral is defined along the central path, reflecting the geometric structure of the central path. Integrating curvature along the central path, we obtain a precise estimate of the number of iterations to solve the problem. It has been shown for LP that the number of iterations is asymptotically precisely estimated with the integral divided by $\sqrt{\beta}$ , where β is the opening parameter of the neighborhood of the central path in MTY-PC algorithms. Furthermore, this estimate agrees quite well with the observed number of iterations of the algorithm even when β is close to one and when applied to solve large LP instances from NETLIB. The purpose of this paper is to develop direct extensions of these two results to SDP and symmetric cone programs. More specifically, we give concrete formulas for curvature integrals in SDP and symmetric cone programs and give asymptotic estimates for iteration complexities. Through numerical experiments with large SDP instances from SDPLIB, we demonstrate that the number of iterations is explained quite well with the integral even for a large step size which is enough to solve practical large problems.     

An iterative scheme for variational inequalities

In this paper we introduce and study a general iterative scheme for the numerical solution of finite dimensional variational inequalities. This iterative scheme not only contains, as special cases the projection, linear approximation and relaxation methods but also induces new algorithms. Then, we show that under appropriate assumptions the proposed iterative scheme converges by establishing contraction estimates involving a sequence of norms in Eⁿ induced by symmetric positive definite matrices G_m. Thus, in contrast to the above mentioned methods, this technique allows the possibility of adjusting the norm at each step of the algorithm. This flexibility will generally yield convergence under weaker assumptions.     

Smoothing Technique and its Applications in Semidefinite Optimization

In this paper we extend the smoothing technique (Nesterov in Math Program 103(1): 127–152, 2005; Nesterov in Unconstrained convex mimimization in relative scale, 2003) onto the problems of semidefinite optimization. For that, we develop a simple framework for estimating a Lipschitz constant for the gradient of some symmetric functions of eigenvalues of symmetric matrices. Using this technique, we can justify the Lipschitz constants for some natural approximations of maximal eigenvalue and the spectral radius of symmetric matrices. We analyze the efficiency of the special gradient-type schemes on the problems of minimizing the maximal eigenvalue or the spectral radius of the matrix, which depends linearly on the design variables. We show that in the first case the number of iterations of the method is bounded by \(O({1}/{\epsilon})\), where \(\epsilon\) is the required absolute accuracy of the problem. In the second case, the number of iterations is bounded by \({({4}/{\delta})} \sqrt{(1 + \delta) r\, \ln r }\), where δ is the required relative accuracy and r is the maximal rank of corresponding linear matrix inequality. Thus, the latter method is a fully polynomial approximation scheme.     

Estimation of the number of iterations in integer programming algorithms using the regular partitions method

We review the results of studying integer linear programming algorithms which exploit properties of problem relaxation sets. The main attention is paid to the estimation of the number of iterations of these algorithms by means of the regular partitions method and other approaches. We present such estimates for some cutting plane, branch and bound (Land and Doig scheme), and L-class enumeration algorithms and consider questions of their stability. We establish the upper bounds for the average number of iterations of the mentioned algorithms as applied to the knapsack problem and the set packing one.     

A UNIFIED FRAMEWORK FOR SOME INEXACT PROXIMAL POINT ALGORITHMS*

We present a unified framework for the design and convergence analysis of a class of algorithms based on approximate solution of proximal point subproblems. Our development further enhances the constructive approximation approach of the recently proposed hybrid projection–proximal and extragradient–proximal methods. Specifically, we introduce an even more flexible error tolerance criterion, as well as provide a unified view of these two algorithms. Our general method possesses global convergence and local (super)linear rate of convergence under standard assumptions, while using a constructive approximation criterion suitable for a number of specific implementations. For example, we show that close to a regular solution of a monotone system of semismooth equations, two Newton iterations are sufficient to solve the proximal subproblem within the required error tolerance. Such systems of equations arise naturally when reformulating the nonlinear complementarity problem.

     

Rate of Convergence Analysis of Discretization and Smoothing Algorithms for Semiinfinite Minimax Problems

Discretization algorithms for semiinfinite minimax problems replace the original problem, containing an infinite number of functions, by an approximation involving a finite number, and then solve the resulting approximate problem. The approximation gives rise to a discretization error, and suboptimal solution of the approximate problem gives rise to an optimization error. Accounting for both discretization and optimization errors, we determine the rate of convergence of discretization algorithms, as a computing budget tends to infinity. We find that the rate of convergence depends on the class of optimization algorithms used to solve the approximate problem as well as the policy for selecting discretization level and number of optimization iterations. We construct optimal policies that achieve the best possible rate of convergence and find that, under certain circumstances, the better rate is obtained by inexpensive gradient methods.     

A strongly polynomial FPTAS for the symmetric quadratic knapsack problem

The symmetric quadratic knapsack problem (SQKP), which has several applications in machine scheduling, is NP-hard. An approximation scheme for this problem is known to achieve an approximation ratio of (1 + ?) for any ? > 0. To ensure a polynomial time complexity, this approximation scheme needs an input of a lower bound and an upper bound on the optimal objective value, and requires the ratio of the bounds to be bounded by a polynomial in the size of the problem instance. However, such bounds are not mentioned in any previous literature. In this paper, we present the first such bounds and develop a polynomial time algorithm to compute them. The bounds are applied, so that we have obtained for problem (SQKP) a fully polynomial time approximation scheme (FPTAS) that is also strongly polynomial time, in the sense that the running time is bounded by a polynomial only in the number of integers in the problem instance.     

Subdivision of simplices relative to a cutting plane and finite concave minimization

In this article, our primary concern is the classical problem of minimizing globally a concave function over a compact polyhedron (Problem (P)). We present a new simplicial branch and bound approach, which combines triangulations of intersections of simplices with halfspaces and ideas from outer approximation in such a way, that a class of finite algorithms for solving (P) results. For arbitrary compact convex feasible sets one obtains a not necessarily finite but convergent algorithm. Theoretical investigations include determination of the number of simplices in each applied triangulation step and bounds on the number of iterations in the resulting algorithms. Preliminary numerical results are given, and additional applications are sketched.     

Lower Rank Approximation of Matrices Based on Fast and Robust Alternating Regression

We introduce fast and robust algorithms for lower rank approximation to given matrices based on robust alternating regression. The alternating least squares regression, also called criss-cross regression, was used for lower rank approximation of matrices, but it lacks robustness against outliers in these matrices. We use robust regression estimators and address some of the complications arising from this approach. We find it helpful to use high breakdown estimators in the initial iterations, followed by M estimators with monotone score functions in later iterations towards convergence. In addition to robustness, the computational speed is another important consideration in the development of our proposed algorithm, because alternating robust regression can be computationally intensive for large matrices. Based on a mix of the least trimmed squares (LTS) and Huber's M estimators, we demonstrate that fast and robust lower rank approximations are possible for modestly large matrices.     

Construction of multiwavelets with high approximation order and symmetry

In this paper, based on existing symmetric multiwavelets, we give an explicit algorithm for constructing multiwavelets with high approximation order and symmetry. Concretely, suppose Φ(x) := (φ1(x), . . . , φr(x))T is a symmetric refinable function vectors with approximation order m. For an arbitrary nonnegative integer n, a new symmetric refinable function vector Φnew(x) := (φn1 ew(x), . . . , φrn ew(x))T with approximation order m + n can be constructed through the algorithm mentioned above. Additionally,...     

A feasible direction algorithm for convex optimization: Global convergence rates

At each iteration, the algorithm determines a feasible descent direction by minimizing a linear or quadratic approximation to the cost on the feasible set. The algorithm is easy to implement if the approximation is easy to minimize on the feasible set, which happens in some important cases. Convergence rate information is obtained, which is sufficient to enable deduction of the number of iterations needed to achieve a specified reduction in the distance from the optimum (measured in terms of the cost). Existing convergence rates for algorithms for solving such convex problems are either asymptotic (and so do not enable the required number of iterations to be deduced) or decrease as the number of constraints increases. The convergence rate information obtained here, however, is independent of the number of constraints. For the case where the quadratic approximation to the cost is not strictly convex (which includes the linear approximation case), the diameter is the only property of the feasible set which affects the convergence rate information. If the quadratic approximation is strictly convex, the convergence rate is independent of the size and geometry of the feasible set. An application to a control-constrained optimal control problem is outlined.     

An asymptotically optimal Schur complement reduction for the Stokes equation

Summary. In this paper we develop an efficient Schur complement method for solving the 2D Stokes equation. As a basic algorithm, we apply a decomposition approach with respect to the trace of the pressure. The alternative stream function-vorticity reduction is also discussed. The original problem is reduced to solving the equivalent boundary (interface) equation with symmetric and positive definite operator in the appropriate trace space. We apply a mixed finite element approximation to the interface operator by iso triangular elements and prove the optimal error estimates in the presence of stabilizing bubble functions. The norm equivalences for the corresponding discrete operators are established. Then we propose an asymptotically optimal compression technique for the related stiffness matrix (in the absence of bubble functions) providing a sparse factorized approximation to the Schur complement. In this case, the algorithm is shown to have an optimal complexity of the order , q = 2 or q = 3, depending on the geometry, where N is the number of degrees of freedom on the interface. In the presence of bubble functions, our method has the complexity arithmetical operations. The Schur complement interface equation is resolved by the PCG iterations with an optimal preconditioner. Received March 20, 1996 / Revised version received October 28, 1997     

Circulant Embedding of Approximate Covariances for Inference From Gaussian Data on Large Lattices

Recently proposed computationally efficient Markov chain Monte Carlo (MCMC) and Monte Carlo expectation–maximization (EM) methods for estimating covariance parameters from lattice data rely on successive imputations of values on an embedding lattice that is at least two times larger in each dimension. These methods can be considered exact in some sense, but we demonstrate that using such a large number of imputed values leads to slowly converging Markov chains and EM algorithms. We propose instead the use of a discrete spectral approximation to allow for the implementation of these methods on smaller embedding lattices. While our methods are approximate, our examples indicate that the error introduced by this approximation is small compared to the Monte Carlo errors present in long Markov chains or many iterations of Monte Carlo EM algorithms. Our results are demonstrated in simulation studies, as well as in numerical studies that explore both increasing domain and fixed domain asymptotics. We compare the exact methods to our approximate methods on a large satellite dataset, and show that the approximate methods are also faster to compute, especially when the aliased spectral density is modeled directly. Supplementary materials for this article are available online.     

Multilevel approaches for FSAI preconditioning

Factorized sparse approximate inverse (FSAI) preconditioners are robust algorithms for symmetric positive matrices, which are particularly attractive in a parallel computational environment because of their inherent and almost perfect scalability. Their parallel degree is even redundant with respect to the actual capabilities of the current computational architectures. In this work, we present two new approaches for FSAI preconditioners with the aim of improving the algorithm effectiveness by adding some sequentiality to the native formulation. The first one, denoted as block tridiagonal FSAI, is based on a block tridiagonal factorization strategy, whereas the second one, domain decomposition FSAI, is built by reordering the matrix graph according to a multilevel k‐way partitioning method followed by a bandwidth minimization algorithm. We test these preconditioners by solving a set of symmetric positive definite problems arising from different engineering applications. The results are evaluated in terms of performance, scalability, and robustness, showing that both strategies lead to faster convergent schemes regarding the number of iterations and total computational time in comparison with the native FSAI with no significant loss in the algorithmic parallel degree.     

非奇异阵特征极值和条件数的近似计算

本文从共轭梯度法的公式推导出对称正定阵A与三对角阵B的相似关系,B的元素由共轭梯度法的迭代参数确定.因此,对称正定阵的条件数计算可以化成三对角阵条件数的计算,并且可以在共轭梯度法的计算中顺带完成.它只需增加O(s)次的计算量,s为迭代次数.这与共轭梯度法的计算量相比是可以忽略的.当A为非对称正定阵时,只要A非奇异,即可用共轭梯度法计算ATA的特征极值和条件数,从而得出A的条件数.对不同算例的计算表明,这是一种快速有效的简便方法.     

Eigenvalues and switching algorithms for Quasi-Newton updates

Two switching algorithms QNSWl and QNSW2 are proposed in this paper. These algorithms are developed based on the eigenvalues of matrices which are inertial to the symmetric rank-one (SR1) updates and the BFGS updates. First, theoretical results on the eigenvalues and condition numbers of these matrices are presented. Second, switch-ing mechanisms are then developed based on theoretical results obtained so that each proposed algorithm has the capability of applying appropriate updating formulae at each iterative point during the whole minimization process. Third, the performance of

each of the proposed algorithms is evaluated over a wide range of test problems with variable dimensions. These results are then compared to the results obtained by some well-known minimization packages. Comparative results show that among the tested methods, the QNSW2 algorithm has the best overall performance for the problems examined. In some cases, the number of iterations and the number function/gradient calls required by certain existing methods are more than a four-fold increase over that required by the proposed switching algorithms     

Lagrange插值在—重积分Wiener空间下的同时逼近平均误差

在加权L_p范数逼近意义下,确定了基于扩充的第二类Chebyshev结点组的Lagrange插值多项式列,在一重积分Wiener空间下同时逼近平均误差的渐近阶.结果显示,在L_p范数逼近意义下,Lagrange插值多项式列逼近函数及其导数的平均误差都弱等价于相应的最佳逼近多项式列的平均误差.同时,在信息基复杂性的意义下,若可允许信息泛函为标准信息,则上述插值算子列逼近函数及其导数的平均误差均弱等价于相应的最小非自适应信息半径.     

Characterizations of the Optimal Descartes' Rules of Signs

A system of functions satisfies Descartes' rule of signs if the number of zeros (with multiplicities) of a linear combination of these functions is less than or equal to the number of variations of strict sign in the sequence of the coefficients. In this paper we characterize the systems of functions satisfying a stronger property than the above mentioned Descartes' rule: The difference between the number of zeros and the changes of sign in the sequence of coefficients must be always a nonnegative even number. We show that the approximation to the number of zeros given by these systems of functions is better than the approximation provided by any other systems of functions satisfying a Descartes' rule of signs. This last result improves, in the particular case of polynomials, the main theorem of [14].     

A wavelet-based nested iteration–inexact conjugate gradient algorithm for adaptively solving elliptic PDEs

In Cohen et al. (Math Comput 70:27–75, 2001), a new paradigm for the adaptive solution of linear elliptic partial differential equations (PDEs) was proposed, based on wavelet discretizations. Starting from a well-conditioned representation of the linear operator equation in infinite wavelet coordinates, one performs perturbed gradient iterations involving approximate matrix–vector multiplications of finite portions of the operator. In a bootstrap-type fashion, increasingly smaller tolerances guarantee convergence of the adaptive method. In addition, coarsening performed on the iterates allow one to prove asymptotically optimal complexity results when compared to the wavelet best N-term approximation. In the present paper, we study adaptive wavelet schemes for symmetric operators employing inexact conjugate gradient routines. Inspired by fast schemes on uniform grids, we incorporate coarsening and the adaptive application of the elliptic operator into a nested iteration algorithm. Our numerical results demonstrate that the runtime of the algorithm is linear in the number of unknowns and substantial savings in memory can be achieved in two and three space dimensions.