Extensions of CGS algorithms: Generalized least-square solutions

The CGS (conjugate Gram-Schmidt) algorithms of Hestenes and Stiefel are formulated so as to obtain least-square solutions of a system of equationsg(x)=0 inn independent variables. Both the linear caseg(x)=Ax–h and the nonlinear case are discussed. In the linear case, a least-square solution is obtained in no more thann steps, and a method of obtaining the least-square solution of minimum length is given. In the nonlinear case, the CGS algorithm is combined with the Gauss-Newton process to minimize sums of squares of nonlinear functions. Results of numerical experiments with several versions of CGS on test functions indicate that the algorithms are effective.The author wishes to express appreciation and to acknowledge the ideas and help of Professor M. R. Hestenes which made this paper possible.     

Convergence Rate of the Augmented Lagrangian SQP Method

In this paper, the augmented Lagrangian SQP method is considered for the numerical solution of optimization problems with equality constraints. The problem is formulated in a Hilbert space setting. Since the augmented Lagrangian SQP method is a type of Newton method for the nonlinear system of necessary optimality conditions, it is conceivable that q-quadratic convergence can be shown to hold locally in the pair (x, ). Our interest lies in the convergence of the variable x alone. We improve convergence estimates for the Newton multiplier update which does not satisfy the same convergence properties in x as for example the least-square multiplier update. We discuss these updates in the context of parameter identification problems. Furthermore, we extend the convergence results to inexact augmented Lagrangian methods. Numerical results for a control problem are also presented.     

Analytical Distribution of Waiting Time in the M/{iD}/1 Queue

We give an analytical formula for the steady-state distribution of queue-wait in the M/G/1 queue, where the service time for each customer is a positive integer multiple of a constant D > 0. We call this an M/{iD}/1 queue. We give numerical algorithms to calculate the distribution. In addition, in the case that the service distribution is sparse, we give revised algorithms that can compute the distribution more quickly.AMS subject classification: 60K25, 90B22     

Matrix conditioning and nonlinear optimization

In a series of recent papers, Oren, Oren and Luenberger, Oren and Spedicato, and Spedicato have developed the self-scaling variable metric algorithms. These algorithms alter Broyden's single parameter family of approximations to the inverse Hessian to a double parameter family. Conditions are given on the new parameter to minimize a bound on the condition number of the approximated inverse Hessian while insuring improved step-wise convergence.Davidon has devised an update which also minimizes the bound on the condition number while remaining in the Broyden single parameter family.This paper derives initial scalings for the approximate inverse Hessian which makes members of the Broyden class self-scaling. The Davidon, BFGS, and Oren—Spedicato updates are tested for computational efficiency and stability on numerous test functions, with the results indicating strong superiority computationally for the Davidon and BFGS update over the self-scaling update, except on a special class of functions, the homogeneous functions.     

Convergence of cascade algorithms in Sobolev spaces and integrals of wavelets

Summary. The cascade algorithm with mask a and dilation M generates a sequence by the iterative process from a starting function where M is a dilation matrix. A complete characterization is given for the strong convergence of cascade algorithms in Sobolev spaces for the case in which M is isotropic. The results on the convergence of cascade algorithms are used to deduce simple conditions for the computation of integrals of products of derivatives of refinable functions and wavelets. Received May 5, 1999 / Revised version received June 24, 1999 / Published online June 20, 2001     

A characterization of the minimizer of a function

This paper is intended to give a characterization of the minimum point of a function in terms of the gradient of the function at some other point using some concepts from differential geometry. The function is assumed to have continuous partial derivatives up to and including order four. It is also assumed to have a positive-definite Hessian matrix onR ⁿ and a unique minimum point.     

Refined Error Estimates for Radial Basis Function Interpolation

We discuss new and refined error estimates for radial-function scattered-data interpolants and their derivatives. These estimates hold on R ^d , the d-torus, and the 2-sphere. We employ a new technique, involving norming sets, that enables us to obtain error estimates, which in many cases give bounds orders of magnitude smaller than those previously known.     

Functions of small growth with no unbounded Fatou components

We prove a form of the cos πρ theorem which gives strong estimates for the minimum modulus of a transcendental entire function of order zero. We also prove a generalisation of a result of Hinkkanen that gives a sufficient condition for a transcendental entire function to have no unbounded Fatou components. These two results enable us to show that there is a large class of entire functions of order zero which have no unbounded Fatou components. On the other hand, we give examples which show that there are in fact functions of order zero which not only fail to satisfy Hinkkanen’s condition but also fail to satisfy our more general condition. We also give a new regularity condition that is sufficient to ensure that a transcendental entire function of order less than 1/2 has no unbounded Fatou components. Finally, we observe that all the conditions given here which guarantee that a transcendental entire function has no unbounded Fatou components also guarantee that the escaping set is connected, thus answering a question of Eremenko for such functions.     

Estimates for the curvature of a submanifold which lies inside a tube

We give some estimates for the supremun of the sectional curvature of a submanifold N of a Riemannian or Kaehlerian manifoldM such that N is contained in a tube about a submanifoldP ofM. These estimates depend on the sectional curvature ofM, the Weingarten map ofP and the radius of the tube. Then we apply them to get theorems of non immersibility.Work partially supported by a DGICYT NO. PB94-0972     

Regular type of real hyper-surfaces in (almost) complex manifolds

The regular type of a real hyper-surface M in an (almost) complex manifold at some point p is the maximal contact order at p of M with germs of non singular (pseudo) holomorphic disks. The main purpose of this paper is to give two intrinsic characterizations the type : one in terms of Lie brackets of a complex tangent vector field on M, the other in terms of some kind of derivatives of the Levi form.Mathematics Subject Classification (2000): 32T25,32Q60     

Distributed multi-agent optimization with state-dependent communication

We study distributed algorithms for solving global optimization problems in which the objective function is the sum of local objective functions of agents and the constraint set is given by the intersection of local constraint sets of agents. We assume that each agent knows only his own local objective function and constraint set, and exchanges information with the other agents over a randomly varying network topology to update his information state. We assume a state-dependent communication model over this topology: communication is Markovian with respect to the states of the agents and the probability with which the links are available depends on the states of the agents. We study a projected multi-agent subgradient algorithm under state-dependent communication. The state-dependence of the communication introduces significant challenges and couples the study of information exchange with the analysis of subgradient steps and projection errors. We first show that the multi-agent subgradient algorithm when used with a constant stepsize may result in the agent estimates to diverge with probability one. Under some assumptions on the stepsize sequence, we provide convergence rate bounds on a “disagreement metric” between the agent estimates. Our bounds are time-nonhomogeneous in the sense that they depend on the initial starting time. Despite this, we show that agent estimates reach an almost sure consensus and converge to the same optimal solution of the global optimization problem with probability one under different assumptions on the local constraint sets and the stepsize sequence.     

Inclusion functions and global optimization II

This paper discusses algorithms of Moore, Skelboe, Ichida, Fujii and Hansen for solving the global unconstrained optimization problem. These algorithms have been tried on computers, but a thorough theoretical discussion of their convergence properties has been missing. The discussion was started in part I of this paper (Mathematical Programming 33 (1985) 300–317) where the convergence to the global minimum was studied. The present paper is concerned with the different behaviours of these algorithms when they are used for the determination of global minimum points. The solution sets of the algorithms can be a subset of the set of global minimum points,G, a superset ofG, or exactlyG. The algorithms are applicable to a very general class of functions: functions which are continuous, and have suitable inclusion functions. The number of global minimum points can be infinite.This work was supported by the Deutsche Forschungsgemeinschaft.     

A new exclusion test for finding the global minimum

Exclusion algorithms have been used recently to find all solutions of a system of nonlinear equations or to find the global minimum of a function over a compact domain. These algorithms are based on a minimization condition that can be applied to each cell in the domain. In this paper, we consider Lipschitz functions of order α and give a new minimization condition for the exclusion algorithm. Furthermore, convergence and complexity results are presented for such algorithm.     

Dynamic Euclidean minimum spanning trees and extrema of binary functions

We maintain the minimum spanning tree of a point set in the plane subject to point insertions and deletions, in amortized timeO(n ^1/2 log² n) per update operation. We reduce the problem to maintaining bichromatic closest pairs, which we solve in timeO(n ^e ) per update. Our algorithm uses a novel construction, theordered nearest neighbor path of a set of points. Our results generalize to higher dimensions, and to fully dynamic algorithms for maintaining minima of binary functions, including the diameter of a point set and the bichromatic farthest pair. This research was supported in part by NSF Grant CCR-9258355     

Constructing Minimal Projective Resolutions

Let Λ be a monomial algebra. For any Λ-module M, we describe graphically its minimal projective resolution as a weighted graph Δ(M). If M is finitely generated we give two algorithms for computing Δ(M). We also give a short computation-like argument to reprove a famous Syzygy Theorem.     

Long vectors for quasi-Newton updates

This work concerns the derivation of formulae for updating quasi-Newton matrices used in algorithms for computing approximate minima of smooth unconstrained functions. The paper concentrates strictly on the techniques used to derive update formulae. It demonstrates a technique in which problems of finding matrices in ℝ ^{n ×n} of minimum Frobenius norm are converted to equivalent problems, using vector representations in ℝ ⁿ² and ℝ ^n(n+1)/2 of these matrices, and then solvingl ₂-minimization problems. These problems are more directly dealt with, and indeed, the paper demonstrates how this technique may be used to handle weighted sparse updates.     

Local convergence of collinear scaling algorithms related to direct least‐change secant update methods for minimization

Collinear scaling algorithms related to direct fixed-scale and rescaled least-change secant update methods for unconstrained minimization are derived. Theorems on local and q-superlinear convergence of these algorithms are presented. These results are extensions of those of Dennis and Walker [14] for direct least-change secant update methods. This revised version was published online in August 2006 with corrections to the Cover Date.     

Acoustic estimation of suspended sediment concentration

In this paper, the acoustic estimation of suspended sediment concentration is discussed and two estimation methods of suspended sediment concentration are presented. The first method is curve fitting method, in which, according to the acoustic backscattering theory we assume that the fitting factor K₁ (r) between the concentration M(r) obtained by acoustic observation and the concentration M₀ ( r) obtained by sampling water is a high order power function of distancer. Using least-square algorithm, we can determine the coefficients of the high order power function by minimizing the difference betweenM( r) and M₀ ( r) in the whole water profile. To the absorption coefficient of sound due to the suspension in water we do not give constraint in the first method. The second method is recursive fitting method, in which we take M₀ ( r) as the conditions of initialization and decision and give rational constraints to some parameters. The recursive process is stable. We analyzed the two methods with a lot of experimental data. The analytical results show that the estimate error of the first method is less than that of the second method and the latter can not only estimate the concentration of suspended sediment but also give the absorption coefficient of sound. Good results have been obtained with the two methods.     

An Inversion-Free Estimating Equations Approach for Gaussian Process Models

One of the scalability bottlenecks for the large-scale usage of Gaussian processes is the computation of the maximum likelihood estimates of the parameters of the covariance matrix. The classical approach requires a Cholesky factorization of the dense covariance matrix for each optimization iteration. In this work, we present an estimating equations approach for the parameters of zero-mean Gaussian processes. The distinguishing feature of this approach is that no linear system needs to be solved with the covariance matrix. Our approach requires solving an optimization problem for which the main computational expense for the calculation of its objective and gradient is the evaluation of traces of products of the covariance matrix with itself and with its derivatives. For many problems, this is an O(nlog?n) effort, and it is always no larger than O(n²). We prove that when the covariance matrix has a bounded condition number, our approach has the same convergence rate as does maximum likelihood in that the Godambe information matrix of the resulting estimator is at least as large as a fixed fraction of the Fisher information matrix. We demonstrate the effectiveness of the proposed approach on two synthetic examples, one of which involves more than 1 million data points.