Solving Nonlinear Programming Problems with Noisy Function Values and Noisy Gradients

An efficient algorithm for solving nonlinear programs with noisy equality constraints is introduced and analyzed. The unknown exact constraints are replaced by surrogates based on the bundle idea, a well-known strategy from nonsmooth optimization. This concept allows us to perform a fast computation of the surrogates by solving simple quadratic optimization problems, control the memory needed by the algorithm, and prove the differentiability properties of the surrogate functions. The latter aspect allows us to invoke a sequential quadratic programming method. The overall algorithm is of the quasi-Newton type. Besides convergence theorems, qualification results are given and numerical test runs are discussed.     

A Framework for Globally Convergent Algorithms Using Gradient Bounding Functions

This paper presents a novel framework for developing globally convergent algorithms without evaluating the value of a given function. One way to implement such a framework for a twice continuously differentiable function is to apply linear bounding functions (LBFs) to its gradient function. The algorithm thus obtained can get a better point in each iteration without using a line search. Under certain conditions, it can achieve at least superlinear convergent rate 1.618 without calculating explicitly the Hessian. Furthermore, the strategy of switching from the negative gradient direction to the Newton-alike direction is derived in a natural manner and is computationally effective. Numerical examples are used to show how the algorithm works.     

Approximate Gradient Projection Method with Runge-Kutta Schemes for Optimal Control Problems

We consider an optimal control problem for systems governed by ordinary differential equations with control constraints. The state equation is discretized by the explicit fourth order Runge-Kutta scheme and the controls are approximated by discontinuous piecewise affine ones. We then propose an approximate gradient projection method that generates sequences of discrete controls and progressively refines the discretization during the iterations. Instead of using the exact discrete directional derivative, which is difficult to calculate, we use an approximate derivative of the cost functional defined by discretizing the continuous adjoint equation by the same Runge-Kutta scheme and the integral involved by Simpson's integration rule, both involving intermediate approximations. The main result is that accumulation points, if they exist, of sequences constructed by this method satisfy the weak necessary conditions for optimality for the continuous problem. Finally, numerical examples are given.     

Transportation to Random Zeroes by the Gradient Flow

We consider the zeroes of the random Gaussian entire function ( , . . . are Gaussian i.i.d. complex random variables) and show that their basins under the gradient flow of the random potential partition the complex plane into domains of equal area. We find three characteristic exponents 1, 8/5, and 4 of this random partition: the probability that the diameter of a particular basin is greater than R is exponentially small in R; the probability that a given point z lies at a distance larger than R from the zero, it is attracted to decays as ; and the probability that, after throwing away 1% of the area of the basin, its diameter is still larger than R decays as . We also introduce a combinatorial procedure that modifies a small portion of each basin in such a way that the probability that the diameter of a particular modified basin is greater than R decays as . F.N. and A.V. partially supported by the National Science Foundation, DMS grant 0501067. M.S. partially supported by the Israel Science Foundation of the Israel Academy of Sciences and Humanities, grant 357/04. Received: May 2006 Revision: February 2007 Accepted: March 2007     

Novel Algorithms for Noisy Minimization Problems with Applications to Neural Networks Training

The supervisor and searcher cooperation framework (SSC), introduced in Refs. 1 and 2, provides an effective way to design efficient optimization algorithms combining the desirable features of the two existing ones. This work aims to develop efficient algorithms for a wide range of noisy optimization problems including those posed by feedforward neural networks training. It introduces two basic SSC algorithms. The first seems suited for generic problems. The second is motivated by neural networks training problems. It introduces also inexact variants of the two algorithms, which seem to possess desirable properties. It establishes general theoretical results about the convergence and speed of SSC algorithms and illustrates their appealing attributes through numerical tests on deterministic, stochastic, and neural networks training problems.     

涉及导数与差分的亚纯函数的小函数的收敛指数与级

设f(z)是一个复平面上的亚纯函数,c是一个非零有穷复数,a(z)是f(z)的一个小函数,本文研究f(z)-a(z),f(z + c) - a(z)及Δncf(z)-a(z)(n ∈N+)的零点收敛指数与f(z)的级之间的关系.由此改进了涉及导数与差分的亚纯函数值分布的一些相关结果.     

涉及导数与差分的亚纯函数唯一性

设f,g是两个非常数亚纯函数,a是一个非零有穷复数,n≥5是一个正整数.若[f(z)]~n与[g(z)]~n CM分担a,f(z)与g(z) CM分担∞,且N_(1))(r,f)=S(r,f),则或者f(z)三tg(z),其中t~n=1;或者f(z)g(z)≡t,其中t~n=a~2.由此改进了涉及导数与差分的一些亚纯函数唯一性的结果.     

Gradient Estimates of Solutions to the Conductivity Problem with Flatter Insulators

We study the insulated conductivity problem with inclusions embedded in a bounded domain in R~n. When the distance of inclusions, denoted by ε, goes to 0, the gradient of solutions may blow up. When two inclusions are strictly convex, it was known that an upper bound of the blow-up rate is of order ε~(-1/2) for n = 2, and is of order ε~(-1/2+β)for some β 0 when dimension n ≥ 3. In this paper, we generalize the above results for insulators with flatter boundaries near touching points.     

Training Multilayer Perceptrons Via Minimization of Sum of Ridge Functions

Motivated by the problem of training multilayer perceptrons in neural networks, we consider the problem of minimizing E(x)= _i=1 ⁿ f _i ( _i x), where _i R ^s , 1in, and each f _i ( _i x) is a ridge function. We show that when n is small the problem of minimizing E can be treated as one of minimizing univariate functions, and we use the gradient algorithms for minimizing E when n is moderately large. For large n, we present the online gradient algorithms and especially show the monotonicity and weak convergence of the algorithms.     

Simulated Annealing for noisy cost functions

We generalize a classical convergence result for the Simulated Annealing algorithm to a stochastic optimization context, i.e., to the case where cost function observations are disturbed by random noise. It is shown that for a certain class of noise distributions, the convergence assertion remains valid, provided that the standard deviation of the noise is reduced in the successive steps of cost function evaluation (e.g., by repeated observation) with a speed O(k ^- ), where is an arbitrary constant larger than one.     

Aggregation on Finite Ordinal Scales by Scale Independent Functions

We define and investigate the scale independent aggregation functions that are meaningful to aggregate finite ordinal numerical scales. Here scale independence means that the functions always have discrete representatives when the ordinal scales are considered as totally ordered finite sets. We also show that those scale independent functions identify with the so-called order invariant functions, which have been described recently. In particular, this identification allows us to justify the continuity property for certain order invariant functions in a natural way. Mathematics Subject Classifications (2000) Primary: 91C05, 91E45; Secondary: 06A99, 39A12.Jean-Luc Marichal: Partially supported by a grant from the David M. Kennedy Center for International Studies, Brigham Young University.Radko Mesiar: Partially supported by grants VEGA 1/0273/03 and APVT-20-023402.     

改进的预处理共轭斜量法及其在工程有限元分析中的应用

本文就预处理共轭斜量法(PCCG法)给出了两个具有理论和实际意义的定理,它们分别讨论了迭代解的定性性质和迭代矩阵的构造原则.作者提出了新的非M-矩阵的不完全LU分解技术和迭代矩阵的构造方法.用此改进的PCCG法,对病态问题和大型三维有限元问题进行了计算并与其他方法作了对比,分析了PCCG法在求解病态方程组时的反常现象.计算结果表明本文建议的方法是求解大型有限元方程组和病态方程组的一种十分有效的方法.     

The Partition of Unity Method for High-Order Finite Volume Schemes Using Radial Basis Functions Reconstruction

This paper introduces the use of partition of unity method for the develop-ment of a high order finite volume discretization scheme on unstructured grids for solv-ing diffusion models based on partial differential equations. The unknown function and its gradient can be accurately reconstructed using high order optimal recovery based on radial basis functions. The methodology proposed is applied to the noise removal prob-lem in functional surfaces and images. Numerical results demonstrate the effectiveness of the new numerical approach and provide experimental order of convergence.     

亚纯倒星象函数和倒凸象函数类的拟Hadamard卷积

引进了去心圆盘U~*={z:0|z|1}内的亚纯倒星象函数和倒凸象函数的某些新子类,研究了该类中函数的拟Hadamard卷积,所得结果推广了前人的某些工作.     

Stochastic Comparison Algorithm for Discrete Optimization with Estimation of Time-Varying Objective Functions

In this paper, the optimization of time-varying objective functions, known only through estimates, is considered. Recent research defined algorithms for static optimization problems. Based on one of these algorithms, we derive an optimization scheme for the time-varying case. In stochastic optimization problems, convergence of an algorithm to the optimum prevents the algorithm from being efficiently adaptive to changes of the objective function if it is time-varying. So, convergence cannot be required in a time-varying scenario. Rather, we require convergence to the optimum with high probability together with a satisfactory dynamical behavior. Analytical and simulative results illustrate the performance of the proposed algorithm compared with other optimization techniques.     

Numerical Schemes for Linear and Non-Linear Enhancement of DW-MRI

We consider the linear and non-linear enhancement of diffusion weighted magnetic resonance images (DW-MRI) to use contextual information in denoising and inferring fiber crossings. We describe the space of DW-MRI images in a moving frame of reference, attached to fiber fragments which allows for convection-diffusion along the fibers. Because of this approach, our method is naturally able to handle crossings in data. We will perform experiments showing the ability of the enhancement to infer information about crossing structures, even in diffusion tensor images (DTI) which are incapable of representing crossings themselves. We will present a novel non-linear enhancement technique which performs better than linear methods in areas around ventricles, thereby eliminating the need for additional preprocessing steps to segment out the ventricles. We pay special attention to the details of implementation of the various numerical schemes.     

Factor screening for simulation with multiple responses: Sequential bifurcation

The goal of factor screening is to find the really important inputs (factors) among the many inputs that may be changed in a realistic simulation experiment. A specific method is sequential bifurcation (SB), which is a sequential method that changes groups of inputs simultaneously. SB is most efficient and effective if the following assumptions are satisfied: (i) second-order polynomials are adequate approximations of the input/output functions implied by the simulation model; (ii) the signs of all first-order effects are known; and (iii) if two inputs have no important first-order effects, then they have no important second-order effects either (heredity property). This paper examines SB for random simulation with multiple responses (outputs), called multi-response SB (MSB). This MSB selects groups of inputs such that—within a group—all inputs have the same sign for a specific type of output, so no cancellation of first-order effects occurs. To obtain enough replicates (replications) for correctly classifying a group effect or an individual effect as being important or unimportant, MSB applies Wald’s sequential probability ratio test (SPRT). The initial number of replicates in this SPRT is also selected efficiently by MSB. Moreover, MSB includes a procedure to validate the three assumptions of MSB. The paper evaluates the performance of MSB through extensive Monte Carlo experiments that satisfy all MSB assumptions, and through a case study representing a logistic system in China; the results are very promising.     

Formulas for Liapunov Functions for Systems of Linear Difference Equations

Explicit quadratic Liapunov functions that provide necessaryand sufficient conditions for the asymptotic stability of thesystem of linear difference equations x (t + 1) = Ax(t) areconstructed by transforming the original systems to y (t + 1)= Gy(t), where G is a companion matrix associated with the characteristicpolynomial of A. A necessary and sufficient condition for allroots of the characteristic polynomial to lie in the unit circle|z| < 1 on the complex plane is also derived. 2000 MathematicalsSubject Classification 39A11, 93D05.     

Meromorphic Functions of Derivatives Covering Certain Finite Sets at the Same Points

ＭｅｒｏｍｏｒｐｈｉｃＦｕｎｃｔｉｏｎｓｏｆＤｅｒｉｖａｔｉｖｅｓＣｏｖｅｒｉｎｇＣｅｒｔａｉｎＦｉｎｉｔｅＳｅｔｓａｔｔｈｅＳａｍｅＰｏｉｎｔｓ￥ＷｅｎＺｈｏｎｇｌｉａｎｇ（ＷｅｎｚｈｏｕＴｅａｃｈｅｒｓＣｏｌｌｅｇｅ，Ｚｈｅｊｉａｎｇ，３２５００...     

Spline Estimation of Discontinuous Regression Functions

Abstract

This article deals with regression function estimation when the regression function is smooth at all but a finite number of points. An important question is: How can one produce discontinuous output without knowledge of the location of discontinuity points? Unlike most commonly used smoothers that tend to blur discontinuity in the data, we need to find a smoother that can detect such discontinuity. In this article, linear splines are used to estimate discontinuous regression functions. A procedure of knot-merging is introduced for the estimation of regression functions near discontinuous points. The basic idea is to use multiple knots for spline estimates. We use an automatic procedure involving the least squares method, stepwise knot addition, stepwise basis deletion, knot-merging, and the Bayes information criterion to select the final model. The proposed method can produce discontinuous outputs. Numerical examples using both simulated and real data are given to illustrate the performance of the proposed method.