The interior proximal extragradient method for solving equilibrium problems

In this article we present a new and efficient method for solving equilibrium problems on polyhedra. The method is based on an interior-quadratic proximal term which replaces the usual quadratic proximal term. This leads to an interior proximal type algorithm. Each iteration consists in a prediction step followed by a correction step as in the extragradient method. In a first algorithm each of these steps is obtained by solving an unconstrained minimization problem, while in a second algorithm the correction step is replaced by an Armijo-backtracking linesearch followed by an hyperplane projection step. We prove that our algorithms are convergent under mild assumptions: pseudomonotonicity for the two algorithms and a Lipschitz property for the first one. Finally we present some numerical experiments to illustrate the behavior of the proposed algorithms.     

A novel feedforward–feedback suboptimal control of linear time-delay systems

This paper presents a new approach for solving the optimal control problem of linear time-delay systems with a quadratic cost functional. In this approach, a method of successive substitution is employed to convert the original time-delay optimal control problem into a sequence of linear time-invariant ordinary differential equations (ODEs) without delay and advance terms. The obtained optimal control consists of a linear state feedback term and a forward term. The feedback term is determined by solving a matrix Riccati differential equation. The forward term is an infinite sum of adjoint vectors, which can be obtained by solving recursively the above-mentioned sequence of linear non-delay ODEs. A fast-converging iterative algorithm for this purpose is presented which provides a promising possible reduction of computational efforts. Numerical examples demonstrating the efficiency, simplicity and high accuracy of the suggested technique have been included. Simulation results reveal that just a few iterations of the proposed algorithm are required to find an accurate enough feedforward–feedback suboptimal control.     

Explicit and exact travelling wave solutions for the generalized derivative Schrödinger equation

In this paper, a new auxiliary equation expansion method and its algorithm is proposed by studying a first order nonlinear ordinary differential equation with a sixth-degree nonlinear term. Being concise and straightforward, the method is applied to the generalized derivative Schrödinger equation. As a result, some new exact travelling wave solutions are obtained which include bright and dark solitary wave solutions, triangular periodic wave solutions and singular solutions. This algorithm can also be applied to other nonlinear wave equations in mathematical physics.     

求解带线性约束的凸优化的一类自适应不定线性化增广拉格朗日方法

增广拉格朗日方法是求解带线性约束的凸优化问题的有效算法.线性化增广拉格朗日方法通过线性化增广拉格朗日函数的二次罚项并加上一个临近正则项,使得子问题容易求解,其中正则项系数的恰当选取对算法的收敛性和收敛速度至关重要.较大的系数可保证算法收敛性,但容易导致小步长.较小的系数允许迭代步长增大,但容易导致算法不收敛.本文考虑求解带线性等式或不等式约束的凸优化问题.我们利用自适应技术设计了一类不定线性化增广拉格朗日方法,即利用当前迭代点的信息自适应选取合适的正则项系数,在保证收敛性的前提下尽量使得子问题步长选择范围更大,从而提高算法收敛速度.我们从理论上证明了算法的全局收敛性,并利用数值实验说明了算法的有效性.     

Orthogonality correction in the conjugate-gradient method

This paper describes a simple algorithm for calculating the carryover term β in the conjugate-gradient method. The proposed algorithm incorporates an orthogonality correction as well as an automatic restart. Its performance is compared with alternate β forms reported, using five test functions and two cases of parameter estimation.     

A duality method for the compressible Reynolds equation. Application to simulation of read/write processes in magnetic storage devices

In this work the authors present a new numerical algorithm to approximate the solution of compressible Reynolds equation with additional first-order slip flow terms. This equation appears when modelling read/write processes in magnetic storage devices such as hard disks. The proposed numerical method is based on characteristics approximation for convection (dominating) terms and a duality method applied to a maximal monotone operator which represents the nonlinear diffusive term. Several test examples illustrate the good performance of the method.     

热传导(对流-扩散)方程源项识别的粒子群优化算法

提出了利用粒子群优化(PSO)算法反演热传导方程与对流-扩散方程源项的一种新方法,在已有文献方法的基础上,求解出这两类方程正问题的解析解,再把源项识别问题转化为最优化问题,结合粒子群优化算法寻优求解.通过数值模拟与统计检验,结果表明,此方法可快速有效地实现热传导方程与对流-扩散方程源项的识别,并可推广应用到其它数学物理方程的源项或参数的反演识别.     

A proximal-based decomposition method for convex minimization problems

This paper presents a decomposition method for solving convex minimization problems. At each iteration, the algorithm computes two proximal steps in the dual variables and one proximal step in the primal variables. We derive this algorithm from Rockafellar's proximal method of multipliers, which involves an augmented Lagrangian with an additional quadratic proximal term. The algorithm preserves the good features of the proximal method of multipliers, with the additional advantage that it leads to a decoupling of the constraints, and is thus suitable for parallel implementation. We allow for computing approximately the proximal minimization steps and we prove that under mild assumptions on the problem's data, the method is globally convergent and at a linear rate. The method is compared with alternating direction type methods and applied to the particular case of minimizing a convex function over a finite intersection of closed convex sets.Corresponding author. Partially supported by Air Force Office of Scientific Research Grant 91-0008 and National Science Foundation Grant DMS-9201297.     

An efficient code for the minimization of highly nonlinear and large residual least squares functions

A new code for solving the unconstrained least squares problem is given, in which a Quasi-NEWTON approximation to the second order term of the Hessian is added to the first order term of the GAUSS-NEWTON method and a line search based upon a quartile model is used. The new algorithm is shown numerically to be more efficient on large residual problems than the GAUSS-NEWTON method and a general purpose minimization algorithm based upon BFGS formula. The listing and the user's guide of the code is also given.     

求解非光滑凸规划的一种混合束方法

提出一种求解非光滑凸规划问题的混合束方法. 该方法通过对目标函数增加迫近项, 且对可行域增加信赖域约束进行迭代, 做为迫近束方法与信赖域束方法的有机结合, 混合束方法自动在二者之间切换, 收敛性分析表明该方法具有全局收敛性. 最后的数值算例验证了算法的有效性．     

A combined penalty function and gradient projection method for nonlinear programming

A new programming algorithm for nonlinear constrained optimization problems is proposed. The method is based on the penalty function approach and thereby circumyents the necessity to maintain feasibility at each iteration, but it also behaves much like the gradient projection method. Although only first-order information is used, the algorithm converges asymptotically at a rate which is independent of the magnitude of the penalty term; hence, unlike the simple gradient method, the asymptotic rate of the proposed method is not affected by the ill-conditioning associated with the introduction of the penalty term. It is shown that the asymptotic rate of convergence of the proposed method is identical with that of the gradient projection method.Dedicated to Professor M. R. HestenesThis research was supported by the National Science Foundation, Grant No. GK-16125.     

Online Gradient Descent Learning Algorithms

This paper considers the least-square online gradient descent algorithm in a reproducing kernel Hilbert space (RKHS) without an explicit regularization term. We present a novel capacity independent approach to derive error bounds and convergence results for this algorithm. The essential element in our analysis is the interplay between the generalization error and a weighted cumulative error which we define in the paper. We show that, although the algorithm does not involve an explicit RKHS regularization term, choosing the step sizes appropriately can yield competitive error rates with those in the literature.     

Preconditioning and globalizing conjugate gradients in dual space for quadratically penalized nonlinear-least squares problems

When solving nonlinear least-squares problems, it is often useful to regularize the problem using a quadratic term, a practice which is especially common in applications arising in inverse calculations. A solution method derived from a trust-region Gauss-Newton algorithm is analyzed for such applications, where, contrary to the standard algorithm, the least-squares subproblem solved at each iteration of the method is rewritten as a quadratic minimization subject to linear equality constraints. This allows the exploitation of duality properties of the associated linearized problems. This paper considers a recent conjugate-gradient-like method which performs the quadratic minimization in the dual space and produces, in exact arithmetic, the same iterates as those produced by a standard conjugate-gradients method in the primal space. This dual algorithm is computationally interesting whenever the dimension of the dual space is significantly smaller than that of the primal space, yielding gains in terms of both memory usage and computational cost. The relation between this dual space solver and PSAS (Physical-space Statistical Analysis System), another well-known dual space technique used in data assimilation problems, is explained. The use of an effective preconditioning technique is proposed and refined convergence bounds derived, which results in a practical solution method. Finally, stopping rules adequate for a trust-region solver are proposed in the dual space, providing iterates that are equivalent to those obtained with a Steihaug-Toint truncated conjugate-gradient method in the primal space.     

Simultaneous identification of diffusion coefficient,spacewise dependent source and initial value for one‐dimensional heat equation

This paper deals with an inverse problem of determining the diffusion coefficient, spacewise dependent source term, and the initial value simultaneously for a one‐dimensional heat equation based on the boundary control, boundary measurement, and temperature distribution at a given single instant in time. By a Dirichlet series representation for the boundary observation, the identification of the diffusion coefficient and initial value can be transformed into a spectral estimation problem of an exponential series with measurement error, which is solved by the matrix pencil method. For the identification of the source term, a finite difference approximation method in conjunction with the truncated singular value decomposition is adopted, where the regularization parameter is determined by the generalized cross‐validation criterion. Numerical simulations are performed to verify the result of the proposed algorithm. Copyright © 2016 John Wiley & Sons, Ltd.     

SOURCE TERM IDENTIFICATION WITH DISCONTINUOUS DUAL RECIPROCITY APPROXIMATION AND QUASI-NEWTON METHOD FROM BOUNDARY OBSERVATIONS

This paper deals with discontinuous dual reciprocity boundary element method for solving an inverse source problem.The aim of this work is to determine the source term in elliptic equations for nonhomogenous anisotropic media,where some additional boundary measurements are required.An equivalent formulation to the primary inverse problem is established based on the minimization of a functional cost,where a regularization term is employed to eliminate the oscillations of the noisy data.Moreover,an efficient algorithm is presented and tested for some numerical examples.     

A “Nonconvex+Nonconvex” approach for image restoration with impulse noise removal

In this paper, we propose a new method for image restoration problems, which are degraded by impulsive noise, with nonconvex data fitting term and nonconvex regularizer.The proposed method possesses the advantages of nonconvex data fitting and nonconvex regularizer simultaneously, namely, robustness for impulsive noise and efficiency for restoring neat edge images.Further, we propose an efficient algorithm to solve the “Nonconvex+Nonconvex” structure problem via using the alternating direction minimization, and prove that the algorithm is globally convergent when the regularization parameter is known. However, the regularization parameter is unavailable in general. Thereby, we combine the algorithm with the continuation technique and modified Morozov’s discrepancy principle to get an improved algorithm in which a suitable regularization parameter can be chosen automatically. The experiments reveal the superior performances of the proposed algorithm in comparison with some existing methods.     

Automatic Linear Correction of Rounding Errors

A new automatic method to correct the first-order effect of floating point rounding errors on the result of a numerical algorithm is presented. A correcting term and a confidence threshold are computed using algorithmic differentiation, computation of elementary rounding error and running error analysis. Algorithms for which the accuracy of the result is not affected by higher order terms are identified. The correction is applied to the final result or to sensitive intermediate results to improve the accuracy of the computed result and/or the stability of the algorithm.This revised version was published online in October 2005 with corrections to the Cover Date.     

An improved enumerative algorithm for solving quadratic zero-one programming

An improved algorithm is proposed for the minimization of a quadratic function in zero-one variables under quadratic constraints which is based on the idea of additive penalties proposed by P. Hansen. At first, the quadratic function in which all the coefficients except the constant term are nonnegative is obtained by the introduction of the negative variables, and the constant term is a tighter bound of the function. Starting from the tighter bound, some properties are obtained and some efficient tests are established. To obtain a much tighter bound, a simple heuristic method is suggested instead of solving a linear programming problem. Furthermore, the flexibility dealing with some tests is discussed and it is also helpful to the algorithm.     

Numerical Experience with a Reduced Hessian Method for Large Scale Constrained Optimization

The reduced Hessian SQP algorithm presented in Biegler et al. [SIAM J. Optimization, Vol. 5, no. 2, pp. 314–347, 1995.] is developed in this paper into a practical method for large-scale optimization. The novelty of the algorithm lies in the incorporation of a correction vector that approximates the cross term Z^TWYp_Y. This improves the stability and robustness of the algorithm without increasing its computational cost. The paper studies how to implement the algorithm efficiently, and presents a set of tests illustrating its numerical performance. An analytic example, showing the benefits of the correction term, is also presented.