The Global Convergence of Self-Scaling BFGS Algorithm with Nonmonotone Line Search for Unconstrained Nonconvex Optimization Problems

The self-scaling quasi-Newton method solves an unconstrained optimization problem by scaling the Hessian approximation matrix before it is updated at each iteration to avoid the possible large eigenvalues in the Hessian approximation matrices of the objective function. It has been proved in the literature that this method has the global and superlinear convergence when the objective function is convex （or even uniformly convex）. We propose to solve unconstrained nonconvex optimization problems by a self-scaling BFGS algorithm with nonmonotone linear search. Nonmonotone line search has been recognized in numerical practices as a competitive approach for solving large-scale nonlinear problems. We consider two different nonmonotone line search forms and study the global convergence of these nonmonotone self-scale BFGS algorithms. We prove that, under some weaker condition than that in the literature, both forms of the self-scaling BFGS algorithm are globally convergent for unconstrained nonconvex optimization problems.     

TWO NOVEL GRADIENT METHODS WITH OPTIMAL STEP SIZES

In this work we introduce two new Barzilai and Borwein-like steps sizes for the classical gradient method for strictly convex quadratic optimization problems.The proposed step sizes employ second-order information in order to obtain faster gradient-type methods.Both step sizes are derived from two unconstrained optimization models that involve approximate information of the Hessian of the objective function.A convergence analysis of the proposed algorithm is provided.Some numerical experiments are performed in order to compare the efficiency and effectiveness of the proposed methods with similar methods in the literature.Experimentally,it is observed that our proposals accelerate the gradient method at nearly no extra computational cost,which makes our proposal a good alternative to solve large-scale problems.     

A Constrained Optimization Approach for LCP

In this paper, LCP is converted to an equivalent nonsmooth nonlinear equation system H(x,y) = 0 by using the famous NCP function-Fischer-Burmeister function. Note that some equations in H(x, y) = 0 are nonsmooth and nonlinear hence difficult to solve while the others are linear hence easy to solve. Then we further convert the nonlinear equation system H(x, y) = 0 to an optimization problem with linear equality constraints. After that we study the conditions under which the K-T points of the optimization problem are the solutions of the original LCP and propose a method to solve the optimization problem. In this algorithm, the search direction is obtained by solving a strict convex programming at each iterative point, However, our algorithm is essentially different from traditional SQP method. The global convergence of the method is proved under mild conditions. In addition, we can prove that the algorithm is convergent superlinearly under the conditions: M is P0 matrix and the limit point is a strict complementarity solution of LCP. Preliminary numerical experiments are reported with this method.     

无记忆非拟 Newton 算法的导出和全局收敛性

In this paper, a new class of memoryless non-quasi-Newton method for solving unconstrained optimization problems is proposed, and the global convergence of this method with inexact line search is proved. Furthermore, we propose a hybrid method that mixes both the memoryless non-quasi-Newton method and the memoryless Perry-Shanno quasi-Newton method. The global convergence of this hybrid memoryless method is proved under mild assumptions. The initial results show that these new methods are efficient for the given test problems. Especially the memoryless non-quasi-Newton method requires little storage and computation, so it is able to efficiently solve large scale optimization problems.     

SINGLY DIAGONALLY IMPLICIT RUNGE-KUTTA METHODS COMBINING LINE SEARCH TECHNIQUES FOR UNCONSTRAINED OPTIMIZATION

There exists a strong connection between numerical methods for the integration of ordinary differential equations and optimization problems. In this paper, we try to discover further their links. And we transform unconstrained problems to the equivalent ordinary differential equations and construct the LRKOPT method to solve them by combining the second order singly diagonally implicit Runge-Kutta formulas and line search techniques.Moreover we analyze the global convergence and the local convergence of the LRKOPT method. Promising numerical results are also reported.     

A new look at the Lagrange method for continuous-time stochastic optimization

We formulate a Lagrange method for continuous-time stochastic optimization in an appropriate normed space by using a proper stochastic process as the Lagrange multiplier.The obtained optimality conditions are applied to different types of problems.Some examples selected from control theory and economic theory are studied to test and illustrate the potential applications of the method.     

BLOCK-TRIANGULAR PRECONDITIONERS FOR SYSTEMS ARISING FROM EDGE-PRESERVING IMAGE RESTORATION

Signal and image restoration problems are often solved by minimizing a cost function consisting of an l2 data-fidelity term and a regularization term. We consider a class of convex and edge-preserving regularization functions. In specific, half-quadratic regularization as a fixed-point iteration method is usually employed to solve this problem. The main aim of this paper is to solve the above-described signal and image restoration problems with the half-quadratic regularization technique by making use of the Newton method. At each iteration of the Newton method, the Newton equation is a structured system of linear equations of a symmetric positive definite coefficient matrix, and may be efficiently solved by the preconditioned conjugate gradient method accelerated with the modified block SSOR preconditioner. Our experimental results show that the modified block-SSOR preconditioned conjugate gradient method is feasible and effective for further improving the numerical performance of the half-quadratic regularization approach.     

AN ADAPTIVE MULTI-SCALE CONJUGATE GRADIENT METHOD FOR DISTRIBUTED PARAMETER ESTIMATION OF 2-D WAVE EQUATION

An adaptive multi-scale conjugate gradient method for distributed parameter estimations (or inverse problems) of wave equation is presented. The identification of the coefficients of wave equations in two dimensions is considered. First, the conjugate gradient method for optimization is adopted to solve the inverse problems. Second, the idea of multi-scale inversion and the necessary conditions that the optimal solution should be the fixed point of multi-scale inversion method is considered. An adaptive multi-scale inversion method for the inoerse problem is developed in conjunction with the conjugate gradient method. Finally, some numerical results are shown to indicate the robustness and effectiveness of our method.     

On First Order Optimality Conditions for Vector Optimization

We develop first order optimality conditions for constrained vector optimization. The partial orders for the objective and the constraints are induced by closed and convex cones with nonempty interior. After presenting some well known existence results for these problems, based on a scalarization approach, we establish necessity of the optimality conditions under a Slater-like constraint qualification, and then sufficiency for the K-convex case. We present two alternative sets of optimality conditions, with the same properties in connection with necessity and sufficiency, but which are different with respect to the dimension of the spaces to which the dual multipliers belong. We introduce a duality scheme, with a point-to-set dual objective, for which strong duality holds. Some examples and open problems for future research are also presented,     

关于吴方法应用在非线性规划问题中的几个测试问题

The nonlinear programming problems are important in mathematics applications.They are usually solved by various kinds of numerical methods.This will give solutions in the from of local extremal values but not necessarily global optimal ones.The present paper shows how to solve the nonlinear programming problems by the MM-method(Mathematics-Mechanization method)or Wu‘s method.Wu‘s method is different from the numerical method in that the computations are symbolic instead of numerial ones.Theoretically it is based on computer algebra and algebraic geometry.The author uses the computer to get complete global solutions of some practical test problems in the nonlinear programming.The computations shows that Wu‘s methtod is concise for solving the nonlinear programming problems,and is also quite efficient.     

Exponential Stability for Uncertain Dynamic Systems with Time-Varying Delays: LMI Optimization Approach

In this paper, we consider dynamic systems with uncertainties and time-varying delays. Based on the Lyapunov method and convex optimization approach, a delay-dependent criterion for exponential stability of the system is derived in terms of LMI (linear matrix inequality). In order to solve effectively the LMI convex optimization problem, an interior-point algorithm is utilized in this work. Numerical examples are illustrated to show the effectiveness of our results.     

Adaptive mesh method for topology optimization of fluid flow

In order to solve the topology optimization problems of fluid flow and obtain higher resolution of the interface with a minimum of additional expense, an automatic local adaptive mesh refinement method is proposed. The optimization problem is solved by a simple but robust optimality criteria (OC) algorithm. A material distribution information based adaptive mesh method is adopted during the optimization process. The optimization procedure is provided and verified with several benchmark examples.     

广义黏性逼近方法及其应用

黏性逼近方法在非扩张映射不动点问题的研究中扮演着重要的角色。提出了一类广义黏性逼近方法，在一定条件下，证明了该算法的收敛性.作为应用，将所得的收敛性结果应用于求解约束凸优化问题与双层优化问题。     

Global Convergence of a Nonlinear Programming Method Using Convex Approximations

The method of moving asymptotes (MMA) and its globally convergent extension SCP (sequential convex programming) are known to work well for certain problems arising in structural optimization. In this paper, the methods are extended for a general mathematical programming framework and a new scheme to update certain penalty parameters is defined, which leads to a considerable improvement in the performance. Properties of the approximation functions are outlined in detail. All convergence results of the traditional methods are preserved.     

Shape and topology optimization for elliptic boundary value problems using a piecewise constant level set method

The aim of this paper is to propose a variational piecewise constant level set method for solving elliptic shape and topology optimization problems. The original model is approximated by a two-phase optimal shape design problem by the ersatz material approach. Under the piecewise constant level set framework, we first reformulate the two-phase design problem to be a new constrained optimization problem with respect to the piecewise constant level set function. Then we solve it by the projection Lagrangian method. A gradient-type iterative algorithm is presented. Comparisons between our numerical results and those obtained by level set approaches show the effectiveness, accuracy and efficiency of our algorithm.     

Bundle-based decomposition for large-scale convex optimization: Error estimate and application to block-angular linear programs

Robinson has proposed the bundle-based decomposition algorithm to solve a class of structured large-scale convex optimization problems. In this method, the original problem is transformed (by dualization) to an unconstrained nonsmooth concave optimization problem which is in turn solved by using a modified bundle method. In this paper, we give a posteriori error estimates on the approximate primal optimal solution and on the duality gap. We describe implementation and present computational experience with a special case of this class of problems, namely, block-angular linear programming problems. We observe that the method is efficient in obtaining the approximate optimal solution and compares favorably with MINOS and an advanced implementation of the Dantzig—Wolfe decomposition method.     

Design of planar articulated mechanisms using branch and bound

This paper considers an optimization model and a solution method for the design of two-dimensional mechanical mechanisms. The mechanism design problem is modeled as a nonconvex mixed integer program which allows the optimal topology and geometry of the mechanism to be determined simultaneously. The underlying mechanical analysis model is based on a truss representation allowing for large displacements. For mechanisms undergoing large displacements elastic stability is of major concern. We derive conditions, modeled by nonlinear matrix inequalities, which guarantee that a stable equilibrium is found and that buckling is prevented. The feasible set of the design problem is described by nonlinear differentiable and non-differentiable constraints as well as nonlinear matrix inequalities.To solve the mechanism design problem a branch and bound method based on convex relaxations is developed. To guarantee convergence of the method, two different types of convex relaxations are derived. The relaxations are strengthened by adding valid inequalities to the feasible set and by solving bound contraction sub-problems. Encouraging computational results indicate that the branch and bound method can reliably solve mechanism design problems of realistic size to global optimality.     

一类不可微二次规划逆问题

本文求解了一类二次规划的逆问题,具体为目标函数是矩阵谱范数与向量无穷范数之和的最小化问题.首先将该问题转化为目标函数可分离变量的凸优化问题,提出用G-ADMM法求解.并结合奇异值阈值算法,Moreau-Yosida正则化算法,matlab优化工具箱的quadprog函数来精确求解相应的子问题.而对于其中一个子问题的精确...     

An effective and simple heuristic for the set covering problem

This paper investigates the development of an effective heuristic to solve the set covering problem (SCP) by applying the meta-heuristic Meta-RaPS (Meta-heuristic for Randomized Priority Search). In Meta-RaPS, a feasible solution is generated by introducing random factors into a construction method. Then the feasible solutions can be improved by an improvement heuristic. In addition to applying the basic Meta-RaPS, the heuristic developed herein integrates the elements of randomizing the selection of priority rules, penalizing the worst columns when the searching space is highly condensed, and defining the core problem to speedup the algorithm. This heuristic has been tested on 80 SCP instances from the OR-Library. The sizes of the problems are up to 1000 rows × 10,000 columns for non-unicost SCP, and 28,160 rows × 11,264 columns for the unicost SCP. This heuristic is only one of two known SCP heuristics to find all optimal/best known solutions for those non-unicost instances. In addition, this heuristic is the best for unicost problems among the heuristics in terms of solution quality. Furthermore, evolving from a simple greedy heuristic, it is simple and easy to code. This heuristic enriches the options of practitioners in the optimization area.     

熵函数法的数学理论

１．引言考虑复合函数其中ｇ;：Ｒ"－Ｒ,ｉ＝１,２,...;。连续可微．因的ｘ）的不可微性,涉及的ｘ）的优化问题,如极大极小问题Ｉｒｌｍｌｎｏｔｘｊ．ｆｉｊＺＦＲ"通常属不可微优化范畴．文山借助最大嫡原理推导出一类一致逼近一（Ｘ）的可微函数（称之为妨函数）Ｏ。ｉｌｌ--一ｉｆｌｙｌｌＸＤＣＱ．ｌｏｌｌ．ＩＪｊ容易证明ｔｉｍ人一中且对任意ｘＥＲ",ＣＭ＋ＯＯｇｂｃｌ（ｌ＞ｏｆｃ。（ｌ,ＶＣＺ＞ＣＩ＞０,（４ｉｌｌｆｆｉ０＜ｕｌ。（）ｄｉ（］＜．（５基于该性质,文山一【４］通过一次取定较大有限值Ｃ。＞０,将…