一种求解单调变分不等式的部分并行分裂LQP交替方向法

LQP交替方向法是求解可分离结构型单调变分不等式问题的一种非常有效的方法.它不仅可以充分地利用目标函数的可分结构,将原问题分解为多个更易求解的子问题,还更适合求解大规模问题.对于带有三个可分离算子的单调变分不等式问题,结合增广拉格朗日算法和LQP交替方向法提出了一种部分并行分裂LQP交替方向法,构造了新算法的两个下降方向,结合这两个下降方向得到了一个新的下降方向,沿着这个新的下降方向给出了最优步长.并在较弱的假设条件下,证明了新算法的全局收敛性.     

A feasible descent SQP algorithm for general constrained optimization without strict complementarity

In this paper, a class of optimization problems with equality and inequality constraints is discussed. Firstly, the original problem is transformed to an associated simpler problem with only inequality constraints and a parameter. The later problem is shown to be equivalent to the original problem if the parameter is large enough (but finite), then a feasible descent SQP algorithm for the simplified problem is presented. At each iteration of the proposed algorithm, a master direction is obtained by solving a quadratic program (which always has a feasible solution). With two corrections on the master direction by two simple explicit formulas, the algorithm generates a feasible descent direction for the simplified problem and a height-order correction direction which can avoid the Maratos effect without the strict complementarity, then performs a curve search to obtain the next iteration point. Thanks to the new height-order correction technique, under mild conditions without the strict complementarity, the globally and superlinearly convergent properties are obtained. Finally, an efficient implementation of the numerical experiments is reported.     

A descent algorithm without line search for unconstrained optimization

In this paper, a new descent algorithm for solving unconstrained optimization problem is presented. Its search direction is descent and line search procedure can be avoided except for the first iteration. It is globally convergent under mild conditions. The search direction of the new algorithm is generalized and convergence of corresponding algorithm is also proved. Numerical results show that the algorithm is efficient for given test problems.     

Globally Convergent Algorithms for Unconstrained Optimization

A new globalization strategy for solving an unconstrained minimization problem is proposed based on the idea of combining Newton's direction and the steepest descent direction WITHIN each iteration. Global convergence is guaranteed with an arbitrary initial point. The search direction in each iteration is chosen to be as close to the Newton's direction as possible and could be the Newton's direction itself. Asymptotically the Newton step will be taken in each iteration and thus the local convergence is quadratic. Numerical experiments are also reported. Possible combination of a Quasi-Newton direction with the steepest descent direction is also considered in our numerical experiments. The differences between the proposed strategy and a few other strategies are also discussed.     

A constrained optimization reformulation and a feasible descent direction method for L_{1/2} regularization

In this paper, we first propose a constrained optimization reformulation to the \(L_{1/2}\) regularization problem. The constrained problem is to minimize a smooth function subject to some quadratic constraints and nonnegative constraints. A good property of the constrained problem is that at any feasible point, the set of all feasible directions coincides with the set of all linearized feasible directions. Consequently, the KKT point always exists. Moreover, we will show that the KKT points are the same as the stationary points of the \(L_{1/2}\) regularization problem. Based on the constrained optimization reformulation, we propose a feasible descent direction method called feasible steepest descent method for solving the unconstrained \(L_{1/2}\) regularization problem. It is an extension of the steepest descent method for solving smooth unconstrained optimization problem. The feasible steepest descent direction has an explicit expression and the method is easy to implement. Under very mild conditions, we show that the proposed method is globally convergent. We apply the proposed method to solve some practical problems arising from compressed sensing. The results show its efficiency.     

A new LQP alternating direction method for solving variational inequality problems with separable structure

We presented a new logarithmic-quadratic proximal alternating direction scheme for the separable constrained convex programming problem. The predictor is obtained by solving series of related systems of non-linear equations in a parallel wise. The new iterate is obtained by searching the optimal step size along a new descent direction. The new direction is obtained by the linear combination of two descent directions. Global convergence of the proposed method is proved under certain assumptions. We show the O(1 / t) convergence rate for the parallel LQP alternating direction method.     

The relationship between theorems of the alternative,least norm problems,steepest descent directions,and degeneracy: A review

Recently, it has been observed that several nondifferentiable minimization problems share the property that the question of whether a given point is optimal can be answered by solving a certain bounded least squares problem. If the resulting residual vector,r, vanishes then the current point is optimal. Otherwise,r is a descent direction. In fact, as we shall see,r points at the steepest descent direction. On the other hand, it is customary to characterize the optimality conditions (and the steepest descent vector) of a convex nondifferentiable function via its subdifferential. Also, it is well known that optimality conditions are usually related to theorems of the alternative. One aim of our survey is to clarify the relations between these subjects. Another aim is to introduce a new type of theorems of the alternative. The new theorems characterize the optimality conditions of discretel ₁ approximation problems and multifacility location problems, and provide a simple way to obtain the subdifferential and the steepest descent direction in such problems. A further objective of our review is to demonstrate that the ability to compute the steepest descent direction at degenerate dead points opens a new way for handling degeneracy in active set methods.     

A Linearly Convergent Derivative-Free Descent Method for Strongly Monotone Complementarity Problems

We establish the first rate of convergence result for the class of derivative-free descent methods for solving complementarity problems. The algorithm considered here is based on the implicit Lagrangian reformulation [26, 35] of the nonlinear complementarity problem, and makes use of the descent direction proposed in [42], but employs a different Armijo-type linesearch rule. We show that in the strongly monotone case, the iterates generated by the method converge globally at a linear rate to the solution of the problem.     

First fall degree and Weil descent

Polynomial systems arising from a Weil descent have many applications in cryptography, including the HFE cryptosystem and the elliptic curve discrete logarithm problem over small characteristic fields. Understanding the exact complexity of solving these systems is essential for the applications. A first step in that direction is to study the first fall degree of the systems. In this paper, we establish a rigorous general bound on the first fall degree of polynomial systems arising from a Weil descent. We also provide experimental data to study the tightness of our bound in general and its plausible consequences on the complexity of polynomial systems arising from a Weil descent.     

A robust descent type algorithm for geophysical inversion through adaptive regularization

A robust descent type algorithm through adaptive regularization is designed to solve a geophysical inverse problem. The scheme uses a regularized descent direction which is obtained through minimization of smoothed functional in every iterative step. The step length factor is chosen using Armijo's rule. Numerical experiment conducted to invert synthetic and field geophysical data demonstrates a high order of robustness in retrieving the model parameters.     

A Two-Term PRP-Based Descent Method

In this paper, by the use of the project of the PRP (Polak–Ribiére–Polyak) conjugate gradient direction, we develop a PRP-based descent method for solving unconstrained optimization problem. The method provides a sufficient descent direction for the objective function. Moreover, if exact line search is used, the method reduces to the standard PRP method. Under suitable conditions, we show that the method with some backtracking line search or the generalized Wolfe-type line search is globally convergent. We also report some numerical results and compare the performance of the method with some existing conjugate gradient methods. The results show that the proposed method is efficient.     

Feasible Direction Interior-Point Technique for Nonlinear Optimization

We propose a feasible direction approach for the minimization by interior-point algorithms of a smooth function under smooth equality and inequality constraints. It consists of the iterative solution in the primal and dual variables of the Karush–Kuhn–Tucker first-order optimality conditions. At each iteration, a descent direction is defined by solving a linear system. In a second stage, the linear system is perturbed so as to deflect the descent direction and obtain a feasible descent direction. A line search is then performed to get a new interior point and ensure global convergence. Based on this approach, first-order, Newton, and quasi-Newton algorithms can be obtained. To introduce the method, we consider first the inequality constrained problem and present a globally convergent basic algorithm. Particular first-order and quasi-Newton versions of this algorithm are also stated. Then, equality constraints are included. This method, which is simple to code, does not require the solution of quadratic programs and it is neither a penalty method nor a barrier method. Several practical applications and numerical results show that our method is strong and efficient.     

Approximation of the steepest descent direction for the O-D matrix adjustment problem

In this paper, a method to approximate the directions of Clarke's generalized gradient of the upper level function for the demand adjustment problem on traffic networks is presented. Its consistency is analyzed in detail. The theoretical background on which this method relies is the known property of proximal subgradients of approximating subgradients of proximal bounded and lower semicountinuous functions using the Moreau envelopes. A double penalty approach is employed to approximate the proximal subgradients provided by these envelopes. An algorithm based on partial linearization is used to solve the resulting nonconvex problem that approximates the Moreau envelopes, and a method to verify the accuracy of the approximation to the steepest descent direction at points of differentiability is developed, so it may be used as a suitable stopping criterion. Finally, a set of experiments with test problems are presented, illustrating the approximation of the solutions to a steepest descent direction evaluated numerically. Research supported under Spanish CICYT project TRA99-1156-C02-02.     

On an algorithm solving two-level programming problems with nonunique lower level solutions

In the paper, an algorithm is presented for solving two-level programming problems. This algorithm combines a direction finding problem with a regularization of the lower level problem. The upper level objective function is included in the regularzation to yield uniqueness of the follower's solution set. This is possible if the problem functions are convex and the upper level objective function has a positive definite Hessian. The computation of a direction of descent and of the step size is discussed in more detail. Afterwards the convergence proof is given.Last but not least some remarks and examples describing the difficulty of the inclusion of upper-level constraints also depending on the variables of the lower level are added.     

A Subspace Modified PRP Method for Large-scale Nonlinear Box-Constrained Optimization

In this article, we first propose a feasible steepest descent direction for box-constrained optimization. By the use of the direction and recently developed modified PRP method, we propose a subspace modified PRP method for box-constrained optimization. Under appropriate conditions, we show that the method is globally convergent. Numerical experiments are presented using box-constrained problems in the CUTEr test problem libraries.     

一个改进的序列二次规划可行下降算法及其全局收敛性

利用广义投影校正技术对搜索方向进行某种修正,改进假设条件,采用一种新型的一阶修正方向并结合SQP技术,建立了求解非线性约束最优化问题(p)的一个新的SQP可行下降算法,在较温和的假设条件下证明了算法的全局收敛性.由于新算法仅需较小的存储,从而适合大规模最优化问题的计算.     

A new reduced gradient method for solving linearly constrained multiobjective optimization problems

In this paper, we consider the linearly constrained multiobjective minimization, and we propose a new reduced gradient method for solving this problem. Our approach solves iteratively a convex quadratic optimization subproblem to calculate a suitable descent direction for all the objective functions, and then use a bisection algorithm to find an optimal stepsize along this direction. We prove, under natural assumptions, that the proposed algorithm is well-defined and converges globally to Pareto critical points of the problem. Finally, this algorithm is implemented in the MATLAB environment and comparative results of numerical experiments are reported.     

A two-stage feasible directions algorithm for nonlinear constrained optimization

We present a feasible directions algorithm, based on Lagrangian concepts, for the solution of the nonlinear programming problem with equality and inequality constraints. At each iteration a descent direction is defined; by modifying it, we obtain a feasible descent direction. The line search procedure assures the global convergence of the method and the feasibility of all the iterates. We prove the global convergence of the algorithm and apply it to the solution of some test problems. Although the present version of the algorithm does not include any second-order information, like quasi-Newton methods, these numerical results exhibit a behavior comparable to that of the best methods known at present for nonlinear programming. Research performed while the author was on a two years appointment at INRIA, Rocquencourt, France, and partially supported by the Brazilian Research Council (CNPq).     

A geometrical interpretation of the Hungarian method

In this paper a geometrical interpretation of the Hungarian method will be given. This special algorithm to solve the dual transportation problem is not restricted to the edges of the convex polyhedron of feasible solutions. Each covering-step can be considered as a determination of a direction of steepest descent, each reduction-step as movement along that direction to a boundary point of the polyhedron. The dimension of the face that will be crossed depends on the covering that is chosen.