Local Behavior of the Newton Method on Two Equivalent Systems from Linear Programming

Newton's method is a fundamental technique underlying many numerical methods for solving systems of nonlinear equations and optimization problems. However, it is often not fully appreciated that Newton's method can produce significantly different behavior when applied to equivalent systems, i.e., problems with the same solution but different mathematical formulations. In this paper, we investigate differences in the local behavior of Newton's method when applied to two different but equivalent systems from linear programming: the optimality conditions of the logarithmic barrier function formulation and the equations in the so-called perturbed optimality conditions. Through theoretical analysis and numerical results, we provide an explanation of why Newton's method performs more effectively on the latter system.     

线性等式约束多目标规划的一个降维算法

本文提出具有线性等式约束多目标规划问题的一个降维算法．当目标函数全是二次或线性但至少有一个二次型时，用线性加权法转化原问题为单目标二次规划，再用降维方法转化为求解一个线性方程组．若目标函数非上述情形，首先用线性加权法将原问题转化为具有线性等式约束的非线性规划，然后，对这一非线性规划的目标函数二次逼近，构成线性等式约束二次规划序列，用降维法求解，直到满足精度要求为止．     

A new algorithm for the huber estimator in linear models

This paper considers algorithms for solving the linear robust regression problem by minimizing the Huber function. In the computational methods for this problem used so far, the scale estimate is adjusted separately. The new algorithm, based on Newton's method, treats both the scale and the location parameters as independent variables. The special form of the Hessian allows for an efficient updating scheme.     

Enlarging the region of convergence of Newton's method for constrained optimization

In this paper, we consider Newton's method for solving the system of necessary optimality conditions of optimization problems with equality and inequality constraints. The principal drawbacks of the method are the need for a good starting point, the inability to distinguish between local maxima and local minima, and, when inequality constraints are present, the necessity to solve a quadratic programming problem at each iteration. We show that all these drawbacks can be overcome to a great extent without sacrificing the superlinear convergence rate by making use of exact differentiable penalty functions introduced by Di Pillo and Grippo (Ref. 1). We also show that there is a close relationship between the class of penalty functions of Di Pillo and Grippo and the class of Fletcher (Ref. 2), and that the region of convergence of a variation of Newton's method can be enlarged by making use of one of Fletcher's penalty functions.This work was supported by the National Science Foundation, Grant No. ENG-79-06332.     

Linearly Constrained Global Optimization and Stochastic Differential Equations

A stochastic algorithm is proposed for the global optimization of nonconvex functions subject to linear constraints. Our method follows the trajectory of an appropriately defined Stochastic Differential Equation (SDE). The feasible set is assumed to be comprised of linear equality constraints, and possibly box constraints. Feasibility of the trajectory is achieved by projecting its dynamics onto the set defined by the linear equality constraints. A barrier term is used for the purpose of forcing the trajectory to stay within the box constraints. Using Laplace’s method we give a characterization of a probability measure (Π) that is defined on the set of global minima of the problem. We then study the transition density associated with the projected diffusion process and show that its weak limit is given by Π. Numerical experiments using standard test problems from the literature are reported. Our results suggest that the method is robust and applicable to large-scale problems.     

Robust regression for mixed Poisson–Gaussian model

This paper focuses on efficient computational approaches to compute approximate solutions of a linear inverse problem that is contaminated with mixed Poisson–Gaussian noise, and when there are additional outliers in the measured data. The Poisson–Gaussian noise leads to a weighted minimization problem, with solution-dependent weights. To address outliers, the standard least squares fit-to-data metric is replaced by the Talwar robust regression function. Convexity, regularization parameter selection schemes, and incorporation of non-negative constraints are investigated. A projected Newton algorithm is used to solve the resulting constrained optimization problem, and a preconditioner is proposed to accelerate conjugate gradient Hessian solves. Numerical experiments on problems from image deblurring illustrate the effectiveness of the methods.     

On the implementation of LIR: the case of simple linear regression with interval data

This paper considers the problem of simple linear regression with interval-censored data. That is, \(n\) pairs of intervals are observed instead of the \(n\) pairs of precise values for the two variables (dependent and independent). Each of these intervals is closed but possibly unbounded, and contains the corresponding (unobserved) value of the dependent or independent variable. The goal of the regression is to describe the relationship between (the precise values of) these two variables by means of a linear function. Likelihood-based Imprecise Regression (LIR) is a recently introduced, very general approach to regression for imprecisely observed quantities. The result of a LIR analysis is in general set-valued: it consists of all regression functions that cannot be excluded on the basis of likelihood inference. These regression functions are said to be undominated. Since the interval data can be unbounded, a robust regression method is necessary. Hence, we consider the robust LIR method based on the minimization of the residuals’ quantiles. For this method, we prove that the set of all the intercept-slope pairs corresponding to the undominated regression functions is the union of finitely many polygons. We give an exact algorithm for determining this set (i.e., for determining the set-valued result of the robust LIR analysis), and show that it has worst-case time complexity \(O(n^{3}\log n)\) . We have implemented this exact algorithm as part of the R package linLIR.     

半定优化与半光滑牛顿算法

本文讨论半光滑牛顿算法的基本概念与其在求解半定优化问题中的应用．特别地，该算法可用于求解线性或非线性半定互补问题．本文同时综述最近在矩阵方程，增广拉格朗日公式和半定优化稳定性方面的、源于半光滑牛顿算法的理论成果．     

An interior-point method for semi-infinite programming problems

This work examines the generalization of a certain interior-point method, namely the method of analytic centers, to semi-infinite linear programming problems. We define an analytic center for these problems and an appropriate norm to examine Newton's method for computing this center. A simple algorithm of order zero is constructed and a convergence proof for that algorithm is given. Finally, we describe a more practical implementation of a predictor-corrector method and give some numerical results. In particular we concentrate on practical integration rules that take care of the specific structure of the integrals.     

Analysis of a self-scaling quasi-Newton method

We study the self-scaling BFGS method of Oren and Luenberger (1974) for solving unconstrained optimization problems. For general convex functions, we prove that the method is globally convergent with inexact line searches. We also show that the directions generated by the self-scaling BFGS method approach Newton's direction asymptotically. This would ensure superlinear convergence if, in addition, the search directions were well-scaled, but we show that this is not always the case. We find that the method has a major drawback: to achieve superlinear convergence it may be necessary to evaluate the function twice per iteration, even very near the solution. An example is constructed to show that the step-sizes required to achieve a superlinear rate converge to 2 and 0.5 alternately.This work was supported by National Science Foundation Grant CCR-9101359, and by the Department of Energy Grant DE-FG02-87ER25047.This work was performed while the author was visiting Northwestern University.     

Dual characterizations of set containments involving uncertain polyhedral sets in Banach spaces with applications

In this paper, we first give a dual characterization for set containments defined by lower semi-continuous and sublinear functions on Banach spaces. Next, we provide dual characterizations for robust polyhedral containments where a robust counterpart of an uncertain polyhedral set is contained in another polyhedral set or a polyhedral set is contained in a robust counterpart of an uncertain polyhedral set. Finally, as an application, we derive Lagrange multiplier characterizations for robust solutions of the robust uncertain linear programming problems.     

Block Based Bivariate Blending Rational Interpolation via Symmetric Branched Continued Fractions

This paper constructs a new kind of block based bivariate blending rational interpolation via symmetric branched continued fractions. The construction process may be outlined as follows. The first step is to divide the original set of support points into some subsets (blocks). Then construct each block by using symmetric branched continued fraction. Finally assemble these blocks by Newton’s method to shape the whole interpolation scheme. Our new method offers many flexible bivariate blending rational interpolation schemes which include the classical bivariate Newton’s polynomial interpolation and symmetric branched continued fraction interpolation as its special cases. The block based bivariate blending rational interpolation is in fact a kind of tradeoff between the purely linear interpolation and the purely nonlinear interpolation. Finally, numerical examples are given to show the effectiveness of the proposed method.     

On generalized Nash equilibrium problems with linear coupling constraints and mixed-integer variables

ABSTRACT

We define and discuss different enumerative methods to compute solutions of generalized Nash equilibrium problems with linear coupling constraints and mixed-integer variables. We propose both branch-and-bound methods based on merit functions for the mixed-integer game, and branch-and-prune methods that exploit the concept of dominance to make effective cuts. We show that under mild assumptions the equilibrium set of the game is finite and we define an enumerative method to compute the whole of it. We show that our branch-and-prune method can be suitably modified in order to make a general equilibrium selection over the solution set of the mixed-integer game. We define an application in economics that can be modelled as a Nash game with linear coupling constraints and mixed-integer variables, and we adapt the branch-and-prune method to efficiently solve it.     

Trust-region problems with linear inequality constraints: exact SDP relaxation,global optimality and robust optimization

The trust-region problem, which minimizes a nonconvex quadratic function over a ball, is a key subproblem in trust-region methods for solving nonlinear optimization problems. It enjoys many attractive properties such as an exact semi-definite linear programming relaxation (SDP-relaxation) and strong duality. Unfortunately, such properties do not, in general, hold for an extended trust-region problem having extra linear constraints. This paper shows that two useful and powerful features of the classical trust-region problem continue to hold for an extended trust-region problem with linear inequality constraints under a new dimension condition. First, we establish that the class of extended trust-region problems has an exact SDP-relaxation, which holds without the Slater constraint qualification. This is achieved by proving that a system of quadratic and affine functions involved in the model satisfies a range-convexity whenever the dimension condition is fulfilled. Second, we show that the dimension condition together with the Slater condition ensures that a set of combined first and second-order Lagrange multiplier conditions is necessary and sufficient for global optimality of the extended trust-region problem and consequently for strong duality. Through simple examples we also provide an insightful account of our development from SDP-relaxation to strong duality. Finally, we show that the dimension condition is easily satisfied for the extended trust-region model that arises from the reformulation of a robust least squares problem (LSP) as well as a robust second order cone programming model problem (SOCP) as an equivalent semi-definite linear programming problem. This leads us to conclude that, under mild assumptions, solving a robust LSP or SOCP under matrix-norm uncertainty or polyhedral uncertainty is equivalent to solving a semi-definite linear programming problem and so, their solutions can be validated in polynomial time.     

An iterative method for linear discrete ill-posed problems with box constraints

Many questions in science and engineering give rise to linear discrete ill-posed problems. Often it is desirable that the computed approximate solution satisfies certain constraints, e.g., that some or all elements of the computed solution be nonnegative. This paper describes an iterative method of active set-type for the solution of large-scale problems of this kind. The method employs conjugate gradient iteration with a stopping criterion based on the discrepancy principle and allows updates of the active set by more than one index at a time.     

求解方程的一类迭代公式

Using the forms of Newton iterative function, the iterative function of Newton's method to handle the problem of multiple roots and the Halley iterative function, we give a class of iterative formulae for solving equations in one variable in this paper and show that their convergence order is at least quadratic. At last we employ our methods to solve some non-linear equations and compare them with Newton's method and Halley's method. Numerical results show that our iteration schemes are convergent if we choose two suitable parametric functions λ(x) and μ(x). Therefore, our iteration schemes are feasible and effective.     

On the choice of parameters for the weighting method in vector optimization

We present a geometrical interpretation of the weighting method for constrained (finite dimensional) vector optimization. This approach is based on rigid movements which separate the image set from the negative of the ordering cone. We study conditions on the existence of such translations in terms of the boundedness of the scalar problems produced by the weighting method. Finally, using recession cones, we obtain the main result of our work: a sufficient condition under which weighting vectors yield solvable scalar problems. An erratum to this article can be found at     

一种新的改进Newton法(英文)

本文给出了求解非线性方程的一种新的改进方法.利用Newton法和Heron平均,将新改进方法与其它一些迭代法作比较.数值结果表明该方法具有一定的实用价值.     

An Extragradient-Based Alternating Direction Method for Convex Minimization

In this paper, we consider the problem of minimizing the sum of two convex functions subject to linear linking constraints. The classical alternating direction type methods usually assume that the two convex functions have relatively easy proximal mappings. However, many problems arising from statistics, image processing and other fields have the structure that while one of the two functions has an easy proximal mapping, the other function is smoothly convex but does not have an easy proximal mapping. Therefore, the classical alternating direction methods cannot be applied. To deal with the difficulty, we propose in this paper an alternating direction method based on extragradients. Under the assumption that the smooth function has a Lipschitz continuous gradient, we prove that the proposed method returns an \(\epsilon \)-optimal solution within \(O(1/\epsilon )\) iterations. We apply the proposed method to solve a new statistical model called fused logistic regression. Our numerical experiments show that the proposed method performs very well when solving the test problems. We also test the performance of the proposed method through solving the lasso problem arising from statistics and compare the result with several existing efficient solvers for this problem; the results are very encouraging.     

一类含时变分布时滞的退化中立型系统的鲁棒稳定性(英文)

考虑含分布时滞的退化中立型系统的鲁棒稳定性.利用算子Ω的稳定性和线性矩阵不等式得到一个新的鲁棒稳定性判据,本判据将中立型时滞、时变离散时滞、时变分布时滞和退化中立型系统一起考虑,相比已有文献具有较低的保守性.利用Matlab可以验证本判据的有效性.