The convergence of subspace trust region methods

The trust region method is an effective approach for solving optimization problems due to its robustness and strong convergence. However, the subproblem in the trust region method is difficult or time-consuming to solve in practical computation, especially in large-scale problems. In this paper we consider a new class of trust region methods, specifically subspace trust region methods. The subproblem in these methods has an adequate initial trust region radius and can be solved in a simple subspace. It is easier to solve than the original subproblem because the dimension of the subproblem in the subspace is reduced substantially. We investigate the global convergence and convergence rate of these methods.     

Affine scaling interior Levenberg–Marquardt method for bound-constrained semismooth equations under local error bound conditions

We develop and analyze a new affine scaling Levenberg–Marquardt method with nonmonotonic interior backtracking line search technique for solving bound-constrained semismooth equations under local error bound conditions. The affine scaling Levenberg–Marquardt equation is based on a minimization of the squared Euclidean norm of linear model adding a quadratic affine scaling matrix to find a solution that belongs to the bounded constraints on variable. The global convergence results are developed in a very general setting of computing trial directions by a semismooth Levenberg–Marquardt method where a backtracking line search technique projects trial steps onto the feasible interior set. We establish that close to the solution set the affine scaling interior Levenberg–Marquardt algorithm is shown to converge locally Q-superlinearly depending on the quality of the semismooth and Levenberg–Marquardt parameter under an error bound assumption that is much weaker than the standard nonsingularity condition, that is, BD-regular condition under nonsmooth case. A nonmonotonic criterion should bring about speed up the convergence progress in the contours of objective function with large curvature.     

Two strong convergence theorems for Bregman strongly nonexpansive operators in reflexive Banach spaces

We study the convergence of two iterative algorithms for finding common fixed points of finitely many Bregman strongly nonexpansive operators in reflexive Banach spaces. Both algorithms take into account possible computational errors. We establish two strong convergence theorems and then apply them to the solution of convex feasibility, variational inequality and equilibrium problems.     

On the global convergence of path-following methods to determine all solutions to a system of nonlinear equations

In this paper we prove a general theorem stating a sufficient condition for the inverse image of a point under a continuously differentiable map from ℝ ⁿ to ℝ ^k to be connected. This result is applied to the trajectories generated by the Newton flow. Several examples demonstrate the applicability of the results to nontrivial problems. This work was supported by the Deutsche Forschungsgemeinschaft.     

On the convergence of a new trust region algorithm

Summary. In this paper we present a new trust region algorithm for general nonlinear constrained optimization problems. The algorithm is based on the exact penalty function. Under very mild conditions, global convergence results for the algorithm are given. Local convergence properties are also studied. It is shown that the penalty parameter generated by the algorithm will be eventually not less than the norm of the Lagrange multipliers at the accumulation point. It is proved that the method is equivalent to the sequential quadratic programming method for all large , hence superlinearly convergent results of the SQP method can be applied. Numerical results are also reported. Received March 21, 1993     

The unique minimality of an averaging projection

In this paper we will prove that an averaging projection P _a : K (H) → Y, given by the formula , is the only norm-one projection. Here, K (H) is a space of compact operators on a separable real Hilbert space H, and Y is the subspace of K(H) consisting of all symmetric operators. Author’s address: Faculty of Applied Mathematics, AGH University of Science and Technology, al. Mickiewicza 30, 30-059 Kraków, Poland     

On the convergence of von Neumann''s alternating projection algorithm for two sets

We give several unifying results, interpretations, and examples regarding the convergence of the von Neumann alternating projection algorithm for two arbitrary closed convex nonempty subsets of a Hilbert space. Our research is formulated within the framework of Fejér monotonicity, convex and set-valued analysis. We also discuss the case of finitely many sets.     

A general iterative method for addressing mixed equilibrium problems and optimization problems

In this paper, we introduce a new general iterative method for finding a common element of the set of solutions of a mixed equilibrium problem (MEP), the set of fixed points of an infinite family of nonexpansive mappings and the set of solutions of variational inequalities for a ξ-inverse-strongly monotone mapping in Hilbert spaces. Furthermore, we establish the strong convergence theorem for the iterative sequence generated by the proposed iterative algorithm under some suitable conditions, which solves some optimization problems. Our results extend and improve the recent results of Yao et al. [Y. Yao, M.A. Noor, S. Zainab, Y.C. Liou, Mixed equilibrium problems and optimization problems, J. Math. Anal. Appl. 354 (2009) 319-329; Y. Yao, M. A. Noor, Y.C. Liou, On iterative methods for equilibrium problems, Nonlinear Anal. 70 (1) (2009) 479-509] and many others.     

A superlinearly convergent strongly sub-feasible SSLE-type algorithm with working set for nonlinearly constrained optimization

In this paper, by means of a new efficient identification technique of active constraints and the method of strongly sub-feasible direction, we propose a new sequential system of linear equations (SSLE) algorithm for solving inequality constrained optimization problems, in which the initial point is arbitrary. At each iteration, we first yield the working set by a pivoting operation and a generalized projection; then, three or four reduced linear equations with a same coefficient are solved to obtain the search direction. After a finite number of iterations, the algorithm can produced a feasible iteration point, and it becomes the method of feasible directions. Moreover, after finitely many iterations, the working set becomes independent of the iterates and is essentially the same as the active set of the KKT point. Under some mild conditions, the proposed algorithm is proved to be globally, strongly and superlinearly convergent. Finally, some preliminary numerical experiments are reported to show that the algorithm is practicable and effective.     

Improved local convergence of Newton’s method under weak majorant condition

We provide a local convergence analysis for Newton’s method under a weak majorant condition in a Banach space setting. Our results provide under the same information a larger radius of convergence and tighter error estimates on the distances involved than before [14]. Special cases and numerical examples are also provided in this study.     

Mathematical foundations of nonsmooth embedding methods

This paper establishes a mathematical foundation for application of the well known classical embedding approach to a class of nonsmooth functions, using a recently developed analogue of the derivative in cases where the functions involved fail to be differentiable in the usual sense. As part of this development we show how to obtain an extension to this case of the classical Hadamard theorem giving conditions for a map of ⁿ to itself to be a homeomorphism.The research reported here was sponsored by the National Science Foundation under Grant CCR-8801489, and by the Air Force Systems Command, USAF, under Grants AFOSR-88-0090 and AFOSR-89-0058. The US Government has certain rights in this material, and is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon.     

On the convergence of the Newton/log-barrier method

In the Newton/log-barrier method, Newton steps are taken for the log-barrier function for a fixed value of the barrier parameter until a certain convergence criterion is satisfied. The barrier parameter is then decreased and the Newton process is repeated. A naive analysis indicates that Newton’s method does not exhibit superlinear convergence to the minimizer of each instance of the log-barrier function until it reaches a very small neighborhood, namely within O(μ²) of the minimizer, where μ is the barrier parameter. By analyzing the structure of the barrier Hessian and gradient in terms of the subspace of active constraint gradients and the associated null space, we show that this neighborhood is in fact much larger –O(μ^σ) for any σ∈(1,2] – thus explaining why reasonably fast local convergence can be attained in practice. Moreover, we show that the overall convergence rate of the Newton/log-barrier algorithm is superlinear in the number of function/derivative evaluations, provided that the nonlinear program is formulated with a linear objective and that the schedule for decreasing the barrier parameter is related in a certain way to the step length and convergence criteria for each Newton process. Received: October 10, 1997 / Accepted: September 10, 2000?Published online February 22, 2001     

A novel method for robust minimisation of univariate functions with quadratic convergence

We describe a novel method for minimisation of univariate functions which exhibits an essentially quadratic convergence and whose convergence interval is only limited by the existence of near maxima. Minimisation is achieved through a fixed-point iterative algorithm, involving only the first and second-order derivatives, that eliminates the effects of near inflexion points on convergence, as usually observed in other minimisation methods based on the quadratic approximation. Comparative numerical studies against the standard quadratic and Brent's methods demonstrate clearly the high robustness, high precision and convergence rate of the new method, even when a finite difference approximation is used in the evaluation of the second-order derivative.     

Asymptotic behavior of coupled dynamical systems with multiscale aspects

We study the asymptotic behavior, as time variable t goes to +∞, of nonautonomous dynamical systems involving multiscale features. As a benchmark case, given H a general Hilbert space, and two closed convex functions, and β a function of t which tends to +∞ as t goes to +∞, we consider the differential inclusion

     

On the nonnegative self-adjoint solutions of the operator Riccati equation for infinite dimensional systems

The nonnegative self-adjoint solutions of the operator Riccati equation (ORE) are studied for stabilizable semigroup Hilbert state space systems with bounded sensing and control. Basic properties of the maximal solution of the ORE are investigated: stability of the corresponding closed loop system, structure of the kernel, Hilbert-Schmidt property. Similar properties are obtained for the nonnegative self-adjoint solutions of the ORE. The analysis leads to a complete classification of all nonnegative self-adjoint solutions, which is based on a bijection between these solutions and finite dimensional semigroup invariant subspaces contained in the antistable unobservable subspace.     

Strong convergence theorems obtained by a generalized projections hybrid method for families of mappings in Banach spaces

Let C be a nonempty, closed and convex subset of a uniformly convex and smooth Banach space and let {T_n} be a family of mappings of C into itself such that the set of all common fixed points of {T_n} is nonempty. We consider a sequence {x_n} generated by the hybrid method by generalized projection in mathematical programming. We give conditions on {T_n} under which {x_n} converges strongly to a common fixed point of {T_n} and generalize the results given in [12], [14], [13] and [11].     

On diagonally structured problems in unconstrained optimization using an inexact super Halley method

We consider solving the unconstrained minimization problem using an iterative method derived from the third order super Halley method. Each iteration of the super Halley method requires the solution of two linear systems of equations. We show a practical implementation using an iterative method to solve the linear systems. This paper introduces an array of arrays (jagged) data structure for storing the second and third derivative of a multivariate function and suitable termination criteria for the (inner) iterative method to achieve a cubic rate of convergence. Using a jagged compressed diagonal storage of the Hessian matrices and for the tensor, numerical results show that storing the diagonals are more efficient than the row or column oriented approach when we use an iterative method for solving the linear systems of equations.     

A new family of conjugate gradient methods

In this paper we develop a new class of conjugate gradient methods for unconstrained optimization problems. A new nonmonotone line search technique is proposed to guarantee the global convergence of these conjugate gradient methods under some mild conditions. In particular, Polak–Ribiére–Polyak and Liu–Storey conjugate gradient methods are special cases of the new class of conjugate gradient methods. By estimating the local Lipschitz constant of the derivative of objective functions, we can find an adequate step size and substantially decrease the function evaluations at each iteration. Numerical results show that these new conjugate gradient methods are effective in minimizing large-scale non-convex non-quadratic functions.     

Strong convergence of viscosity approximation methods for finding zeros of accretive operators in Banach spaces

In this paper, we introduce a composite iterative scheme by viscosity approximation method for finding a zero of an accretive operator in Banach spaces. Then, we establish strong convergence theorems for the composite iterative scheme. The main theorems improve and generalize the recent corresponding results of Kim and Xu [T.H. Kim, H.K. Xu, Strong convergence of modified Mann iterations, Nonlinear Anal. 61 (2005) 51-60], Qin and Su [X. Qin, Y. Su, Approximation of a zero point of accretive operator in Banach spaces, J. Math. Anal. Appl. 329 (2007) 415-424] and Xu [H.K. Xu, Strong convergence of an iterative method for nonexpansive and accretive operators, J. Math. Anal. Appl. 314 (2006) 631-643] as well as Aoyama et al. [K. Aoyama, Y Kimura, W. Takahashi, M. Toyoda, Approximation of common fixed points of a countable family of nonexpansive mappings in Banach spaces, Nonlinear Anal. 67 (2007) 2350-2360], Benavides et al. [T.D. Benavides, G.L. Acedo, H.K. Xu, Iterative solutions for zeros of accretive operators, Math. Nachr. 248-249 (2003) 62-71], Chen and Zhu [R. Chen, Z. Zhu, Viscosity approximation fixed points for nonexpansive and m-accretive operators, Fixed Point Theory and Appl. 2006 (2006) 1-10] and Kamimura and Takahashi [S. Kamimura, W. Takahashi, Approximation solutions of maximal monotone operators in Hilberts spaces, J. Approx. Theory 106 (2000) 226-240].