共轭下降法的全局收敛性

共轭下降法最早由Ｆｌｅｔｃｈｅｒ提出，本文证明了一类非精确线搜索条件能保证共轭下的降法的收敛性，并且构造了反例表明，如果线搜索条件放松，则共轭下降法可能不收敛，此外，我们还得到了与Ｆｌｅｃｈｅｒ－Ｒｅｅｖｅｓ方法有关的一类方法的结论。     

Sparse two-sided rank-one updates for nonlinear equations

The two-sided rank-one (TR1) update method was introduced by Griewank and Walther (2002) for solving nonlinear equations. It generates dense approximations of the Jacobian and thus is not applicable to large-scale sparse problems. To overcome this difficulty, we propose sparse extensions of the TR1 update and give some convergence analysis. The numerical experiments show that some of our extensions are superior to the TR1 update method. Some convergence analysis is also presented.     

Hedge algorithm and Dual Averaging schemes

We show that the Hedge algorithm, a method that is widely used in Machine Learning, can be interpreted as a particular instance of Dual Averaging schemes, which have recently been introduced by Nesterov for regret minimization. Based on this interpretation, we establish three alternative methods of the Hedge algorithm: one in the form of the original method, but with optimal parameters, one that requires less a priori information, and one that is better adapted to the context of the Hedge algorithm. All our modified methods have convergence results that are better or at least as good as the performance guarantees of the vanilla method. In numerical experiments, our methods significantly outperform the original scheme.     

THE QUALOCATION METHOD FOR LINEAR SECOND-ORDER TWO-POINT BOUNDARY-VALUE PROBLEMS

A qualocation method for linear second-order two-point boundary-value problems is developed. The results are extended those in [2]. On the other hand i this method can also be regarded as a discrete, version of the H1-Galerkin method. From this point, our results show that the effect of numerical integration does not decrease the order of convergence.     

On the cubic convergence of the Paardekooper method

We show a simple way how asymptotic convergence results can be conveyed from a simple Jacobi method to a block Jacobi method. Our pilot methods are the well known symmetric Jacobi method and the Paardekooper method for reducing a skew-symmetric matrix to the real Schur form. We show resemblance in the quadratic and cubic convergence estimates, but also discrepances in the asymptotic assumptions. By numerical tests we confirm that our asymptotic assumptions for the Paardekooper method are most general.     

A best-response approach for equilibrium selection in two-player generalized Nash equilibrium problems

ABSTRACT

In this paper, we propose a best-response approach to select an equilibrium in a two-player generalized Nash equilibrium problem. In our model we solve, at each of a finite number of time steps, two independent optimization problems. We prove that convergence of our Jacobi-type method, for the number of time steps going to infinity, implies the selection of the same equilibrium as in a recently introduced continuous equilibrium selection theory. Thus the presented approach is a different motivation for the existing equilibrium selection theory, and it can also be seen as a numerical method. We show convergence of our numerical scheme for some special cases of generalized Nash equilibrium problems with linear constraints and linear or quadratic cost functions.     

A generalization of almost convergence

In this paper, we discuss almost convergent sequences. The concept of almost convergence was first introduced by G. G. Lorentz in 1948. Since then, various types of generalizations for almost convergence have been constructed and studied. Our concern is also to generalize the almost convergence. We construct a generalized almost convergence (GAC) in a natural way through the use of Lorentz-type definition, and give an analogue of Lorentz’s result. It should be noted that the definition of our GAC does not require normed space nor boundedness of sequences, although Lorentz-type definitions usually require it. On the other hand, we also show that our GAC eventually has a certain type of boundedness in spite of its definition.     

Weighted Statistical Relative Invariant Mean in Modular Function Spaces with Related approximation Results

Abstract

The idea of statistical relative convergence on modular spaces has been introduced by Orhan and Demirci. The notion of σ-statistical convergence was introduced by Mursaleen and Edely and further extended based on a fractional order difference operator by Kadak. The concern of this paper is to define two new summability methods for double sequences by combining the concepts of statistical relative convergence and σ-statistical convergence in modular spaces. Furthermore, we give some inclusion relations involving the newly proposed methods and present an illustrative example to show that our methods are nontrivial generalizations of the existing results in the literature. We also prove a Korovkin-type approximation theorem and estimate the rate of convergence by means of the modulus of continuity. Finally, using the bivariate type of Stancu-Schurer-Kantorovich operators, we display an example such that our approximation results are more powerful than the classical, statistical, and relative modular cases of Korovkin-type approximation theorems.     

The solution methods for the largest eigenvalue (singular value) of nonnegative tensors and convergence analysis

In this paper we study two solution methods for finding the largest eigenvalue (singular value) of general square (rectangular) nonnegative tensors. For a positive tensor, one can find the largest eigenvalue (singular value) based on the properties of the positive tensor and the power-type method. While for a general nonnegative tensor, we use a series of decreasing positive perturbations of the original tensor and repeatedly recall power-type method for finding the largest eigenvalue (singular value) of a positive tensor with an inexact strategy. We prove the convergence of the method for the general nonnegative tensor. Under a certain assumption, the computing complexity of the method is established. Motivated by the interior-point method for the convex optimization, we put forward a one-step inner iteration power-type method, whose convergence is also established under certain assumption. Additionally, by using embedding technique, we show the relationship between the singular values of the rectangular tensor and the eigenvalues of related square tensor, which suggests another way for finding the largest singular value of nonnegative rectangular tensor besides direct power-type method for this problem. Finally, numerical examples of our algorithms are reported, which demonstrate the convergence behaviors of our methods and show that the algorithms presented are promising.     

Enlarging the region of convergence of Newton's method for constrained optimization

In this paper, we consider Newton's method for solving the system of necessary optimality conditions of optimization problems with equality and inequality constraints. The principal drawbacks of the method are the need for a good starting point, the inability to distinguish between local maxima and local minima, and, when inequality constraints are present, the necessity to solve a quadratic programming problem at each iteration. We show that all these drawbacks can be overcome to a great extent without sacrificing the superlinear convergence rate by making use of exact differentiable penalty functions introduced by Di Pillo and Grippo (Ref. 1). We also show that there is a close relationship between the class of penalty functions of Di Pillo and Grippo and the class of Fletcher (Ref. 2), and that the region of convergence of a variation of Newton's method can be enlarged by making use of one of Fletcher's penalty functions.This work was supported by the National Science Foundation, Grant No. ENG-79-06332.     

Chebyshev center based column generation

The classical column generation approach often shows a very slow convergence. Many different acceleration techniques have been proposed recently to improve the convergence. Here, we briefly survey these methods and propose a novel algorithm based on the Chebyshev center of the dual polyhedron. The Chebyshev center can be obtained by solving a linear program; consequently, the proposed method can be applied with small modifications on the classical column generation procedure. We also show that the performance of our algorithm can be enhanced by introducing proximity parameters which enable the position of the Chebyshev center to be adjusted. Numerical experiments are conducted on the binpacking, vehicle routing problem with time windows, and the generalized assignment problem. The computational results of these experiments demonstrate the effectiveness of our proposed method.     

Local linear convergence analysis of Primal–Dual splitting methods

In this paper, we study the local linear convergence properties of a versatile class of Primal–Dual splitting methods for minimizing composite non-smooth convex optimization problems. Under the assumption that the non-smooth components of the problem are partly smooth relative to smooth manifolds, we present a unified local convergence analysis framework for these methods. More precisely, in our framework, we first show that (i) the sequences generated by Primal–Dual splitting methods identify a pair of primal and dual smooth manifolds in a finite number of iterations, and then (ii) enter a local linear convergence regime, which is characterized based on the structure of the underlying active smooth manifolds. We also show how our results for Primal–Dual splitting can be specialized to cover existing ones on Forward–Backward splitting and Douglas–Rachford splitting/ADMM (alternating direction methods of multipliers). Moreover, based on these obtained local convergence analysis result, several practical acceleration techniques are discussed. To exemplify the usefulness of the obtained result, we consider several concrete numerical experiments arising from fields including signal/image processing, inverse problems and machine learning. The demonstration not only verifies the local linear convergence behaviour of Primal–Dual splitting methods, but also the insights on how to accelerate them in practice.     

A NEW SQP-FILTER METHOD PROGRAMMING FOR SOLVING NONLINEAR PROBLEMS

In [4], Fletcher and Leyffer present a new method that solves nonlinear programming problems without a penalty function by SQP-Filter algorithm. It has attracted much attention due to its good numerical results. In this paper we propose a new SQP-Filter method which can overcome Maratos effect more effectively. We give stricter acceptant criteria when the iterative points are far from the optimal points and looser ones vice-versa. About this new method, the proof of global convergence is also presented under standard assumptions. Numerical results show that our method is efficient.     

Lower convergence of approximate solutions to vector quasi-variational problems

In this article, various types of approximate solutions for vector quasi-variational problems in Banach spaces are introduced. Motivated by [M.B. Lignola, J. Morgan, On convergence results for weak efficiency in vector optimization problems with equilibrium constraints, J. Optim. Theor. Appl. 133 (2007), pp. 117–121] and in line with the results obtained in optimization, game theory and scalar variational inequalities, our aim is to investigate lower convergence properties (in the sense of Painlevé–Kuratowski) for such approximate solution sets in the presence of perturbations on the data. Sufficient conditions are obtained for the lower convergence of ‘strict approximate’ solution sets but counterexamples show that, in general, the other types of solutions do not lower converge. Moreover, we prove that any exact solution to the limit problem can be obtained as the limit of a sequence of approximate solutions to the perturbed problems.     

Global Convergence of Algorithms with Nonmonotone Line Search Strategy in Unconstrained Optimization

In this paper we state some nonmonotone line search strategies for unconstrained optimization algorithms. Abstracting from the concrete line search strategy we prove two general convergence results. Using this theory we can show the global convergence of the BFGS method with nonmonotone line search strategy. In contrast to some former results about nonmonotone line search strategies, both our convergence results and their proofs are natural generalizations of known results for the monotone case.     

Finite element algorithm based on high-order time approximation for time fractional convection-diffusion equation

In this paper, finite element method with high-order approximation for time fractional derivative is considered and discussed to find the numerical solution of time fractional convection-diffusion equation. Some lemmas are introduced and proved, further the stability and error estimates are discussed and analyzed, respectively. The convergence result $O(h^{r+1}+\tau^{3-\alpha})$ can be derived, which illustrates that time convergence rate is higher than the order $(2-\alpha)$ derived by $L1$-approximation. Finally, to validate our theoretical results, some computing data are provided.     

On the convergence of nonstationary column-oriented version of algebraic iterative methods

Recently, Elfving, Hansen, and Nikazad introduced a successful nonstationary block-column iterative method for solving linear system of equations based on flagging idea (called BCI-F). Their numerical tests show that the column-action method provides a basis for saving computational work using flagging technique in BCI algorithm. However, they did not present a general convergence analysis. In this paper, we give a convergence analysis of BCI-F. Furthermore, we consider a fully flexible version of block-column iterative method (FBCI), in which the relaxation parameters and weight matrices can be updated in each iteration and the column partitioning of coefficient matrix is allowed to update in each cycle. We also provide the convergence analysis of algorithm FBCI under mild conditions.     

Convergence and Stability of the Truncated Euler-Maruyama Method for Stochastic Differential Equations with Piecewise Continuous Arguments

In this paper, we develop the truncated Euler-Maruyama (EM) method for stochastic differential equations with piecewise continuous arguments (SDEPCAs), and consider the strong convergence theory under the local Lipschitz condition plus the Khasminskii-type condition. The order of convergence is obtained. Moreover, we show that the truncated EM method can preserve the exponential mean square stability of SDEPCAs. Numerical examples are provided to support our conclusions.     

The residual method for regularizing ill-posed problems

Although the residual method, or constrained regularization, is frequently used in applications, a detailed study of its properties is still missing. This sharply contrasts the progress of the theory of Tikhonov regularization, where a series of new results for regularization in Banach spaces has been published in the recent years. The present paper intends to bridge the gap between the existing theories as far as possible. We develop a stability and convergence theory for the residual method in general topological spaces. In addition, we prove convergence rates in terms of (generalized) Bregman distances, which can also be applied to non-convex regularization functionals.We provide three examples that show the applicability of our theory. The first example is the regularized solution of linear operator equations on L^p-spaces, where we show that the results of Tikhonov regularization generalize unchanged to the residual method. As a second example, we consider the problem of density estimation from a finite number of sampling points, using the Wasserstein distance as a fidelity term and an entropy measure as regularization term. It is shown that the densities obtained in this way depend continuously on the location of the sampled points and that the underlying density can be recovered as the number of sampling points tends to infinity. Finally, we apply our theory to compressed sensing. Here, we show the well-posedness of the method and derive convergence rates both for convex and non-convex regularization under rather weak conditions.