A family of derivative-free conjugate gradient methods for large-scale nonlinear systems of equations

In this paper, we propose a family of derivative-free conjugate gradient methods for large-scale nonlinear systems of equations. They come from two modified conjugate gradient methods [W.Y. Cheng, A two term PRP based descent Method, Numer. Funct. Anal. Optim. 28 (2007) 1217–1230; L. Zhang, W.J. Zhou, D.H. Li, A descent modified Polak–Ribiére–Polyak conjugate gradient method and its global convergence, IMA J. Numer. Anal. 26 (2006) 629–640] recently proposed for unconstrained optimization problems. Under appropriate conditions, the global convergence of the proposed method is established. Preliminary numerical results show that the proposed method is promising.     

Levitin–Polyak well-posedness of generalized quasivariational inequalities with functional constraints

In this paper, we study Levitin–Polyak type well-posedness for generalized quasivariational inequality problems with explicit constraints. Four types of Levitin–Polyak well-posedness will be introduced. Various criteria and characterizations will be derived for these types of Levitin–Polyak well-posedness.     

Localizing Vector Optimization Problems with Application to Welfare Economics

In the present paper, the Polyak’s principle, concerning convexity of the images of small balls through C^{1, 1} mappings, is employed in the study of vector optimization problems. This leads to extend to such a context achievements of local programming, an approach to nonlinear optimization, due to B.T. Polyak, which consists in exploiting the benefits of the convex local behaviour of certain nonconvex problems. In doing so, solution existence and optimality conditions are established for localizations of vector optimization problems, whose data satisfy proper assumptions. Such results are subsequently applied in the analysis of welfare economics, in the case of an exchange economy model with infinite-dimensional commodity space. In such a setting, the localization of an economy yields existence of Pareto optimal allocations, which, under certain additional assumptions, lead to competitive equilibria.     

求解约束优化问题的一个对偶算法

１．引言 考虑下述形式的不等式约束优化问题：其中 ＝０,１,…,ｍ,是连续可微函数．求解（１．１）的数值方法有很多,传统方法有乘子法,序列一次规划方法,等等（见 Ｂｅｒｔｓｅｋａｓ（１９８２）, Ｈａｎ（１９７６, １９７７））．近年来对求解（１．１）的原始－对偶算法的研究已成为非线性规划领域的新的热点,如ＥＩ－Ｂａｋｒｙ,Ｔａｐｉａ,Ｔｓｕｃｈｉｙａ　＆　Ｚｈａｎｇ（１９９６）,Ｙａｍａｓｈｉｔａ（１９９２,１９９６,１９９７）等;尽管这些原始－对偶算法具有好的收敛性质和计算效果,但其算法结构相对…     

Support vector machine via nonlinear rescaling method

In this paper we construct the linear support vector machine (SVM) based on the nonlinear rescaling (NR) methodology (see [Polyak in Math Program 54:177–222, 1992; Polyak in Math Program Ser A 92:197–235, 2002; Polyak and Teboulle in Math Program 76:265–284, 1997] and references therein). The formulation of the linear SVM based on the NR method leads to an algorithm which reduces the number of support vectors without compromising the classification performance compared to the linear soft-margin SVM formulation. The NR algorithm computes both the primal and the dual approximation at each step. The dual variables associated with the given data-set provide important information about each data point and play the key role in selecting the set of support vectors. Experimental results on ten benchmark classification problems show that the NR formulation is feasible. The quality of discrimination, in most instances, is comparable to the linear soft-margin SVM while the number of support vectors in several instances were substantially reduced.     

Levitin–Polyak well-posedness of vector equilibrium problems

In this paper, two types of Levitin–Polyak well-posedness of vector equilibrium problems with variable domination structures are investigated. Criteria and characterizations for two types of Levitin–Polyak well-posedness of vector equilibrium problems are shown. Moreover, by virtue of a gap function for vector equilibrium problems, the equivalent relations between the Levitin–Polyak well-posedness for an optimization problem and the Levitin–Polyak well-posedness for a vector equilibrium problem are obtained. This research was partially supported by the National Natural Science Foundation of China (Grant number: 60574073) and Natural Science Foundation Project of CQ CSTC (Grant number: 2007BB6117).     

Symmetry in RLT-type relaxations for the quadratic assignment and standard quadratic optimization problems

The reformulation–linearization technique (RLT), introduced in [Sherali, H. D., Adams. W. P. (1990). A hierarchy of relaxations between the continuous and convex hull representations for zero-one programming problems. SIAM Journal on Discrete Mathematics 3(3), 411–430], provides a way to compute a hierarchy of linear programming bounds on the optimal values of NP-hard combinatorial optimization problems. In this paper we show that, in the presence of suitable algebraic symmetry in the original problem data, it is sometimes possible to compute level two RLT bounds with additional linear matrix inequality constraints. As an illustration of our methodology, we compute the best-known bounds for certain graph partitioning problems on strongly regular graphs.     

A new family of conjugate gradient methods

In this paper we develop a new class of conjugate gradient methods for unconstrained optimization problems. A new nonmonotone line search technique is proposed to guarantee the global convergence of these conjugate gradient methods under some mild conditions. In particular, Polak–Ribiére–Polyak and Liu–Storey conjugate gradient methods are special cases of the new class of conjugate gradient methods. By estimating the local Lipschitz constant of the derivative of objective functions, we can find an adequate step size and substantially decrease the function evaluations at each iteration. Numerical results show that these new conjugate gradient methods are effective in minimizing large-scale non-convex non-quadratic functions.     

A particle swarm optimization algorithm for makespan and total flowtime minimization in the permutation flowshop sequencing problem

In this paper, a particle swarm optimization algorithm (PSO) is presented to solve the permutation flowshop sequencing problem (PFSP) with the objectives of minimizing makespan and the total flowtime of jobs. For this purpose, a heuristic rule called the smallest position value (SPV) borrowed from the random key representation of Bean [J.C. Bean, Genetic algorithm and random keys for sequencing and optimization, ORSA Journal of Computing 6(2) (1994) 154–160] was developed to enable the continuous particle swarm optimization algorithm to be applied to all classes of sequencing problems. In addition, a very efficient local search, called variable neighborhood search (VNS), was embedded in the PSO algorithm to solve the well known benchmark suites in the literature. The PSO algorithm was applied to both the 90 benchmark instances provided by Taillard [E. Taillard, Benchmarks for basic scheduling problems, European Journal of Operational Research, 64 (1993) 278–285], and the 14,000 random, narrow random and structured benchmark instances provided by Watson et al. [J.P. Watson, L. Barbulescu, L.D. Whitley, A.E. Howe, Contrasting structured and random permutation flowshop scheduling problems: Search space topology and algorithm performance, ORSA Journal of Computing 14(2) (2002) 98–123]. For makespan criterion, the solution quality was evaluated according to the best known solutions provided either by Taillard, or Watson et al. The total flowtime criterion was evaluated with the best known solutions provided by Liu and Reeves [J. Liu, C.R. Reeves, Constructive and composite heuristic solutions to the P∥∑C_i scheduling problem, European Journal of Operational Research 132 (2001) 439–452], and Rajendran and Ziegler [C. Rajendran, H. Ziegler, Ant-colony algorithms for permutation flowshop scheduling to minimize makespan/total flowtime of jobs, European Journal of Operational Research, 155(2) (2004) 426–438]. For the total flowtime criterion, 57 out of the 90 best known solutions reported by Liu and Reeves, and Rajendran and Ziegler were improved whereas for the makespan criterion, 195 out of the 800 best known solutions for the random and narrow random problems reported by Watson et al. were improved by the VNS version of the PSO algorithm.     

Levitin–Polyak Well-Posedness for Optimization Problems with Generalized Equilibrium Constraints

In this paper, we consider Levitin–Polyak well-posedness of parametric generalized equilibrium problems and optimization problems with generalized equilibrium constraints. Some criteria for these types of well-posedness are derived. In particular, under certain conditions, we show that generalized Levitin–Polyak well-posedness of a parametric generalized equilibrium problem is equivalent to the nonemptiness and compactness of its solution set. Finally, for an optimization problem with generalized equilibrium constraints, we also obtain that, under certain conditions, Levitin–Polyak well-posedness in the generalized sense is equivalent to the nonemptiness and compactness of its solution set.     

Exploiting nested inequalities and surrogate constraints

The exploitation of nested inequalities and surrogate constraints as originally proposed in Glover [Glover, F., 1965. A multiphase-dual algorithm for the zero–one integer programming problem. Operations Research 13, 879–919; Glover, F., 1971. Flows in arborescences. Management Science 17, 568–586] has been specialized to multidimensional knapsack problems in Osorio et al. [Osorio, M.A., Glover, F., Hammer, P., 2002. Cutting and surrogate constraint analysis for improved multidimensional knapsack solutions. Annals of Operations Research 117, 71–93]. We show how this specialized exploitation can be strengthened to give better results. This outcome results by a series of observations based on surrogate constraint duality and properties of nested inequalities. The consequences of these observations are illustrated by numerical examples to provide insights into uses of surrogate constraints and nested inequalities that can be useful in a variety of problem settings.     

Solving the Karush–Kuhn–Tucker system of a nonconvex programming problem on an unbounded set

In the papers [G.C. Feng, B. Yu, Combined homotopy interior point method for nonlinear programming problems, in: H. Fujita, M. Yamaguti (Eds.), Advances in Numerical Mathematics; Proceedings of the Second Japan–China Seminar on Numerical Mathematics, in: Lecture Notes in Numerical and Applied Analysis, vol. 14, Kinokuniya, Tokyo, 1995, pp. 9–16; G.C. Feng, Z.H. Lin, B. Yu, Existence of an interior pathway to a Karush–Kuhn–Tucker point of a nonconvex programming problem, Nonlinear Analysis 32 (1998) 761–768; Z.H. Lin, B. Yu, G.C. Feng, A combined homotopy interior point method for convex programming problem, Applied Mathematics and Computation 84 (1997) 193–211], a combined homotopy interior method was presented and global convergence results obtained for nonconvex nonlinear programming when the feasible set is bounded and satisfies the so called normal cone condition. However, for when the feasible set is not bounded, no result has so far been obtained. In this paper, a combined homotopy interior method for nonconvex programming problems on the unbounded feasible set is considered. Under suitable additional assumptions, boundedness of the homotopy path, and hence global convergence, is proven.     

A modified nearly exact method for solving low-rank trust region subproblem

In this paper, we first discuss how the nearly exact (NE) method proposed by Moré and Sorensen [14] for solving trust region (TR) subproblems can be modified to solve large-scale “low-rank” TR subproblems efficiently. Our modified algorithm completely avoids computation of Cholesky factorizations by instead relying primarily on the Sherman–Morrison–Woodbury formula for computing inverses of “diagonal plus low-rank” type matrices. We also implement a specific version of the modified log-barrier (MLB) algorithm proposed by Polyak [17] where the generated log-barrier subproblems are solved by a trust region method. The corresponding direction finding TR subproblems are of the low-rank type and are then solved by our modified NE method. We finally discuss the computational results of our implementation of the MLB method and its comparison with a version of LANCELOT [5] based on a collection extracted from CUTEr [12] of nonlinear programming problems with simple bound constraints.      

Levitin–Polyak well-posedness in set optimization concerning Pareto efficiency

This article aims to elaborate on various notions of Levitin–Polyak well-posedness for set optimization problems concerning Pareto efficient solutions. These notions are categorized into two classes including pointwise and global Levitin–Polyak well-posedness. We give various characterizations of both pointwise and global Levitin–Polyak well-posedness notions for set optimization problems. The hierarchical structure of their relationships is also established. Under suitable conditions on the input data of set optimization problems, we investigate the closedness of Pareto efficient solution sets in which they are different from the weakly efficient ones. Furthermore, we provide sufficient conditions for global Levitin–Polyak well-posedness properties of the reference problems without imposing the information on efficient solution sets.

     

A note on article “A tolerance approach to the fuzzy goal programming problems with unbalanced triangular membership function”

Kim and Whang use a tolerance approach for solving fuzzy goal programming problems with unbalanced membership functions [J.S. Kim, K. Whang, A tolerance approach to the fuzzy goal programming problems with unbalanced triangular membership function, European Journal of Operational Research 107 (1998) 614–624]. In this note it is shown that some results in that article are incorrect. The necessary corrections are proposed.     

Parallel radial basis function methods for the global optimization of expensive functions

We introduce a master–worker framework for parallel global optimization of computationally expensive functions using response surface models. In particular, we parallelize two radial basis function (RBF) methods for global optimization, namely, the RBF method by Gutmann [Gutmann, H.M., 2001a. A radial basis function method for global optimization. Journal of Global Optimization 19(3), 201–227] (Gutmann-RBF) and the RBF method by Regis and Shoemaker [Regis, R.G., Shoemaker, C.A., 2005. Constrained global optimization of expensive black box functions using radial basis functions, Journal of Global Optimization 31, 153–171] (CORS-RBF). We modify these algorithms so that they can generate multiple points for simultaneous evaluation in parallel. We compare the performance of the two parallel RBF methods with a parallel multistart derivative-based algorithm, a parallel multistart derivative-free trust-region algorithm, and a parallel evolutionary algorithm on eleven test problems and on a 6-dimensional groundwater bioremediation application. The results indicate that the two parallel RBF algorithms are generally better than the other three alternatives on most of the test problems. Moreover, the two parallel RBF algorithms have comparable performances on the test problems considered. Finally, we report good speedups for both parallel RBF algorithms when using a small number of processors.     

A nonmonotone trust region method based on nonincreasing technique of weighted average of the successive function values

In this paper we propose a nonmonotone trust region method. Unlike traditional nonmonotone trust region method, the nonmonotone technique applied to our method is based on the nonmonotone line search technique proposed by Zhang and Hager [A nonmonotone line search technique and its application to unconstrained optimization, SIAM J. Optim. 14(4) (2004) 1043–1056] instead of that presented by Grippo et al. [A nonmonotone line search technique for Newton's method, SIAM J. Numer. Anal. 23(4) (1986) 707–716]. So the method requires nonincreasing of a special weighted average of the successive function values. Global and superlinear convergence of the method are proved under suitable conditions. Preliminary numerical results show that the method is efficient for unconstrained optimization problems.     

A general iterative method for equilibrium problems and fixed point problems in Hilbert spaces

The purpose of this paper is to study the strong convergence of a general iterative scheme to find a common element of the set of fixed points of a nonexpansive mapping, the set of solutions of variational inequality for a relaxed cocoercive mapping and the set of solutions of an equilibrium problem. Our results extend the recent results of Takahashi and Takahashi [S. Takahashi, W. Takahashi, Viscosity approximation methods for equilibrium problems and fixed point problems in Hilbert spaces, J. Math. Anal. Appl. 331 (2007) 506–515], Marino and Xu [G. Marino, H.K. Xu, A general iterative method for nonexpansive mappings in Hilbert spaces, J. Math. Anal. Appl. 318 (2006) 43–52], Combettes and Hirstoaga [P.L. Combettes, S.A. Hirstoaga, Equilibrium programming in Hilbert spaces, J. Nonlinear Convex Anal. 6 (2005) 486–491], Iiduka and Takahashi, [H. Iiduka, W. Takahashi, Strong convergence theorems for nonexpansive mappings and inverse-strongly monotone mappings, Nonlinear Anal. 61 (2005) 341–350] and many others.     

A globally convergent,implementable multiplier method with automatic penalty limitation

This paper deals with penalty function and multiplier methods for the solution of constrained nonconvex nonlinear programming problems. Starting from an idea introduced several years ago by Polak, we develop a class of implementable methods which, under suitable assumptions, produce a sequence of points converging to a strong local minimum for the problem, regardless of the location of the initial guess. In addition, for sequential minimization type multiplier methods, we make use of a rate of convergence result due to Bertsekas and Polyak, to develop a test for limiting the growth of the penalty parameter and thereby prevent ill-conditioning in the resulting sequence of unconstrained optimization problems.Research sponsored by the National Science Foundation (RANN) Grant ENV76-04264 and the Joint Services Electronics Research Program Contract F44620-76-C-0100.     

Adaptive cubic regularisation methods for unconstrained optimization. Part I: motivation,convergence and numerical results

An Adaptive Regularisation algorithm using Cubics (ARC) is proposed for unconstrained optimization, generalizing at the same time an unpublished method due to Griewank (Technical Report NA/12, 1981, DAMTP, University of Cambridge), an algorithm by Nesterov and Polyak (Math Program 108(1):177–205, 2006) and a proposal by Weiser et al. (Optim Methods Softw 22(3):413–431, 2007). At each iteration of our approach, an approximate global minimizer of a local cubic regularisation of the objective function is determined, and this ensures a significant improvement in the objective so long as the Hessian of the objective is locally Lipschitz continuous. The new method uses an adaptive estimation of the local Lipschitz constant and approximations to the global model-minimizer which remain computationally-viable even for large-scale problems. We show that the excellent global and local convergence properties obtained by Nesterov and Polyak are retained, and sometimes extended to a wider class of problems, by our ARC approach. Preliminary numerical experiments with small-scale test problems from the CUTEr set show encouraging performance of the ARC algorithm when compared to a basic trust-region implementation.