Convergence of Perry and Shanno's Memoryless Quasi-Newton Method for Nonconvex Optimization Problems

1.IntroductionWeconsidertheunconstrainedoptimizationproblem:minj(x).(1.1)acRewheref:R"-- Riscontinuouslydifferelltiable.PerryandShanno'smemorylesssquasiNewtonmethodisoftenusedtosolvetheproblem(1.1)whennislarge.ItwasoriginatedfromtheworksofPerry(1977[l])an…     

A ROBUST SUPERLINEARLY CONVERGENT ALGORITHM FOR LINEARLY CONSTRAINED OPTIMIZATION PROBLEMS UNDER DEGENERACY

1.IntroductionTheproblemconsideredinthispaperiswhereX={xER"laTx5hi,jEI={l,.'.,m}},ajeR"(jEI)areallcolumn*ThisresearchissupportedbytheNationalNaturalSciencesFoundationofChinaandNaturalSciencesFoundationofHunanProvince.vectors,hiERI(j6I)areallscalars,andf:R"-- Risacontinuouslydifferentiablefunction.Weonlyconsiderinequalityconstraintsheresinceanyequalitycanbeexpressedastwoinequalities.Withoutassumingregularityofthelinearconstraints,thereisnotanydifficultyinextendingtheresultstothegenera…     

A feasible and superlinear algorithm for inequality constrained minimization problems

1. IntroductionConsider the optimization problemmin {f(x): gi(x) 5 0, j E I; x E R"}, (l)where f(x), gi(x): Rad - R, j E I ~ {1, 2,...,m}.It is well known that one of the most effective methods to solve problem (1) is thesequential quadratic programming (i.e., SoP) (see [1--6]), due to its property of superlinearconvergence. Especially in recent years, in order to perfect SoP both in theory and application, there have many papers, such as [7--10], been published. These papers focus mainly…     

Orbital stability of solitary waves for the Klein-Gordon-Zakharov equations

1.IntroductionInthetheoreticalinvestigationsofthedynamicsofstrongLangmuirturbulenceinplasmaphysics,varioustypesofZakharovequationstakeanimportantrole(see[3--8]).Intillspaper,weconsiderthefollowingKlein-Gordon-Zakharovequations:{:ti:::::;.;;:~"aiR,(11)withuacomplexfunctionandnarealfunction.Thelocalandglobalekistenceoftheinitialvalueproblemfor(1.l)wasconsideredin[4,6].Inthispaper,weconsidertheorbitalstabilityofthesolitarywavesof(1.l).Byapplyingtheabstracttheoryof[1,2]anddetailedspectralanalys…     

LONG TIME ASYMPTOTIC BEHAVIOR OF SOLUTION OF DIFFERENCE SCHEME FOR A SEMILINEAR PARABOLIC EQUATION (Ⅱ)

1.IntroductionLetfi~(0,l),f(x)EH'(fl),"o(x)EH'(fl)nHI(fl),of(u)=u3,weconsiderthefollowinginitial-boundaryvalue'problem:Bytheusualapproach[1--4]wecangettheglobalexistenceofthesolutionof(1.1),furthermore,thesolutionof(1.1)convergestothesolutionofthefollowingstationaryproblem(1.2)ast-co.l)TheProjectissupportedbyChillaPostdoctoralScienceFoundationandtheScienceFoundationofAcademyofEngineeringPhysicsofChinaIn[6]5[7]?theauthorsconsideredtheexplicitschemefor(l.l)asf(x)=0andonlytheestimateinLZf…     

EXISTENCE OF SOLUTIONS IN THE FUTURE FOR A CLASS OF NONLINEAR DIFFERENTIAL SYSTEMS

1 IntroductionThe problem cited in the title above is an old problem, and has been studiedby many authors. Some good results can be found in [1-5] and referencesquoted therein.In this paper we are concerned with this problem for the nonlinear differential systemJ 1 -- ':)>.--,,<,> ~ F(x), (1 l)where E,P, F, q and g are continuous real functions defined on R. Clearly, ifputting E(z) = z, p(y) = y and q(y) H 1, then (1.1) gives the well-knownLi6nard system{ 1:,.:<">, (l 2)Therefore, (1.1) is…     

A NONMONOTONIC TRUST REGION TECHNIQUE FOR NONLINEAR CONSTRAINED OPTIMIZATION

1.IntroductionConsiderthenonlinearequalityc0nstrainedoptimizati0npr0blemminf(x),s.t.c(x)=0(l-1)wheref:R"-R1andc:R"-R",m5n.RecentlyreducedHessianmethodsarepr0p0sedt0s0lvethispr0blem.C0lemanandConn[1l,andNocedalandOvertonl6lproposedseparatelysimilarquasi-Newt0nmethodsusingapproximatereducedHessian,However,suchmethodscann0tensuregl0balconvergenceandthereforeareavailableonlywhentheinitialstartsaregooden0ugh.TwobaJsicappr0aches,namelythelinesearchandthetrustregion,havebeende-velopedinordertoen…     

A NEW EXTENSION OF THE HIGH-ORDER VIRASORO ALGEBRA

1 IntroductionVirasoro algebra and W algebra have been widely investigated by both physicists andmahematicians. Virasoro algebra can be thought of as the central extended algebra Of diffeOmorphisms Di$S') on the circle S1. In Ref[1], ftaPpat et al. extended the notion ofloop algebra g(S') to algebra g(M), M being a closed surface with dim M > 1, and gave thefolowing commutation relations:c,[Lk,l, L.,.] = [mp(k -- T) -- ftO(l -- 8)]Lk .,l , ti ll ghO(k -- j) -- fl0(l -- j)lok .,O8l ,,…     

CONTINUITY FOR MAXIMAL COMMUTATORS OF BOCHNER-RIESZ OPERATORS WITH BMO FUNCTIONS

1 IntroductionLet b E BMO(R") and let T be a standard Calder6n--Zygnmnd operator. The commutator[6, T generated by b and T is defined by[5, T]f(x) = b(x)Tf(x) -- T(5f)(x).A famous result of Coifmau, Rochberg and Weiss in [1] states that the operator [6, T] is boundedon Lp(R") for p E (1,oc). It is well-known that the standard Calder6n-Zygmund singularintegrals map continuously HP(R"), the standard Hardy space, into Lp(R") and HP,co (R" ),the standaxd weak Hardy space, into LPtco(…     

Decay of global solutions of two nonlinear evolution equations

1.IntroductionTherehavebeenconsiderableliteratuxeonthedecayofsolutionstothebestialvalueproblemsforsomenonlinearevolutionequations[3,4,6,7,161.Undercertainassumptions,LZdecayandLoodecayofsolutionstotheseproblemswereestablished.Thereadersinterestedcanfindsuchworksinourreferences.OurillterestisfocusedonthedecayofsolutionsoftheinitialvalueproblemsfornonlinearBenjamin--OnthBurgers(BOB)l"'19--21]andSchlodinger-Burgers(SB)equationwhereHisHilberttransform,definedbyWewallttoshowthattheLZandLoon…     

A hybrid optimization approach for multi-level capacitated lot-sizing problems

Solving multi-level capacitated lot-sizing problems is still a challenging task, in spite of increasing computational power and faster algorithms. In this paper a new approach combining an ant-based algorithm with an exact solver for (mixed-integer) linear programs is presented. A MAX–MIN ant system is developed to determine the principal production decisions, a LP/MIP solver is used to calculate the corresponding production quantities and inventory levels. Two different local search methods and an improvement strategy based on reduced mixed-integer problems are developed and integrated into the ant algorithm. This hybrid approach provides superior results for small and medium-sized problems in comparison to the existing approaches in the literature. For large-scale problems the performance of this method is among the best.     

求解大规模极大极小问题的光滑化三项共轭梯度算法

基于指数罚函数,对最近提出的一种求解无约束优化问题的三项共轭梯度法进行了修正,并用它求解更复杂的大规模极大极小值问题.证明了该方法生成的搜索方向对每一个光滑子问题是充分下降方向,而且与所用的线搜索规则无关.以此为基础,设计了求解大规模极大极小值问题的算法,并在合理的假设下,证明了算法的全局收敛性.数值实验表明,该算法优于文献中已有的类似算法.     

Modified subspace limited memory BFGS algorithm for large-scale bound constrained optimization

In this paper, a subspace limited memory BFGS algorithm for solving large-scale bound constrained optimization problems is developed. It is modifications of the subspace limited memory quasi-Newton method proposed by Ni and Yuan [Q. Ni, Y.X. Yuan, A subspace limited memory quasi-Newton algorithm for large-scale nonlinear bound constrained optimization, Math. Comput. 66 (1997) 1509–1520]. An important property of our proposed method is that more limited memory BFGS update is used. Under appropriate conditions, the global convergence of the method is established. The implementations of the method on CUTE test problems are presented, which indicate the modifications are beneficial to the performance of the algorithm.     

Hierarchical generating method for large-scale multiobjective systems

This paper investigates large-scale multiobjective systems in the context of a general hierarchical generating method which considers the problem of how to find the set of all noninferior solutions by decomposition and coordination. A new, unified framework of the hierarchical generating method is developed by integrating the envelope analysis approach and the duality theory that is used in multiobjective programming. In this scheme, the vector-valued Lagrangian and the duality theorem provide the basis of a decomposition of the overall multiobjective system into several multiobjective subsystems, and the envelope analysis gives an efficient approach to deal with the coordination at a high level. The following decomposition-coordination schemes for different problems are developed: (i) a spatial decomposition and envelope coordination algorithm for large-scale multiobjective static systems; (ii) a temporal decomposition and envelope coordination algorithm for multiobjective dynamic systems; and (iii) a three-level structure algorithm for large-scale multiobjective dynamic systems.This work was supported by NSF Grant No. CEE-82-11606.     

An investigation of interior-point and block pivoting algorithms for large-scale symmetric monotone linear complementarity problems

In this paper we describe a computational study of block principal pivoting (BP) and interior-point predictor-corrector (PC) algorithms for the solution of large-scale linear complementarity problems (LCP) with symmetric positive definite matrices. This study shows that these algorithms are in general quite appropriate for this type of LCPs. The BP algorithm does not seem to be sensitive to bad scaling and degeneracy of the unique solution of the LCP, while these aspects have some effect on the performance of the PC algorithm. On the other hand, the BP method has not performed well in two LCPs with ill-conditioned matrices for which the PC algorithm has behaved quite well.A hybrid algorithm combining these two techniques is also introduced and seems to be the most robust procedure for the solution of large-scale LCPs with symmetric positive definite matrices.Support of this work has been provided by the Instituto de Telecomunicações.     

稀疏线性规划研究

稀疏线性规划在金融计算、工业生产、装配调度等领域应用十分广泛.本文首先给出稀疏线性规划问题的一般模型并证明问题是NP困难问题;其次采用交替方向乘子法(ADMM)求解该问题;最后证明了算法在近似问题上的收敛性.数值实验表明,算法在大规模数值算例上的表现优于已有的混合遗传算法;同时通过对金融实例的计算验证了算法及模型在稀疏投资组合问题上的有效性.     

Multiobjective big data optimization based on a hybrid salp swarm algorithm and differential evolution

This paper developed a multiobjective Big Data optimization approach based on a hybrid salp swarm algorithm and the differential evolution algorithm. The role of the differential evolution algorithm is to enhance the capability of the feature exploitation of the salp swarm algorithm because the operators of the differential evolution algorithm are used as local search operators. In general, the proposed method contains three stages. In the first stage, the population is generated, and the archive is initialized. The second stage updates the solutions using the hybrid salp swarm algorithm and the differential evolution algorithm, and the final stage determines the nondominated solutions and updates the archive. To assess the performance of the proposed approach, a series of experiments were performed. A set of single-objective and multiobjective problems from the 2015 Big Data optimization competition were tested; the dataset contained data with and without noise. The results of our experiments illustrated that the proposed approach outperformed other approaches, including the baseline nondominated sorting genetic algorithm, on all test problems. Moreover, for single-objective problems, the score value of the proposed method was better than that of the traditional multiobjective salp swarm algorithm. When compared with both algorithms, that is, the adaptive DE algorithm with external archive and the hybrid multiobjective firefly algorithm, its score was the largest. In contrast, for the multiobjective functions, the scores of the proposed algorithm were higher than that of the fireworks algorithm framework.     

An algorithm for hierarchical optimization of large-scale problems with nested structure

This paper describes a general concept and a particular optimization algorithm for solving a class of large-scale nonlinear programming problems with a nested block-angular structured system of linear constraints with coupling variables. A primal optimization algorithm is developed, which is based on the recursive application of the partitioning concept to the nested structure in combination with a feasible directions method. The special column by column application of this partitioning concept finally leads to a very clear and efficient algorithm for nested problems, which is called ‘successive partitioning method’. It is shown that the reduced-gradient method can be represented as a special application of the concept.     

A hybrid optimization technique coupling an evolutionary and a local search algorithm

Evolutionary algorithms are robust and powerful global optimization techniques for solving large-scale problems that have many local optima. However, they require high CPU times, and they are very poor in terms of convergence performance. On the other hand, local search algorithms can converge in a few iterations but lack a global perspective. The combination of global and local search procedures should offer the advantages of both optimization methods while offsetting their disadvantages. This paper proposes a new hybrid optimization technique that merges a genetic algorithm with a local search strategy based on the interior point method. The efficiency of this hybrid approach is demonstrated by solving a constrained multi-objective mathematical test-case.     

A stochastic extra-step quasi-Newton method for nonsmooth nonconvex optimization

In this paper, a novel stochastic extra-step quasi-Newton method is developed to solve a class of nonsmooth nonconvex composite optimization problems. We assume that the gradient of the smooth part of the objective function can only be approximated by stochastic oracles. The proposed method combines general stochastic higher order steps derived from an underlying proximal type fixed-point equation with additional stochastic proximal gradient steps to guarantee convergence. Based on suitable bounds on the step sizes, we establish global convergence to stationary points in expectation and an extension of the approach using variance reduction techniques is discussed. Motivated by large-scale and big data applications, we investigate a stochastic coordinate-type quasi-Newton scheme that allows to generate cheap and tractable stochastic higher order directions. Finally, numerical results on large-scale logistic regression and deep learning problems show that our proposed algorithm compares favorably with other state-of-the-art methods.