Equivalent Conditions for Jacobian Nonsingularity in Linear Symmetric Cone Programming

In this paper we consider the linear symmetric cone programming (SCP). At a Karush-Kuhn-Tucker (KKT) point of SCP, we present the important conditions equivalent to the nonsingularity of Clarke’s generalized Jacobian of the KKT nonsmooth system, such as primal and dual constraint nondegeneracy, the strong regularity, and the nonsingularity of the B-subdifferential of the KKT system. This affirmatively answers an open question by Chan and Sun (SIAM J. Optim. 19:370–396, 2008).     

Globally Convergent Three-Term Conjugate Gradient Methods that Use Secant Conditions and Generate Descent Search Directions for Unconstrained Optimization

In this paper, we propose a three-term conjugate gradient method based on secant conditions for unconstrained optimization problems. Specifically, we apply the idea of Dai and Liao (in Appl. Math. Optim. 43: 87–101, 2001) to the three-term conjugate gradient method proposed by Narushima et al. (in SIAM J. Optim. 21: 212–230, 2011). Moreover, we derive a special-purpose three-term conjugate gradient method for a problem, whose objective function has a special structure, and apply it to nonlinear least squares problems. We prove the global convergence properties of the proposed methods. Finally, some numerical results are given to show the performance of our methods.     

A note on global optimization via the heat diffusion equation

We consider the interesting smoothing method of global optimization recently proposed in Lau and Kwong (J Glob Optim 34:369–398, 2006) . In this method smoothed functions are solutions of an initial-value problem for a heat diffusion equation with external heat source. As shown in Lau and Kwong (J Glob Optim 34:369–398, 2006), the source helps to control global minima of the smoothed functions—they are not shifted during the smoothing. In this note we point out that for certain (families of) objective functions the proposed method unfortunately does not affect the functions, in the sense, that the smoothed functions coincide with the respective objective function. The key point here is that the Laplacian might be too weak in order to smooth out critical points.     

Multi-step nonlinear conjugate gradient methods for unconstrained minimization

Conjugate gradient methods are appealing for large scale nonlinear optimization problems, because they avoid the storage of matrices. Recently, seeking fast convergence of these methods, Dai and Liao (Appl. Math. Optim. 43:87–101, 2001) proposed a conjugate gradient method based on the secant condition of quasi-Newton methods, and later Yabe and Takano (Comput. Optim. Appl. 28:203–225, 2004) proposed another conjugate gradient method based on the modified secant condition. In this paper, we make use of a multi-step secant condition given by Ford and Moghrabi (Optim. Methods Softw. 2:357–370, 1993; J. Comput. Appl. Math. 50:305–323, 1994) and propose two new conjugate gradient methods based on this condition. The methods are shown to be globally convergent under certain assumptions. Numerical results are reported.     

A block coordinate gradient descent method for regularized convex separable optimization and covariance selection

We consider a class of unconstrained nonsmooth convex optimization problems, in which the objective function is the sum of a convex smooth function on an open subset of matrices and a separable convex function on a set of matrices. This problem includes the covariance selection problem that can be expressed as an ℓ ₁-penalized maximum likelihood estimation problem. In this paper, we propose a block coordinate gradient descent method (abbreviated as BCGD) for solving this class of nonsmooth separable problems with the coordinate block chosen by a Gauss-Seidel rule. The method is simple, highly parallelizable, and suited for large-scale problems. We establish global convergence and, under a local Lipschizian error bound assumption, linear rate of convergence for this method. For the covariance selection problem, the method can terminate in O(n³/e){O(n^3/\epsilon)} iterations with an e{\epsilon}-optimal solution. We compare the performance of the BCGD method with the first-order methods proposed by Lu (SIAM J Optim 19:1807–1827, 2009; SIAM J Matrix Anal Appl 31:2000–2016, 2010) for solving the covariance selection problem on randomly generated instances. Our numerical experience suggests that the BCGD method can be efficient for large-scale covariance selection problems with constraints.     

Feasible Method for Generalized Semi-Infinite Programming

In this paper, we analyze the outer approximation property of the algorithm for generalized semi-infinite programming from Stein and Still (SIAM J. Control Optim. 42:769–788, 2003). A simple bound on the regularization error is found and used to formulate a feasible numerical method for generalized semi-infinite programming with convex lower-level problems. That is, all iterates of the numerical method are feasible points of the original optimization problem. The new method has the same computational cost as the original algorithm from Stein and Still (SIAM J. Control Optim. 42:769–788, 2003). We also discuss the merits of this approach for the adaptive convexification algorithm, a feasible point method for standard semi-infinite programming from Floudas and Stein (SIAM J. Optim. 18:1187–1208, 2007).     

Globally Convergent Cutting Plane Method for Nonconvex Nonsmooth Minimization

Nowadays, solving nonsmooth (not necessarily differentiable) optimization problems plays a very important role in many areas of industrial applications. Most of the algorithms developed so far deal only with nonsmooth convex functions. In this paper, we propose a new algorithm for solving nonsmooth optimization problems that are not assumed to be convex. The algorithm combines the traditional cutting plane method with some features of bundle methods, and the search direction calculation of feasible direction interior point algorithm (Herskovits, J. Optim. Theory Appl. 99(1):121–146, 1998). The algorithm to be presented generates a sequence of interior points to the epigraph of the objective function. The accumulation points of this sequence are solutions to the original problem. We prove the global convergence of the method for locally Lipschitz continuous functions and give some preliminary results from numerical experiments.     

A priori error estimates for space–time finite element discretization of semilinear parabolic optimal control problems

In this paper, a priori error estimates for space–time finite element discretizations of optimal control problems governed by semilinear parabolic PDEs and subject to pointwise control constraints are derived. We extend the approach from Meidner and Vexler (SIAM Control Optim 47(3):1150–1177, 2008; SIAM Control Optim 47(3):1301–1329, 2008) where linear-quadratic problems have been considered, discretizing the state equation by usual conforming finite elements in space and a discontinuous Galerkin method in time. Error estimates for controls discretized by piecewise constant functions in time and cellwise constant functions in space are derived in detail and we explain how error estimate for further discretization approaches, e.g., cellwise linear discretization in space, the postprocessing approach from Meyer and R?sch (SIAM J Control Optim 43:970–985, 2004), and the variationally discrete approach from Hinze (J Comput Optim Appl 30:45–63, 2005) can be obtained. In addition, we derive an estimate for a setting with finitely many time-dependent controls.     

On a Zero Duality Gap Result in Extended Monotropic Programming

In this note we correct and improve a zero duality gap result in extended monotropic programming given by Bertsekas (J. Optim. Theory Appl. 139:209–225, 2008).     

On Implicit Multifunction Theorems

We give implicit multifunction results generalizing to multifunctions the Robinson’s implicit function theorem (Robinson, Math Oper Res 16(2):292–309, 1991). To this end, we use parametric error bounds estimates for a suitable function refining the one given in Azé and Corvellec (ESAIM Control Optim Calc Var 10:409–425, 2004). Sharp approximations of the implicit multifunctions are given extending the results of Nachi and Penot (Control Cybernet 35:871–901, 2005). Dedicated to Boris Mordukhovich in honour of his 60th birthday.     

Further Results on the Robust Regulation of a One-Dimensional Dam-River System

In this paper, we consider a one-dimensional dam-river system studied by Chentouf and Wang (SIAM J. Control Optim. 47: 2275–2302, 2008). Then, using the frequency multiplier method, we provide a simple and alternative proof of stabilization and regulation results obtained in the work cited above. Moreover, we relax the conditions on the feedback gains involved in the feedback law and give a partial answer to the open problem left by the authors Chentouf and Wang (J. Optim. Theory Appl. 134: 223–239, 2007 and SIAM J. Control Optim. 47: 2275–2302, 2008) concerning the tuning of the gains.     

Solution Methods for Pseudomonotone Variational Inequalities

We extend some results due to Thanh-Hao (Acta Math. Vietnam. 31: 283–289, [2006]) and Noor (J. Optim. Theory Appl. 115:447–452, [2002]). The first paper established a convergence theorem for the Tikhonov regularization method (TRM) applied to finite-dimensional pseudomonotone variational inequalities (VIs), answering in the affirmative an open question stated by Facchinei and Pang (Finite-Dimensional Variational Inequalities and Complementarity Problems, Springer, New York, [2003]). The second paper discussed the application of the proximal point algorithm (PPA) to pseudomonotone VIs. In this paper, new facts on the convergence of TRM and PPA (both the exact and inexact versions of PPA) for pseudomonotone VIs in Hilbert spaces are obtained and a partial answer to a question stated in (Acta Math. Vietnam. 31:283–289, [2006]) is given. As a byproduct, we show that the convergence theorem for inexact PPA applied to infinite-dimensional monotone variational inequalities can be proved without using the theory of maximal monotone operators. This research was supported in part by a grant from the National Sun Yat-Sen University, Kaohsiung, Taiwan. It has been carried out under the agreement between the National Sun Yat-Sen University, Kaohsiung, Taiwan and the University of Pisa, Pisa, Italy. The authors thank the anonymous referee for useful comments and suggestions.     

A new smoothing Broyden-like method for solving nonlinear complementarity problem with a <Emphasis Type="Italic">P</Emphasis><Subscript>0</Subscript>-function

In this paper, we propose a new smoothing Broyden-like method for solving nonlinear complementarity problem with P ₀ function. The presented algorithm is based on the smoothing symmetrically perturbed minimum function φ(a, b) = min{a, b} and makes use of the derivative-free line search rule of Li et al. (J Optim Theory Appl 109(1):123–167, 2001). Without requiring any strict complementarity assumption at the P ₀-NCP solution, we show that the iteration sequence generated by the suggested algorithm converges globally and superlinearly under suitable conditions. Furthermore, the algorithm has local quadratic convergence under mild assumptions. Some numerical results are also reported in this paper.     

Error estimates for abstract linear–quadratic optimal control problems using proper orthogonal decomposition

In this paper we investigate POD discretizations of abstract linear–quadratic optimal control problems with control constraints. We apply the discrete technique developed by Hinze (Comput. Optim. Appl. 30:45–61, 2005) and prove error estimates for the corresponding discrete controls, where we combine error estimates for the state and the adjoint system from Kunisch and Volkwein (Numer. Math. 90:117–148, 2001; SIAM J. Numer. Anal. 40:492–515, 2002). Finally, we present numerical examples that illustrate the theoretical results.     

On the Surrogate Gradient Algorithm for Lagrangian Relaxation

When applied to large-scale separable optimization problems, the recently developed surrogate subgradient method for Lagrangian relaxation (Zhao et al.: J. Optim. Theory Appl. 100, 699–712, 1999) does not need to solve optimally all the subproblems to update the multipliers, as the traditional subgradient method requires. Based on it, the penalty surrogate subgradient algorithm was further developed to address the homogenous solution issue (Guan et al.: J. Optim. Theory Appl. 113, 65–82, 2002; Zhai et al.: IEEE Trans. Power Syst. 17, 1250–1257, 2002). There were flaws in the proofs of Zhao et al., Guan et al., and Zhai et al.: for problems with inequality constraints, projection is necessary to keep the multipliers nonnegative; however, the effects of projection were not properly considered. This note corrects the flaw, completes the proofs, and asserts the correctness of the methods. This work is supported by the NSFC Grant Nos. 60274011, 60574067, the NCET program (No. NCET-04-0094) of China. The third author was supported in part by US National Science Foundation under Grants ECS-0323685 and DMI-0423607.     

On the Parametric Cubic Spline Approach for Solving Second Order Boundary Value Problems

In this article some comments on the paper “parametric cubic spline approach to the solution of a system of second order boundary value problems” in (Khan and Aziz, J. Optim. Theory Appl. 118:45–54, 2003) are given. This paper concerns with a numerical method for solving a second order boundary value problem associated with obstacle, unilateral and contact problems. Corrections are given for the convergence analysis of the numerical method and the computational experiments.     

Auxiliary Principle and Iterative Algorithms for Lions-Stampacchia Variational Inequalities

In this paper, we extend the auxiliary principle (Cohen in J. Optim. Theory Appl. 49:325–333, 1988) to study a class of Lions-Stampacchia variational inequalities in Hilbert spaces. Our method consists in approximating, in the subproblems, the nonsmooth convex function by a sequence of piecewise linear and convex functions, as in the bundle method for nonsmooth optimization. This makes the subproblems more tractable. We show the existence of a solution for this Lions-Stampacchia variational inequality and explain how to build a new iterative scheme and a new stopping criterion. This iterative scheme and criterion are different from those commonly used in the special case of nonsmooth optimization. We study also the convergence of iterative sequences generated by the algorithm. This work was supported by the National Natural Science Foundation of China (10671135), the Specialized Research Fund for the Doctoral Program of Higher Education (20060610005), the National Natural Science Foundation of Sichuan Education Department of China (07ZB068) and the Open Fund (PLN0703) of State Key Laboratory of Oil and Gas Reservoir Geology and Exploitation (Southwest Petroleum University).     

Proximal-Like Algorithm Using the Quasi D-Function for Convex Second-Order Cone Programming

In this paper, we present a measure of distance in a second-order cone based on a class of continuously differentiable strictly convex functions on ℝ₊₊. Since the distance function has some favorable properties similar to those of the D-function (Censor and Zenios in J. Optim. Theory Appl. 73:451–464 [1992]), we refer to it as a quasi D-function. Then, a proximal-like algorithm using the quasi D-function is proposed and applied to the second-cone programming problem, which is to minimize a closed proper convex function with general second-order cone constraints. Like the proximal point algorithm using the D-function (Censor and Zenios in J. Optim. Theory Appl. 73:451–464 [1992]; Chen and Teboulle in SIAM J. Optim. 3:538–543 [1993]), under some mild assumptions we establish the global convergence of the algorithm expressed in terms of function values; we show that the sequence generated by the proposed algorithm is bounded and that every accumulation point is a solution to the considered problem. Research of Shaohua Pan was partially supported by the Doctoral Starting-up Foundation (B13B6050640) of GuangDong Province. Jein-Shan Chen is a member of the Mathematics Division, National Center for Theoretical Sciences, Taipei Office. The author’s work was partially supported by National Science Council of Taiwan.     

A filter inexact-restoration method for nonlinear programming

A new iterative algorithm based on the inexact-restoration (IR) approach combined with the filter strategy to solve nonlinear constrained optimization problems is presented. The high level algorithm is suggested by Gonzaga et al. (SIAM J. Optim. 14:646–669, 2003) but not yet implement—the internal algorithms are not proposed. The filter, a new concept introduced by Fletcher and Leyffer (Math. Program. Ser. A 91:239–269, 2002), replaces the merit function avoiding the penalty parameter estimation and the difficulties related to the nondifferentiability. In the IR approach two independent phases are performed in each iteration, the feasibility and the optimality phases. The line search filter is combined with the first one phase to generate a “more feasible” point, and then it is used in the optimality phase to reach an “optimal” point. Numerical experiences with a collection of AMPL problems and a performance comparison with IPOPT are provided.      

On the convergence of Newton-type methods under mild differentiability conditions

We introduce the new idea of recurrent functions to provide a new semilocal convergence analysis for Newton-type methods, under mild differentiability conditions. It turns out that our sufficient convergence conditions are weaker, and the error bounds are tighter than in earlier studies in some interesting cases (Chen, Ann Inst Stat Math 42:387–401, 1990; Chen, Numer Funct Anal Optim 10:37–48, 1989; Cianciaruso, Numer Funct Anal Optim 24:713–723, 2003; Cianciaruso, Nonlinear Funct Anal Appl 2009; Dennis 1971; Deuflhard 2004; Deuflhard, SIAM J Numer Anal 16:1–10, 1979; Gutiérrez, J Comput Appl Math 79:131–145, 1997; Hernández, J Optim Theory Appl 109:631–648, 2001; Hernández, J Comput Appl Math 115:245–254, 2000; Huang, J Comput Appl Math 47:211–217, 1993; Kantorovich 1982; Miel, Numer Math 33:391–396, 1979; Miel, Math Comput 34:185–202, 1980; Moret, Computing 33:65–73, 1984; Potra, Libertas Mathematica 5:71–84, 1985; Rheinboldt, SIAM J Numer Anal 5:42–63, 1968; Yamamoto, Numer Math 51: 545–557, 1987; Zabrejko, Numer Funct Anal Optim 9:671–684, 1987; Zinc̆ko 1963). Applications and numerical examples, involving a nonlinear integral equation of Chandrasekhar-type, and a differential equation are also provided in this study.