Extending the applicability of the Gauss–Newton method under average Lipschitz–type conditions

We extend the applicability of the Gauss–Newton method for solving singular systems of equations under the notions of average Lipschitz–type conditions introduced recently in Li et al. (J Complex 26(3):268–295, 2010). Using our idea of recurrent functions, we provide a tighter local as well as semilocal convergence analysis for the Gauss–Newton method than in Li et al. (J Complex 26(3):268–295, 2010) who recently extended and improved earlier results (Hu et al. J Comput Appl Math 219:110–122, 2008; Li et al. Comput Math Appl 47:1057–1067, 2004; Wang Math Comput 68(255):169–186, 1999). We also note that our results are obtained under weaker or the same hypotheses as in Li et al. (J Complex 26(3):268–295, 2010). Applications to some special cases of Kantorovich–type conditions are also provided in this study.     

Variational discretization of Lavrentiev-regularized state constrained elliptic optimal control problems

In the present work, we apply a variational discretization proposed by the first author in (Comput. Optim. Appl. 30:45–61, 2005) to Lavrentiev-regularized state constrained elliptic control problems. We extend the results of (Comput. Optim. Appl. 33:187–208, 2006) and prove weak convergence of the adjoint states and multipliers of the regularized problems to their counterparts of the original problem. Further, we prove error estimates for finite element discretizations of the regularized problem and investigate the overall error imposed by the finite element discretization of the regularized problem compared to the continuous solution of the original problem. Finally we present numerical results which confirm our analytical findings.     

Multi-step nonlinear conjugate gradient methods for unconstrained minimization

Conjugate gradient methods are appealing for large scale nonlinear optimization problems, because they avoid the storage of matrices. Recently, seeking fast convergence of these methods, Dai and Liao (Appl. Math. Optim. 43:87–101, 2001) proposed a conjugate gradient method based on the secant condition of quasi-Newton methods, and later Yabe and Takano (Comput. Optim. Appl. 28:203–225, 2004) proposed another conjugate gradient method based on the modified secant condition. In this paper, we make use of a multi-step secant condition given by Ford and Moghrabi (Optim. Methods Softw. 2:357–370, 1993; J. Comput. Appl. Math. 50:305–323, 1994) and propose two new conjugate gradient methods based on this condition. The methods are shown to be globally convergent under certain assumptions. Numerical results are reported.     

On the convergence of Newton-type methods under mild differentiability conditions

We introduce the new idea of recurrent functions to provide a new semilocal convergence analysis for Newton-type methods, under mild differentiability conditions. It turns out that our sufficient convergence conditions are weaker, and the error bounds are tighter than in earlier studies in some interesting cases (Chen, Ann Inst Stat Math 42:387–401, 1990; Chen, Numer Funct Anal Optim 10:37–48, 1989; Cianciaruso, Numer Funct Anal Optim 24:713–723, 2003; Cianciaruso, Nonlinear Funct Anal Appl 2009; Dennis 1971; Deuflhard 2004; Deuflhard, SIAM J Numer Anal 16:1–10, 1979; Gutiérrez, J Comput Appl Math 79:131–145, 1997; Hernández, J Optim Theory Appl 109:631–648, 2001; Hernández, J Comput Appl Math 115:245–254, 2000; Huang, J Comput Appl Math 47:211–217, 1993; Kantorovich 1982; Miel, Numer Math 33:391–396, 1979; Miel, Math Comput 34:185–202, 1980; Moret, Computing 33:65–73, 1984; Potra, Libertas Mathematica 5:71–84, 1985; Rheinboldt, SIAM J Numer Anal 5:42–63, 1968; Yamamoto, Numer Math 51: 545–557, 1987; Zabrejko, Numer Funct Anal Optim 9:671–684, 1987; Zinc̆ko 1963). Applications and numerical examples, involving a nonlinear integral equation of Chandrasekhar-type, and a differential equation are also provided in this study.     

Efficient three-step iterative methods with sixth order convergence for nonlinear equations

In this paper, we present two new three-step iterative methods for solving nonlinear equations with sixth convergence order. The new methods are obtained by composing known methods of third order of convergence with Newton’s method and using an adequate approximation for the derivative, that provides high order of convergence and reduces the required number of functional evaluations per step. The first method is obtained from Potra-Pták’s method and the second one, from Homeier’s method, both reaching an efficiency index of 1.5651. Our methods are comparable with the method of Parhi and Gupta (Appl Math Comput 203:50–55, 2008). Methods proposed by Kou and Li (Appl Math Comput 189:1816–1821, 2007), Wang et al. (Appl Math Comput 204:14–19, 2008) and Chun (Appl Math Comput 190:1432–1437, 2007) reach the same efficiency index, although they start from a fourth order method while we use third order methods and simpler arithmetics. We prove the convergence results and check them with several numerical tests that allow us to compare the convergence order, the computational cost and the efficiency order of our methods with those of the original methods.     

Error estimates for the numerical approximation of Neumann control problems

We continue the discussion of error estimates for the numerical analysis of Neumann boundary control problems we started in Casas et al. (Comput. Optim. Appl. 31:193–219, 2005). In that paper piecewise constant functions were used to approximate the control and a convergence of order O(h) was obtained. Here, we use continuous piecewise linear functions to discretize the control and obtain the rates of convergence in L ²(Γ). Error estimates in the uniform norm are also obtained. We also discuss the approach suggested by Hinze (Comput. Optim. Appl. 30:45–61, 2005) as well as the improvement of the error estimates by making an extra assumption over the set of points corresponding to the active control constraints. Finally, numerical evidence of our estimates is provided. The authors were supported by Ministerio de Ciencia y Tecnología (Spain).     

Two modified Dai-Yuan nonlinear conjugate gradient methods

In this paper, we propose two modified versions of the Dai-Yuan (DY) nonlinear conjugate gradient method. One is based on the MBFGS method (Li and Fukushima, J Comput Appl Math 129:15–35, 2001) and inherits all nice properties of the DY method. Moreover, this method converges globally for nonconvex functions even if the standard Armijo line search is used. The other is based on the ideas of Wei et al. (Appl Math Comput 183:1341–1350, 2006), Zhang et al. (Numer Math 104:561–572, 2006) and possesses good performance of the Hestenes-Stiefel method. Numerical results are also reported. This work was supported by the NSF foundation (10701018) of China.     

Analysis of a class of nonlinear subdivision schemes and associated multiresolution transforms

This paper is devoted to the convergence and stability analysis of a class of nonlinear subdivision schemes and associated multiresolution transforms. As soon as a nonlinear scheme can be written as a specific perturbation of a linear and convergent subdivision scheme, we show that if some contractivity properties are satisfied, then stability and convergence can be achieved. This approach is applied to various schemes, which give different new results. More precisely, we study uncentered Lagrange interpolatory linear schemes, WENO scheme (Liu et al., J Comput Phys 115:200–212, 1994), PPH and Power-P schemes (Amat and Liandrat, Appl Comput Harmon Anal 18(2):198–206, 2005; Serna and Marquina, J Comput Phys 194:632–658, 2004) and a nonlinear scheme using local spherical coordinates (Aspert et al., Comput Aided Geom Des 20:165–187, 2003). Finally, a stability proof is given for the multiresolution transform associated to a nonlinear scheme of Marinov et al. (2005).     

On the Surrogate Gradient Algorithm for Lagrangian Relaxation

When applied to large-scale separable optimization problems, the recently developed surrogate subgradient method for Lagrangian relaxation (Zhao et al.: J. Optim. Theory Appl. 100, 699–712, 1999) does not need to solve optimally all the subproblems to update the multipliers, as the traditional subgradient method requires. Based on it, the penalty surrogate subgradient algorithm was further developed to address the homogenous solution issue (Guan et al.: J. Optim. Theory Appl. 113, 65–82, 2002; Zhai et al.: IEEE Trans. Power Syst. 17, 1250–1257, 2002). There were flaws in the proofs of Zhao et al., Guan et al., and Zhai et al.: for problems with inequality constraints, projection is necessary to keep the multipliers nonnegative; however, the effects of projection were not properly considered. This note corrects the flaw, completes the proofs, and asserts the correctness of the methods. This work is supported by the NSFC Grant Nos. 60274011, 60574067, the NCET program (No. NCET-04-0094) of China. The third author was supported in part by US National Science Foundation under Grants ECS-0323685 and DMI-0423607.     

A priori error estimates for space–time finite element discretization of semilinear parabolic optimal control problems

In this paper, a priori error estimates for space–time finite element discretizations of optimal control problems governed by semilinear parabolic PDEs and subject to pointwise control constraints are derived. We extend the approach from Meidner and Vexler (SIAM Control Optim 47(3):1150–1177, 2008; SIAM Control Optim 47(3):1301–1329, 2008) where linear-quadratic problems have been considered, discretizing the state equation by usual conforming finite elements in space and a discontinuous Galerkin method in time. Error estimates for controls discretized by piecewise constant functions in time and cellwise constant functions in space are derived in detail and we explain how error estimate for further discretization approaches, e.g., cellwise linear discretization in space, the postprocessing approach from Meyer and R?sch (SIAM J Control Optim 43:970–985, 2004), and the variationally discrete approach from Hinze (J Comput Optim Appl 30:45–63, 2005) can be obtained. In addition, we derive an estimate for a setting with finitely many time-dependent controls.     

Error estimates for abstract linear–quadratic optimal control problems using proper orthogonal decomposition

In this paper we investigate POD discretizations of abstract linear–quadratic optimal control problems with control constraints. We apply the discrete technique developed by Hinze (Comput. Optim. Appl. 30:45–61, 2005) and prove error estimates for the corresponding discrete controls, where we combine error estimates for the state and the adjoint system from Kunisch and Volkwein (Numer. Math. 90:117–148, 2001; SIAM J. Numer. Anal. 40:492–515, 2002). Finally, we present numerical examples that illustrate the theoretical results.     

Decomposition-based Method for Sparse Semidefinite Relaxations of Polynomial Optimization Problems

We consider polynomial optimization problems pervaded by a sparsity pattern. It has been shown in Lasserre (SIAM J. Optim. 17(3):822–843, 2006) and Waki et al. (SIAM J. Optim. 17(1):218–248, 2006) that the optimal solution of a polynomial programming problem with structured sparsity can be computed by solving a series of semidefinite relaxations that possess the same kind of sparsity. We aim at solving the former relaxations with a decomposition-based method, which partitions the relaxations according to their sparsity pattern. The decomposition-based method that we propose is an extension to semidefinite programming of the Benders decomposition for linear programs (Benders, Comput. Manag. Sci. 2(1):3–19, 2005).     

Morse potential energy minimization: Improved bounds for optimal configurations

We improve lower bounds on the minimal distance between two points of a minimum energy configuration w.r.t. the Morse potential. This is achieved by generalizing a method that was already applied to the Lennard-Jones potential in Schachinger et al. (Comput. Optim. Appl. 38:329–349, 2007), resulting in improvements of the currently best bounds known for ρ∈[4.967,15] both for minimal distance and for energy of optimal configurations.     

Improved local convergence analysis of the Gauss–Newton method under a majorant condition

We present a local convergence analysis of Gauss-Newton method for solving nonlinear least square problems. Using more precise majorant conditions than in earlier studies such as Chen (Comput Optim Appl 40:97–118, 2008), Chen and Li (Appl Math Comput 170:686–705, 2005), Chen and Li (Appl Math Comput 324:1381–1394, 2006), Ferreira (J Comput Appl Math 235:1515–1522, 2011), Ferreira and Gonçalves (Comput Optim Appl 48:1–21, 2011), Ferreira and Gonçalves (J Complex 27(1):111–125, 2011), Li et al. (J Complex 26:268–295, 2010), Li et al. (Comput Optim Appl 47:1057–1067, 2004), Proinov (J Complex 25:38–62, 2009), Ewing, Gross, Martin (eds.) (The merging of disciplines: new directions in pure, applied and computational mathematics 185–196, 1986), Traup (Iterative methods for the solution of equations, 1964), Wang (J Numer Anal 20:123–134, 2000), we provide a larger radius of convergence; tighter error estimates on the distances involved and a clearer relationship between the majorant function and the associated least squares problem. Moreover, these advantages are obtained under the same computational cost.     

Globally Convergent Three-Term Conjugate Gradient Methods that Use Secant Conditions and Generate Descent Search Directions for Unconstrained Optimization

In this paper, we propose a three-term conjugate gradient method based on secant conditions for unconstrained optimization problems. Specifically, we apply the idea of Dai and Liao (in Appl. Math. Optim. 43: 87–101, 2001) to the three-term conjugate gradient method proposed by Narushima et al. (in SIAM J. Optim. 21: 212–230, 2011). Moreover, we derive a special-purpose three-term conjugate gradient method for a problem, whose objective function has a special structure, and apply it to nonlinear least squares problems. We prove the global convergence properties of the proposed methods. Finally, some numerical results are given to show the performance of our methods.     

On a class of secant-like methods for solving nonlinear equations

We provide a semilocal convergence analysis for a certain class of secant-like methods considered also in Argyros (J Math Anal Appl 298:374–397, 2004, 2007), Potra (Libertas Mathematica 5:71–84, 1985), in order to approximate a locally unique solution of an equation in a Banach space. Using a combination of Lipschitz and center-Lipschitz conditions for the computation of the upper bounds on the inverses of the linear operators involved, instead of only Lipschitz conditions (Potra, Libertas Mathematica 5:71–84, 1985), we provide an analysis with the following advantages over the work in Potra (Libertas Mathematica 5:71–84, 1985) which improved the works in Bosarge and Falb (J Optim Theory Appl 4:156–166, 1969, Numer Math 14:264–286, 1970), Dennis (SIAM J Numer Anal 6(3):493–507, 1969, 1971), Kornstaedt (1975), Larsonen (Ann Acad Sci Fenn, A 450:1–10, 1969), Potra (L’Analyse Numérique et la Théorie de l’Approximation 8(2):203–214, 1979, Aplikace Mathematiky 26:111–120, 1981, 1982, Libertas Mathematica 5:71–84, 1985), Potra and Pták (Math Scand 46:236–250, 1980, Numer Func Anal Optim 2(1):107–120, 1980), Schmidt (Period Math Hung 9(3):241–247, 1978), Schmidt and Schwetlick (Computing 3:215–226, 1968), Traub (1964), Wolfe (Numer Math 31:153–174, 1978): larger convergence domain; weaker sufficient convergence conditions, finer error bounds on the distances involved, and a more precise information on the location of the solution. Numerical examples further validating the results are also provided.     

An LQP Method for Pseudomonotone Variational Inequalities

In this paper, we proposed a modified Logarithmic-Quadratic Proximal (LQP) method [Auslender et al.: Comput. Optim. Appl. 12, 31–40 (1999)] for solving variational inequalities problems. We solved the problem approximately, with constructive accuracy criterion. We show that the method is globally convergence under that the operator is pseudomonotone which is weaker than the monotonicity and the solution set is nonempty. Some preliminary computational results are given.The author was supported by the NSFC grants Nos: 70571033 and 10571083.     

A derivative free iterative method for solving least squares problems

A derivative free iterative method for approximating a solution of nonlinear least squares problems is studied first in Shakhno and Gnatyshyn (Appl Math Comput 161:253–264, 2005). The radius of convergence is determined as well as usable error estimates. We show that this method is faster than its Secant analogue examined in Shakhno and Gnatyshyn (Appl Math Comput 161:253–264, 2005). Numerical example is also provided in this paper.     

Deconvolution: a wavelet frame approach

This paper devotes to analyzing deconvolution algorithms based on wavelet frame approaches, which has already appeared in Chan et al. (SIAM J. Sci. Comput. 24(4), 1408–1432, 2003; Appl. Comput. Hormon. Anal. 17, 91–115, 2004a; Int. J. Imaging Syst. Technol. 14, 91–104, 2004b) as wavelet frame based high resolution image reconstruction methods. We first give a complete formulation of deconvolution in terms of multiresolution analysis and its approximation, which completes the formulation given in Chan et al. (SIAM J. Sci. Comput. 24(4), 1408–1432, 2003; Appl. Comput. Hormon. Anal. 17, 91–115, 2004a; Int. J. Imaging Syst. Technol. 14, 91–104, 2004b). This formulation converts deconvolution to a problem of filling the missing coefficients of wavelet frames which satisfy certain minimization properties. These missing coefficients are recovered iteratively together with a built-in denoising scheme that removes noise in the data set such that noise in the data will not blow up while iterating. This approach has already been proven to be efficient in solving various problems in high resolution image reconstructions as shown by the simulation results given in Chan et al. (SIAM J. Sci. Comput. 24(4), 1408–1432, 2003; Appl. Comput. Hormon. Anal. 17, 91–115, 2004a; Int. J. Imaging Syst. Technol. 14, 91–104, 2004b). However, an analysis of convergence as well as the stability of algorithms and the minimization properties of solutions were absent in those papers. This paper is to establish the theoretical foundation of this wavelet frame approach. In particular, a proof of convergence, an analysis of the stability of algorithms and a study of the minimization property of solutions are given.     

Equivalence of Two Nondegeneracy Conditions for Semidefinite Programs

Nondegeneracy assumptions are often needed in order to prove the local fast convergence of suitable algorithms as well as in the sensitivity analysis for semidefinite programs. One of the more standard nondegeneracy conditions is the geometric condition used by Alizadeh et al. (Math. Program. 77:111–128, 1997). On the other hand, Kanzow and Nagel (SIAM J. Optim. 15:654–672, 2005) recently introduced an algebraic condition that was used in order to prove, for the first time, the local quadratic convergence of a suitable algorithm for the solution of semidefinite programs without using the strict complementarity assumption. The aim of this paper is to show that these two nondegeneracy conditions are equivalent.