Nonmonotone derivative-free methods for nonlinear equations

In this paper we study nonmonotone globalization techniques, in connection with iterative derivative-free methods for solving a system of nonlinear equations in several variables. First we define and analyze a class of nonmonotone derivative-free linesearch techniques for unconstrained minimization of differentiable functions. Then we introduce a globalization scheme, which combines nonmonotone watchdog rules and nonmonotone linesearches, and we study the application of this scheme to some recent extensions of the Barzilai–Borwein gradient method and to hybrid stabilization algorithms employing linesearches along coordinate directions. Numerical results on a set of standard test problems show that the proposed techniques can be of value in the solution of large-dimensional systems of equations.     

A penalty derivative-free algorithm for nonlinear constrained optimization

In this paper, we modify a derivative-free line search algorithm (DFL) proposed in the Ref. (Liuzzi et al. SIAM J Optimiz 20(5):2614–2635, 2010) to minimize a continuously differentiable function of box constrained variables or unconstrained variables with nonlinear constraints. The first-order derivatives of the objective function and of the constraints are assumed to be neither calculated nor explicitly approximated. Different line-searches are used for box-constrained variables and unconstrained variables. Accordingly the convergence to stationary points is proved. The computational behavior of the method has been evaluated on a set of test problems. The performance and data profiles are used to compare with DFL.     

Trust-region algorithms for derivative-free optimization and nonlinear bilevel programming

We briefly describe the contents of the authors PhD thesis (see Colson 2003) discussed on July 2003 at the University of Namur (Belgium) and supervised by Philippe L. Toint. The contributions presented in this thesis are the development of trust-region methods for solving two particular classes of mathematical programs, namely derivative-free optimization (DFO) problems and nonlinear bilevel programming problems. The thesis is written in English and is available via the author.Received: July 2003, AMS classification: 65D05, 90C30, 90C56, 90C59     

Scaled three-term derivative-free methods for solving large-scale nonlinear monotone equations

Numerical Algorithms - In this paper, two effective derivative-free methods are proposed for solving large-scale nonlinear monotone equations, in which the search directions are sufficiently...     

A DERIVATIVE-FREE ALGORITHM FOR UNCONSTRAINED OPTIMIZATION

In this paper a hybrid algorithm which combines the pattern search method and the genetic algorithm for unconstrained optimization is presented. The algorithm is a deterministic pattern search algorithm,but in the search step of pattern search algorithm,the trial points are produced by a way like the genetic algorithm. At each iterate, by reduplication,crossover and mutation, a finite set of points can be used. In theory,the algorithm is globally convergent. The most stir is the numerical results showing that it can find the global minimizer for some problems ,which other pattern search algorithms don＇t bear.     

A family of derivative-free conjugate gradient methods for large-scale nonlinear systems of equations

In this paper, we propose a family of derivative-free conjugate gradient methods for large-scale nonlinear systems of equations. They come from two modified conjugate gradient methods [W.Y. Cheng, A two term PRP based descent Method, Numer. Funct. Anal. Optim. 28 (2007) 1217–1230; L. Zhang, W.J. Zhou, D.H. Li, A descent modified Polak–Ribiére–Polyak conjugate gradient method and its global convergence, IMA J. Numer. Anal. 26 (2006) 629–640] recently proposed for unconstrained optimization problems. Under appropriate conditions, the global convergence of the proposed method is established. Preliminary numerical results show that the proposed method is promising.     

On the local convergence of a derivative-free algorithm for least-squares minimization

In Zhang et al. (accepted by SIAM J. Optim., 2010), we developed a class of derivative-free algorithms, called DFLS, for least-squares minimization. Global convergence of the algorithm as well as its excellent numerical performance within a limited computational budget was established and discussed in the same paper. Here we would like to establish the local quadratic convergence of the algorithm for zero residual problems. Asymptotic convergence performance of the algorithm for both zero and nonzero problems is tested. Our numerical experiments indicate that the algorithm is also very promising for achieving high accuracy solutions compared with software packages that do not exploit the special structure of the least-squares problem or that use finite differences to approximate the gradients.     

A fourth-order derivative-free algorithm for nonlinear equations

A two-step derivative-free iterative algorithm is presented for solving nonlinear equations. Error analysis shows that the algorithm is fourth-order with efficiency index equal to 1.5874. A lot of numerical results show that the algorithm is effective and is preferable to some existing derivative-free methods in terms of computation cost.     

Algorithm for forming derivative-free optimal methods

We develop a simple yet effective and applicable scheme for constructing derivative free optimal iterative methods, consisting of one parameter, for solving nonlinear equations. According to the, still unproved, Kung-Traub conjecture an optimal iterative method based on k+1 evaluations could achieve a maximum convergence order of $2^{k}$ . Through the scheme, we construct derivative free optimal iterative methods of orders two, four and eight which request evaluations of two, three and four functions, respectively. The scheme can be further applied to develop iterative methods of even higher orders. An optimal value of the free-parameter is obtained through optimization and this optimal value is applied adaptively to enhance the convergence order without increasing the functional evaluations. Computational results demonstrate that the developed methods are efficient and robust as compared with many well known methods.     

A new subspace correction method for nonlinear unconstrained convex optimization problems

This paper gives a new subspace correction algorithm for nonlinear unconstrained convex optimization problems based on the multigrid approach proposed by S. Nash in 2000 and the subspace correction algorithm proposed by X. Tai and J. Xu in 2001. Under some reasonable assumptions, we obtain the convergence as well as a convergence rate estimate for the algorithm. Numerical results show that the algorithm is effective.     

On some efficient derivative-free iterative methods with memory for solving systems of nonlinear equations

We present derivative free methods with memory with increasing order of convergence for solving systems of nonlinear equations. These methods relied on the basic family of fourth order methods without memory proposed by Sharma et al. (Appl. Math. Comput. 235, 383–393, 2014). The order of convergence of new family is increased from 4 of the basic family to \(2+\sqrt {5} \approx 4.24\) by suitable variation of a free self-corrected parameter in each iterative step. In a particular case of the family even higher order of convergence \(2+\sqrt {6} \approx 4.45\) is achieved. It is shown that the new methods are more efficient in general. The presented numerical tests confirm the theoretical results.     

A subspace limited memory quasi-Newton algorithm for large-scale nonlinear bound constrained optimization

In this paper we propose a subspace limited memory quasi-Newton method for solving large-scale optimization with simple bounds on the variables. The limited memory quasi-Newton method is used to update the variables with indices outside of the active set, while the projected gradient method is used to update the active variables. The search direction consists of three parts: a subspace quasi-Newton direction, and two subspace gradient and modified gradient directions. Our algorithm can be applied to large-scale problems as there is no need to solve any subproblems. The global convergence of the method is proved and some numerical results are also given.

     

A subspace implementation of quasi-Newton trust region methods for unconstrained optimization

This paper studies subspace properties of trust region methods for unconstrained optimization, assuming the approximate Hessian is updated by quasi- Newton formulae and the initial Hessian approximation is appropriately chosen. It is shown that the trial step obtained by solving the trust region subproblem is in the subspace spanned by all the gradient vectors computed. Thus, the trial step can be defined by minimizing the quasi-Newton quadratic model in the subspace. Based on this observation, some subspace trust region algorithms are proposed and numerical results are also reported.     

Comparison of preconditioned Krylov subspace iteration methods for PDE-constrained optimization problems

We construct a novel multi-step iterative method for solving systems of nonlinear equations by introducing a parameter θ to generalize the multi-step Newton method while keeping its order of convergence and computational cost. By an appropriate selection of θ, the new method can both have faster convergence and have larger radius of convergence. The new iterative method only requires one Jacobian inversion per iteration, and therefore, can be efficiently implemented using Krylov subspace methods. The new method can be used to solve nonlinear systems of partial differential equations, such as complex generalized Zakharov systems of partial differential equations, by transforming them into systems of nonlinear equations by discretizing approaches in both spatial and temporal independent variables such as, for instance, the Chebyshev pseudo-spectral discretizing method. Quite extensive tests show that the new method can have significantly faster convergence and significantly larger radius of convergence than the multi-step Newton method.     

Comparison of preconditioned Krylov subspace iteration methods for PDE-constrained optimization problems

The governing dynamics of fluid flow is stated as a system of partial differential equations referred to as the Navier-Stokes system. In industrial and scientific applications, fluid flow control becomes an optimization problem where the governing partial differential equations of the fluid flow are stated as constraints. When discretized, the optimal control of the Navier-Stokes equations leads to large sparse saddle point systems in two levels. In this paper, we consider distributed optimal control for the Stokes system and test the particular case when the arising linear system can be compressed after eliminating the control function. In that case, a system arises in a form which enables the application of an efficient block matrix preconditioner that previously has been applied to solve complex-valued systems in real arithmetic. Under certain conditions, the condition number of the so preconditioned matrix is bounded by 2. The numerical and computational efficiency of the method in terms of number of iterations and execution time is favorably compared with other published methods.     

Analysis of least-squares mixed finite element methods for nonlinear nonstationary convection-diffusion problems

Some least-squares mixed finite element methods for convection-diffusion problems, steady or nonstationary, are formulated, and convergence of these schemes is analyzed. The main results are that a new optimal a priori error estimate of a least-squares mixed finite element method for a steady convection-diffusion problem is developed and that four fully-discrete least-squares mixed finite element schemes for an initial-boundary value problem of a nonlinear nonstationary convection-diffusion equation are formulated. Also, some systematic theories on convergence of these schemes are established.

     

A derivative-free algorithm for systems of nonlinear inequalities

Recently a new derivative-free algorithm has been proposed for the solution of linearly constrained finite minimax problems. This derivative-free algorithm is based on a smoothing technique that allows one to take into account the non-smoothness of the max function. In this paper, we investigate, both from a theoretical and computational point of view, the behavior of the minmax algorithm when used to solve systems of nonlinear inequalities when derivatives are unavailable. In particular, we show an interesting property of the algorithm, namely, under some mild conditions regarding the regularity of the functions defining the system, it is possible to prove that the algorithm locates a solution of the problem after a finite number of iterations. Furthermore, under a weaker regularity condition, it is possible to show that an accumulation point of the sequence generated by the algorithm exists which is a solution of the system. Moreover, we carried out numerical experimentation and comparison of the method against a standard pattern search minimization method. The obtained results confirm that the good theoretical properties of the method correspond to interesting numerical performance. Moreover, the algorithm compares favorably with a standard derivative-free method, and this seems to indicate that extending the smoothing technique to pattern search algorithms can be beneficial.