Interior-Point Algorithms,Penalty Methods and Equilibrium Problems

In this paper we consider the question of solving equilibrium problems—formulated as complementarity problems and, more generally, mathematical programs with equilibrium constraints (MPECs)—as nonlinear programs, using an interior-point approach. These problems pose theoretical difficulties for nonlinear solvers, including interior-point methods. We examine the use of penalty methods to get around these difficulties and provide substantial numerical results. We go on to show that penalty methods can resolve some problems that interior-point algorithms encounter in general. An erratum to this article is available at .     

A Geometric Integration Approach to Nonsmooth,Nonconvex Optimisation

The optimisation of nonsmooth, nonconvex functions without access to gradients is a particularly challenging problem that is frequently encountered, for example in model parameter optimisation problems. Bilevel optimisation of parameters is a standard setting in areas such as variational regularisation problems and supervised machine learning. We present efficient and robust derivative-free methods called randomised Itoh–Abe methods. These are generalisations of the Itoh–Abe discrete gradient method, a well-known scheme from geometric integration, which has previously only been considered in the smooth setting. We demonstrate that the method and its favourable energy dissipation properties are well defined in the nonsmooth setting. Furthermore, we prove that whenever the objective function is locally Lipschitz continuous, the iterates almost surely converge to a connected set of Clarke stationary points. We present an implementation of the methods, and apply it to various test problems. The numerical results indicate that the randomised Itoh–Abe methods can be superior to state-of-the-art derivative-free optimisation methods in solving nonsmooth problems while still remaining competitive in terms of efficiency.

     

Iterative operator-splitting methods with higher-order time integration methods and applications for parabolic partial differential equations

In this paper we design higher-order time integrators for systems of stiff ordinary differential equations. We combine implicit Runge–Kutta and BDF methods with iterative operator-splitting methods to obtain higher-order methods. The idea of decoupling each complicated operator in simpler operators with an adapted time scale allows to solve the problems more efficiently. We compare our new methods with the higher-order fractional-stepping Runge–Kutta methods, developed for stiff ordinary differential equations. The benefit is the individual handling of each operator with adapted standard higher-order time integrators. The methods are applied to equations for convection–diffusion reactions and we obtain higher-order results. Finally we discuss the applications of the iterative operator-splitting methods to multi-dimensional and multi-physical problems.     

An Algorithm Based on Active Sets and Smoothing for Discretized Semi-Infinite Minimax Problems

We present a new active-set strategy which can be used in conjunction with exponential (entropic) smoothing for solving large-scale minimax problems arising from the discretization of semi-infinite minimax problems. The main effect of the active-set strategy is to dramatically reduce the number of gradient calculations needed in the optimization. Discretization of multidimensional domains gives rise to minimax problems with thousands of component functions. We present an application to minimizing the sum of squares of the Lagrange polynomials to find good points for polynomial interpolation on the unit sphere in ℝ³. Our numerical results show that the active-set strategy results in a modified Armijo gradient or Gauss-Newton like methods requiring less than a quarter of the gradients, as compared to the use of these methods without our active-set strategy. Finally, we show how this strategy can be incorporated in an algorithm for solving semi-infinite minimax problems.     

Combinatorial problems motivated by comma‐free codes

In the paper some combinatorial problems motivated by comma‐free codes are considered. We describe these problems, give the most significant known results and methods used, present some new results and formulate open problems. © 2004 Wiley Periodicals, Inc.     

On regularization methods based on dynamic programming techniques

In this article, we investigate the connection between regularization theory for inverse problems and dynamic programming theory. This is done by developing two new regularization methods, based on dynamic programming techniques. The aim of these methods is to obtain stable approximations to the solution of linear inverse ill-posed problems. We follow two different approaches and derive a continuous and a discrete regularization method. Regularization properties for both methods are proved as well as rates of convergence. A numerical benchmark problem concerning integral operators with convolution kernels is used to illustrate the theoretical results.     

Rank-k modification methods for recursive least squares problems

In least squares problems, it is often desired to solve the same problem repeatedly but with several rows of the data either added, deleted, or both. Methods for quickly solving a problem after adding or deleting one row of data at a time are known. In this paper we introduce fundamental rank-k updating and downdating methods and show how extensions of rank-1 downdating methods based on LINPACK, Corrected Semi-Normal Equations (CSNE), and Gram-Schmidt factorizations, as well as new rank-k downdating methods, can all be derived from these fundamental results. We then analyze the cost of each new algorithm and make comparisons tok applications of the corresponding rank-1 algorithms. We provide experimental results comparing the numerical accuracy of the various algorithms, paying particular attention to the downdating methods, due to their potential numerical difficulties for ill-conditioned problems. We then discuss the computation involved for each downdating method, measured in terms of operation counts and BLAS calls. Finally, we provide serial execution timing results for these algorithms, noting preferable points for improvement and optimization. From our experiments we conclude that the Gram-Schmidt methods perform best in terms of numerical accuracy, but may be too costly for serial execution for large problems.Research supported in part by the Joint Services Electronics Program, contract no. F49620-90-C-0039.     

Boundary and initial-value methods for solving Fredholm equations with semidegenerate kernels

We develop a new approach to the theory and numerical solution of a class of linear and nonlinear Fredholm equations. These equations, which have semidegenerate kernels, are shown to be equivalent to two-point boundary-value problems for a system of ordinary differential equations. Applications of numerical methods for this class of problems allows us to develop a new class of numerical algorithms for the original integral equation. The scope of the paper is primarily theoretical; developing the necessary Fredholm theory and giving comparisons with related methods. For convolution equations, the theory is related to that of boundary-value problems in an appropriate Hilbert space. We believe that the results here have independent interest. In the last section, our methods are extended to certain classes of integrodifferential equations.     

New efficient numerical procedures for solving stochastic variational problems with a priori maximum pointwise error estimates

In a previous paper we gave a new formulation and derived the Euler equations and other necessary conditions to solve strong, pathwise, stochastic variational problems with trajectories driven by Brownian motion. Thus, unlike current methods which minimize the control over deterministic functionals (the expected value), we find the control which gives the critical point solution of random functionals of a Brownian path and then, if we choose, find the expected value.This increase in information is balanced by the fact that our methods are anticipative while current methods are not. However, our methods are more directly connected to the theory and meaningful examples of deterministic variational theory and provide better means of solution for free and constrained problems. In addition, examples indicate that there are methods to obtain nonanticipative solutions from our equations although the anticipative optimal cost function has smaller expected value.In this paper we give new, efficient numerical methods to find the solution of these problems in the quadratic case. Of interest is that our numerical solution has a maximal, a priori, pointwise error of O(h3/2) where h is the node size. We believe our results are unique for any theory of stochastic control and that our methods of proof involve new and sophisticated ideas for strong solutions which extend previous deterministic results by the first author where the error was O(h²).We note that, although our solutions are given in terms of stochastic differential equations, we are not using the now standard numerical methods for stochastic differential equations. Instead we find an approximation to the critical point solution of the variational problem using relations derived from setting to zero the directional derivative of the cost functional in the direction of simple test functions.Our results are even more significant than they first appear because we can reformulate stochastic control problems or constrained calculus of variations problems in the unconstrained, stochastic calculus of variations formulation of this paper. This will allow us to find efficient and accurate numerical solutions for general constrained, stochastic optimization problems. This is not yet being done, even in the deterministic case, except by the first author.     

Novel fitted operator finite difference methods for singularly perturbed elliptic convection–diffusion problems in two dimensions

We consider a class of singularly perturbed elliptic problems posed on a unit square. These problems are solved by using fitted mesh methods by many researchers but no attempts are made to solve them using fitted operator methods, except our recent work on reaction–diffusion problems [J.B. Munyakazi and K.C. Patidar, Higher order numerical methods for singularly perturbed elliptic problems, Neural Parallel Sci. Comput. 18(1) (2010), pp. 75–88]. In this paper, we design two fitted operator finite difference methods (FOFDMs) for singularly perturbed convection–diffusion problems which possess solutions with exponential and parabolic boundary layers, respectively. We observe that both of these FOFDMs are ?-uniformly convergent. This fact contradicts the claim about singularly perturbed convection–diffusion problems [Miller et al. Fitted Numerical Methods for Singular Perturbation Problems, World Scientific, Singapore, 1996] that ‘when parabolic boundary layers are present, …, it is not possible to design an ?-uniform FOFDM if the mesh is restricted to being a uniform mesh’. We confirm our theoretical findings through computational investigations and also found that we obtain better results than those of Linß and Stynes [Appl. Numer. Math. 31 (1999), pp. 255–270].