The DC (Difference of Convex Functions) Programming and DCA Revisited with DC Models of Real World Nonconvex Optimization Problems

The DC programming and its DC algorithm (DCA) address the problem of minimizing a function f=g−h (with g,h being lower semicontinuous proper convex functions on R ⁿ ) on the whole space. Based on local optimality conditions and DC duality, DCA was successfully applied to a lot of different and various nondifferentiable nonconvex optimization problems to which it quite often gave global solutions and proved to be more robust and more efficient than related standard methods, especially in the large scale setting. The computational efficiency of DCA suggests to us a deeper and more complete study on DC programming, using the special class of DC programs (when either g or h is polyhedral convex) called polyhedral DC programs. The DC duality is investigated in an easier way, which is more convenient to the study of optimality conditions. New practical results on local optimality are presented. We emphasize regularization techniques in DC programming in order to construct suitable equivalent DC programs to nondifferentiable nonconvex optimization problems and new significant questions which have to be answered. A deeper insight into DCA is introduced which really sheds new light on DCA and could partly explain its efficiency. Finally DC models of real world nonconvex optimization are reported.     

Necessary and sufficient conditions for linear suboptimality in constrained optimization

This article is devoted to the study of some extremality and optimality notions that are different from conventional concepts of optimal solutions to optimization-related problems. These notions reflect certain amounts of linear subextremality for set systems and linear suboptimality for feasible solutions to multiobjective and scalar optimization problems. In contrast to standard notions of optimality, it is possible to derive necessary and sufficient conditions for linear subextremality and suboptimality in general nonconvex settings, which is done in this article via robust generalized differential constructions of variational analysis in finite-dimensional and infinite-dimensional spaces.      

Transversality condition and optimization of higher order ordinary differential inclusions

In the present paper, a Bolza problem of optimal control theory with a fixed time interval given by convex and nonconvex second-order differential inclusions (P_H) is studied. Our main goal is to derive sufficient optimality conditions for Cauchy problem of sth-order differential inclusions. The sufficient conditions including distinctive transversality condition are proved incorporating the Euler–Lagrange and Hamiltonian type inclusions. The basic concepts involved in obtaining optimality conditions are the locally adjoint mappings. Furthermore, the application of these results is demonstrated by solving the problems with third-order differential inclusions.     

On the convergence of right transforming iterations for the numerical solution of PDE‐constrained optimization problems

We present an iterative solver, called right transforming iterations (or right transformations), for linear systems with a certain structure in the system matrix, such as they typically arise in the framework of Karush–Kuhn–Tucker (KKT) conditions for optimization problems under PDE constraints. The construction of the right transforming scheme depends on an inner approximate solver for the underlying PDE subproblems. We give a rigorous convergence proof for the right transforming iterative scheme in dependence on the convergence properties of the inner solver. Provided that a fast subsolver is available, this iterative scheme represents an efficient way of solving first‐order optimality conditions. Numerical examples endorse the theoretically predicted contraction rates. Copyright © 2011 John Wiley & Sons, Ltd.     

PDE-constrained control using Femlab – Control of the Navier–Stokes equations

We show how the software Femlab can be used to solve PDE-constrained optimal control problems. We give a general formulation for such kind of problems and derive the adjoint equation and optimality system. Then these preliminaries are specified for the stationary Navier–Stokes equations with distributed and boundary control. The main steps to define and solve a PDE with Femlab are described. We describe how the adjoint system can be implemented, and how the optimality system can be used by Femlab’s built-in functions. Special crucial topics concerning efficiency are discussed. Examples with distributed and boundary control for different type of cost functionals in 2 and 3 space dimensions are presented.     

Domain Decomposition of Optimal Control Problems for Dynamic Networks of Elastic Strings

We consider optimal control problems related to exact- and approximate controllability of dynamic networks of elastic strings. In this note we concentrate on problems with linear dynamics, no state and no control constraints. The emphasis is on approximating target states and velocities in part of the network using a dynamic domain decomposition method (d³m) for the optimality system on the network. The decomposition is established via a Uzawa-type saddle-point iteration associated with an augmented Lagrangian relaxation of the transmission conditions at multiple joints. We consider various cost functions and prove convergence of the infinite dimensional scheme for an exemplaric choice of the cost. We also give numerical evidence in the case of simple exemplaric networks.     

On the No-Gap Second-Order Optimality Conditions for a Discrete Optimal Control Problem with Mixed Constraints

Motivated by our recent works on optimality conditions in discrete optimal control problems under a nonconvex cost function, in this paper, we study second-order necessary and sufficient optimality conditions for a discrete optimal control problem with a nonconvex cost function and state-control constraints. By establishing an abstract result on second-order optimality conditions for a mathematical programming problem, we derive second-order necessary and sufficient optimality conditions for a discrete optimal control problem. Using a common critical cone for both the second-order necessary and sufficient optimality conditions, we obtain “no-gap” between second-order optimality conditions.     

$$S_{1/2}$$ regularization methods and fixed point algorithms for affine rank minimization problems

The affine rank minimization problem is to minimize the rank of a matrix under linear constraints. It has many applications in various areas such as statistics, control, system identification and machine learning. Unlike the literatures which use the nuclear norm or the general Schatten \(q~ (0<q<1)\) quasi-norm to approximate the rank of a matrix, in this paper we use the Schatten 1 / 2 quasi-norm approximation which is a better approximation than the nuclear norm but leads to a nonconvex, nonsmooth and non-Lipschitz optimization problem. It is important that we give a global necessary optimality condition for the \(S_{1/2}\) regularization problem by virtue of the special objective function. This is very different from the local optimality conditions usually used for the general \(S_q\) regularization problems. Explicitly, the global necessary optimality condition for the \(S_{1/2}\) regularization problem is a fixed point inclusion associated with the singular value half thresholding operator. Naturally, we propose a fixed point iterative scheme for the problem. We also provide the convergence analysis of this iteration. By discussing the location and setting of the optimal regularization parameter as well as using an approximate singular value decomposition procedure, we get a very efficient algorithm, half norm fixed point algorithm with an approximate SVD (HFPA algorithm), for the \(S_{1/2}\) regularization problem. Numerical experiments on randomly generated and real matrix completion problems are presented to demonstrate the effectiveness of the proposed algorithm.     

A Priori Error Estimates for Finite Element Methods for H (2,1)-Elliptic Equations

The convergence of finite element methods for linear elliptic boundary value problems of second and forth order is well understood. In this article, we introduce finite element approximations of some linear semi-elliptic boundary value problem of mixed order on a two-dimensional rectangular domain Q. The equation is of second order in one direction and forth order in the other and appears in the optimal control of parabolic partial differential equations if one eliminates the control and the state (or the adjoint state) in the first order optimality conditions. We establish a regularity result and estimate for the finite element error of conforming approximations of this equation. The finite elements in use have a tensor product structure, in one dimension we use linear, quadratic or cubic Lagrange elements in the other dimension cubic Hermite elements. For these elements, we prove the error bound O(h ² + τ ^k ) in the energy norm and O((h ² + τ ^k )(h ² + τ)) in the L ²(Q)-norm.     

Optimal control of unilateral obstacle problem with a source term

We consider an optimal control problem for the obstacle problem with an elliptic variational inequality. The obstacle function which is the control function is assumed in H². We use an approximate technique to introduce a family of problems governed by variational equations. We prove optimal solutions existence and give necessary optimality conditions. The author is grateful to Prof. M. Bergounioux for her instructive suggestions.     

Composite finite element approximation for nonlinear parabolic problems in nonconvex polygonal domains

We study a new class of finite elements so‐called composite finite elements (CFEs), introduced earlier by Hackbusch and Sauter, Numer. Math., 1997; 75:447‐472, for the approximation of nonlinear parabolic equation in a nonconvex polygonal domain. A two‐scale CFE discretization is used for the space discretizations, where the coarse‐scale grid discretized the domain at an appropriate distance from the boundary and the fine‐scale grid is used to resolve the boundary. A continuous, piecewise linear CFE space is employed for the spatially semidiscrete finite element approximation and the temporal discretizations is based on modified linearized backward Euler scheme. We derive almost optimal‐order convergence in space and optimal order in time for the CFE method in the L^∞(L²) norm. Numerical experiment is carried out for an L‐shaped domain to illustrate our theoretical findings.     

Quadratic spline collocation methods for elliptic partial differential equations

We consider Quadratic Spline Collocation (QSC) methods for linear second order elliptic Partial Differential Equations (PDEs). The standard formulation of these methods leads to non-optimal approximations. In order to derive optimal QSC approximations, high order perturbations of the PDE problem are generated. These perturbations can be applied either to the PDE problem operators or to the right sides, thus leading to two different formulations of optimal QSC methods. The convergence properties of the QSC methods are studied. OptimalO(h ^3–j ) global error estimates for thejth partial derivative are obtained for a certain class of problems. Moreover,O(h ^4–j ) error bounds for thejth partial derivative are obtained at certain sets of points. Results from numerical experiments verify the theoretical behaviour of the QSC methods. Performance results also show that the QSC methods are very effective from the computational point of view. They have been implemented efficiently on parallel machines.This research was supported in part by David Ross Foundation (U.S.A) and NSERC (Natural Sciences and Engineering Research Council of Canada).     

Efficient dynamic programming implementations of Newton's method for unconstrained optimal control problems

Naive implementations of Newton's method for unconstrainedN-stage discrete-time optimal control problems with Bolza objective functions tend to increase in cost likeN ³ asN increases. However, if the inherent recursive structure of the Bolza problem is properly exploited, the cost of computing a Newton step will increase only linearly withN. The efficient Newton implementation scheme proposed here is similar to Mayne's DDP (differential dynamic programming) method but produces the Newton step exactly, even when the dynamical equations are nonlinear. The proposed scheme is also related to a Riccati treatment of the linear, two-point boundary-value problems that characterize optimal solutions. For discrete-time problems, the dynamic programming approach and the Riccati substitution differ in an interesting way; however, these differences essentially vanish in the continuous-time limit.This work was supported by the National Science Foundation, Grant No. DMS-85-03746.     

A Petrov‐Galerkin method with quadrature for semi‐linear second‐order hyperbolic problems

We propose and analyze a Crank–Nicolson quadrature Petrov–Galerkin (CNQPG) ‐spline method for solving semi‐linear second‐order hyperbolic initial‐boundary value problems. We prove second‐order convergence in time and optimal order H² norm convergence in space for the CNQPG scheme that requires only linear algebraic solvers. We demonstrate numerically optimal order H^k, k = 0,1,2, norm convergence of the scheme for some test problems with smooth and nonsmooth nonlinearities. © 2006 Wiley Periodicals, Inc. Numer Methods Partial Differential Eq, 2006     

Study of a special nonlinear problem arising in convex semi-infinite programming

We consider convex problems of semi-infinite programming (SIP) using an approach based on the implicit optimality criterion. This criterion allows one to replace optimality conditions for a feasible solution x ⁰ of the convex SIP problem by such conditions for x ⁰ in some nonlinear programming (NLP) problem denoted by NLP(I(x ⁰)). This nonlinear problem, constructed on the base of special characteristics of the original SIP problem, so-called immobile indices and their immobility orders, has a special structure and a diversity of important properties. We study these properties and use them to obtain efficient explicit optimality conditions for the problem NLP(I(x ⁰)). Application of these conditions, together with the implicit optimality criterion, gives new efficient optimality conditions for convex SIP problems. Special attention is paid to SIP problems whose constraints do not satisfy the Slater condition and to problems with analytic constraint functions for which we obtain optimality conditions in the form of a criterion. Comparison with some known optimality conditions for convex SIP is provided.     

Finite-difference discretizations of quadratic control problems governed by ordinary elliptic differential equations

In this paper, we analyze finite difference discretizations for a class of control constrained elliptic optimal control problems. If the optimal control has a derivative of bounded variation, we show discrete quadratic convergence in terms of the mesh size h of the discrete optimal controls. Furthermore, based on the optimality conditions, we construct a new discrete control for which we derive continuous error estimates of order h ².     

Global optimization conditions for certain nonconvex minimization problems

By means of suitable dual problems to the following global optimization problems: extremum{f(x): x M X}, wheref is a proper convex and lower-semicontinuous function andM a nonempty, arbitrary subset of a reflexive Banach spaceX, we derive necessary and sufficient optimality conditions for a global minimizer. The method is also applicable to other nonconvex problems and leads to at least necessary global optimality conditions.     

Local discontinuous galerkin approximation of convection‐dominated diffusion optimal control problems with control constraints

In this article, we investigate local discontinuous Galerkin approximation of stationary convection‐dominated diffusion optimal control problems with distributed control constraints. The state variable and adjoint state variable are approximated by piecewise linear polynomials without continuity requirement, whereas the control variable is discretized by variational discretization concept. The discrete first‐order optimality condition is derived. We show that optimization and discretization are commutative for the local discontinuous Galerkin approximation. Because the solutions to convection‐dominated diffusion equations often admit interior or boundary layers, residual type a posteriori error estimate in L² norm is proved, which can be used to guide mesh refinement. Finally, numerical examples are presented to illustrate the theoretical findings. © 2013 Wiley Periodicals, Inc. Numer Methods Partial Differential Eq 30: 339–360, 2014     

L2‐estimates for the DG IIPG‐0 scheme

We discuss the optimality in L² of a variant of the Incomplete Discontinuous Galerkin Interior Penalty method (IIPG) for second order linear elliptic problems. We prove optimal estimate, in two and three dimensions, for the lowest order case under suitable regularity assumptions on the data and on the mesh. We also provide numerical evidence, in one dimension, of the necessity of the regularity assumptions. © 2011 Wiley Periodicals, Inc. Numer Methods Partial Differential Eq 2012     

Global optimality condition and fixed point continuation algorithm for non-Lipschitz ℓ<Subscript><Emphasis Type="Italic">p</Emphasis></Subscript> regularized matrix minimization

Regularized minimization problems with nonconvex, nonsmooth, even non-Lipschitz penalty functions have attracted much attention in recent years, owing to their wide applications in statistics, control, system identification and machine learning. In this paper, the non-Lipschitz ? _p (0 < p < 1) regularized matrix minimization problem is studied. A global necessary optimality condition for this non-Lipschitz optimization problem is firstly obtained, specifically, the global optimal solutions for the problem are fixed points of the so-called p-thresholding operator which is matrix-valued and set-valued. Then a fixed point iterative scheme for the non-Lipschitz model is proposed, and the convergence analysis is also addressed in detail. Moreover, some acceleration techniques are adopted to improve the performance of this algorithm. The effectiveness of the proposed p-thresholding fixed point continuation (p-FPC) algorithm is demonstrated by numerical experiments on randomly generated and real matrix completion problems.