A Dynamical Approach to Constrained Nonsmooth Convex Minimization Problem Coupling with Penalty Function Method in Hilbert Space

This article is concerned with a class of nonsmooth constrained convex optimization in a real Hilbert space. Coupling with the penalty method, we propose an automatic system (AS) and a nonautomatic system (NS) modeled by differential inclusions. Under a suitable assumption on the feasible region and a proper condition on the objective and constrained functions, some valuable convergence properties of (AS) are obtained. In order to obtain strong convergence result in general cases, based on evolution differential inclusion, we propose a nonautomatic system (NS). When the control item ?(t) of (NS) satisfies some basal conditions, global and unique existence of the solution, finite time convergence to the feasible region and slow solution choice are obtained. Moreover, under different conditions of ?(t), we give some strong convergence results of (NS). Furthermore, we end the article by numerical experiments to illustrate the efficiency and good performance of the proposed systems in this article.     

Regularized parametric Kuhn-Tucker theorem in a Hilbert space

For a parametric convex programming problem in a Hilbert space with a strongly convex objective functional, a regularized Kuhn-Tucker theorem in nondifferential form is proved by the dual regularization method. The theorem states (in terms of minimizing sequences) that the solution to the convex programming problem can be approximated by minimizers of its regular Lagrangian (which means that the Lagrange multiplier for the objective functional is unity) with no assumptions made about the regularity of the optimization problem. Points approximating the solution are constructively specified. They are stable with respect to the errors in the initial data, which makes it possible to effectively use the regularized Kuhn-Tucker theorem for solving a broad class of inverse, optimization, and optimal control problems. The relation between this assertion and the differential properties of the value function (S-function) is established. The classical Kuhn-Tucker theorem in nondifferential form is contained in the above theorem as a particular case. A version of the regularized Kuhn-Tucker theorem for convex objective functionals is also considered.     

<Emphasis Type="Italic">α</Emphasis>-Conservative approximation for probabilistically constrained convex programs

In this paper, we address an approximate solution of a probabilistically constrained convex program (PCCP), where a convex objective function is minimized over solutions satisfying, with a given probability, convex constraints that are parameterized by random variables. In order to approach to a solution, we set forth a conservative approximation problem by introducing a parameter α which indicates an approximate accuracy, and formulate it as a D.C. optimization problem.     

Convergence and efficiency of subgradient methods for quasiconvex minimization

We study a general subgradient projection method for minimizing a quasiconvex objective subject to a convex set constraint in a Hilbert space. Our setting is very general: the objective is only upper semicontinuous on its domain, which need not be open, and various subdifferentials may be used. We extend previous results by proving convergence in objective values and to the generalized solution set for classical stepsizes t _k →0, ∑t _k =∞, and weak or strong convergence of the iterates to a solution for {t _k }∈ℓ₂∖ℓ₁ under mild regularity conditions. For bounded constraint sets and suitable stepsizes, the method finds ε-solutions with an efficiency estimate of O(ε^-2), thus being optimal in the sense of Nemirovskii. Received: October 4, 1998 / Accepted: July 24, 2000?Published online January 17, 2001     

Iterative algorithms with the regularization for the constrained convex minimization problem and maximal monotone operators

In this paper, we prove a strong convergence theorem for finding a common element of the solution set of a constrained convex minimization problem and the set of solutions of a finite family of variational inclusion problems in Hilbert space. A strong convergence theorem for finding a common element of the solution set of a constrained convex minimization problem and the solution sets of a finite family of zero points of the maximal monotone operator problem in Hilbert space is also obtained. Using our main result, we have some additional results for various types of non-linear problems in Hilbert space.     

Iteration methods for convexly constrained ill-posed problems in hilbert space

Minimization problems in Hilbert space with quadratic objective function and closed convex constraint set C are considered. In case the minimum is not unique we are looking for the solution of minimal norm. If a problem is ill-posed, i.e. if the solution does not depend continuously on the data, and if the data are subject to errors then it has to be solved by means of regularization methods. The regularizing properties of some gradient projection methods—i.e. convergence for exact data, order of convergence under additional assumptions on the solution and stability for perturbed data—are the main issues of this paper.     

Invariant manifolds for weak solutions to stochastic equations

Viability and invariance problems related to a stochastic equation in a Hilbert space H are studied. Finite dimensional invariant C ² submanifolds of H are characterized. We derive Nagumo type conditions and prove a regularity result: any weak solution, which is viable in a finite dimensional C ² submanifold, is a strong solution. These results are related to finding finite dimensional realizations for stochastic equations. There has recently been increased interest in connection with a model for the stochastic evolution of forward rate curves. Received: 15 April 1999 / Revised version: 4 February 2000 / Published online: 18 September 2000     

Adaptive Projected Subgradient Method for Asymptotic Minimization of Sequence of Nonnegative Convex Functions

Abstract

This paper presents an algorithm, named adaptive projected subgradient method that can minimize asymptotically a certain sequence of nonnegative convex functions over a closed convex set in a real Hilbert space. The proposed algorithm is a natural extension of the Polyak's subgradient algorithm, for nonsmooth convex optimization problem with a fixed target value, to the case where the convex objective itself keeps changing in the whole process. The main theorem, showing the strong convergence of the algorithm as well as the asymptotic optimality of the sequence generated by the algorithm, can serve as a unified guiding principle of a wide range of set theoretic adaptive filtering schemes for nonstationary random processes. These include not only the existing adaptive filtering techniques; e.g., NLMS, Projected NLMS, Constrained NLMS, APA, and Adaptive parallel outer projection algorithm etc., but also new techniques; e.g., Adaptive parallel min-max projection algorithm, and their embedded constraint versions. Numerical examples show that the proposed techniques are well-suited for robust adaptive signal processing problems.     

On Interacting Systems of Hilbert-Space-Valued Diffusions

A nonlinear Hilbert-space-valued stochastic differential equation where L ^-1 (L being the generator of the evolution semigroup) is not nuclear is investigated in this paper. Under the assumption of nuclearity of L ^-1 , the existence of a unique solution lying in the Hilbert space H has been shown by Dawson in an early paper. When L ^-1 is not nuclear, a solution in most cases lies not in H but in a larger Hilbert, Banach, or nuclear space. Part of the motivation of this paper is to prove under suitable conditions that a unique strong solution can still be found to lie in the space H itself. Uniqueness of the weak solution is proved without moment assumptions on the initial random variable. A second problem considered is the asymptotic behavior of the sequence of empirical measures determined by the solutions of an interacting system of H -valued diffusions. It is shown that the sequence converges in probability to the unique solution Λ ₀ of the martingale problem posed by the corresponding McKean—Vlasov equation. Accepted 4 April 1996     

Local characterization of invariant sets of an autonomous differential inclusion

Summary An application in robotics motivates us to characterize the evolution of a subset in state space due to a compact neighborhood of an arbitrary dynamical system—an instance of a differential inclusion. Earlier results of Blagodat·skikh and Filippov (1986) and Butkovskii (1982) characterize the boundary of theattainable set and theforward projection operator of a state. Our first result is a local characterization of the boundary of the forward projection ofa compact regular subset of the state space. Let the collection of states such that the differential inclusion contains an equilibrium point be called asingular invariant set. We show that the fields at the boundary of the forward projection of a singular invariant set are degenerate under some regularity assumptions when the state-wise boundary of the differential inclusion is smooth. Consider instead those differential inclusions such that the state-wise boundary of the problem is a regular convex polytope—a piecewise smooth boundary rather than smooth. Our second result gives conditions for theuniqueness andexistence of the boundary of the forward projection of a singular invariant set. They characterize the bundle of unstable and stable manifolds of such a differential inclusion.     

Continuous Gradient Projection Method in Hilbert Spaces

This paper is concerned with the asymptotic analysis of the trajectories of some dynamical systems built upon the gradient projection method in Hilbert spaces. For a convex function with locally Lipschitz gradient, it is proved that the orbits converge weakly to a constrained minimizer whenever it exists. This result remains valid even if the initial condition is chosen out of the feasible set and it can be extended in some sense to quasiconvex functions. An asymptotic control result, involving a Tykhonov-like regularization, shows that the orbits can be forced to converge strongly toward a well-specified minimizer. In the finite-dimensional framework, we study the differential inclusion obtained by replacing the classical gradient by the subdifferential of a continuous convex function. We prove the existence of a solution whose asymptotic properties are the same as in the smooth case.     

The Adaptive Projected Subgradient Method over the Fixed Point Set of Strongly Attracting Nonexpansive Mappings

This paper presents an algorithmic solution, the adaptive projected subgradient method, to the problem of asymptotically minimizing a certain sequence of non-negative continuous convex functions over the fixed point set of a strongly attracting nonexpansive mapping in a real Hilbert space. The method generalizes Polyak's subgradient algorithm for the convexly constrained minimization of a fixed nonsmooth function. By generating a strongly convergent and asymptotically optimal point sequence, the proposed method not only offers unifying principles for many projection-based adaptive filtering algorithms but also enhances the adaptive filtering methods with the set theoretic estimation's armory by allowing a variety of a priori information on the estimandum in the form, for example, of multiple intersecting closed convex sets.     

Chapter 2 Combined method of iterative penalty,iterative approximation,and stochastic quasigradients for solution of convex constrained optimization problems in Hilbert space

Chapter 2 Combined method of iterative penalty, iterative approximation, and stochastic quasigradients for solution of convex constrained optimization problems in Hilbert space     

About the Lipschitz property of the metric projection in the Hilbert space

We consider the metric projection operator from the real Hilbert space onto a strongly convex set. We prove that the restriction of this operator on the complement of some neighborhood of the strongly convex set is Lipschitz continuous with the Lipschitz constant strictly less than 1. This property characterizes the class of strongly convex sets and (to a certain degree) the Hilbert space. We apply the results obtained to the question concerning the rate of convergence for the gradient projection algorithm with differentiable convex function and strongly convex set.     

Nonsmooth exclusion test for finding all solutions of nonlinear equations

A new approach is proposed for finding all real solutions of systems of nonlinear equations with bound constraints. The zero finding problem is converted to a global optimization problem whose global minima with zero objective value, if any, correspond to all solutions of the original problem. A branch-and-bound algorithm is used with McCormick’s nonsmooth convex relaxations to generate lower bounds. An inclusion relation between the solution set of the relaxed problem and that of the original nonconvex problem is established which motivates a method to generate automatically, starting points for a local Newton-type method. A damped-Newton method with natural level functions employing the restrictive monotonicity test is employed to find solutions robustly and rapidly. Due to the special structure of the objective function, the solution of the convex lower bounding problem yields a nonsmooth root exclusion test which is found to perform better than earlier interval-analysis based exclusion tests. Both the componentwise Krawczyk operator and interval-Newton operator with Gauss-Seidel based root inclusion and exclusion tests are also embedded in the proposed algorithm to refine the variable bounds for efficient fathoming of the search space. The performance of the algorithm on a variety of test problems from the literature is presented, and for most of them, the first solution is found at the first iteration of the algorithm due to the good starting point generation.     

Approximate Controllability for Linear Stochastic Differential Equations in Infinite Dimensions

The objective of the paper is to investigate the approximate controllability property of a linear stochastic control system with values in a separable real Hilbert space. In a first step we prove the existence and uniqueness for the solution of the dual linear backward stochastic differential equation. This equation has the particularity that in addition to an unbounded operator acting on the Y-component of the solution there is still another one acting on the Z-component. With the help of this dual equation we then deduce the duality between approximate controllability and observability. Finally, under the assumption that the unbounded operator acting on the state process of the forward equation is an infinitesimal generator of an exponentially stable semigroup, we show that the generalized Hautus test provides a necessary condition for the approximate controllability. The paper generalizes former results by Buckdahn, Quincampoix and Tessitore (Stochastic Partial Differential Equations and Applications, Series of Lecture Notes in Pure and Appl. Math., vol. 245, pp. 253–260, Chapman and Hall, London, 2006) and Goreac (Applied Analysis and Differential Equations, pp. 153–164, World Scientific, Singapore, 2007) from the finite dimensional to the infinite dimensional case.     

A Note on the Alternating Direction Method of Multipliers

We consider the linearly constrained separable convex programming, whose objective function is separable into m individual convex functions without coupled variables. The alternating direction method of multipliers has been well studied in the literature for the special case m=2, while it remains open whether its convergence can be extended to the general case m≥3. This note shows the global convergence of this extension when the involved functions are further assumed to be strongly convex.     

Interval differential evolution with dimension-reduction interval analysis method for uncertain optimization problems

A constrained nonlinear interval optimization method under the framework of differential evolution algorithm is developed to solve the uncertain structural optimization problems with interval uncertainties. The proposed method is a direct optimization method based on the interval differential evolution and dimension-reduction interval analysis. The interval preferential rule based on the satisfaction value of interval possibility degree model is used to realize the direct interval ranking of different design vectors. At each evolutionary generation, the outer optimizer by differential evolution optimizer searches for the best solution within the design space. The dimension-reduction interval analysis is employed to calculate the intervals of objective and constraints for each design vector in the inner layer. This operation transforms the original nesting optimization problem into a single loop one which improves the computational efficiency of the proposed method. Finally, the effectiveness of the presented direct method is verified by two numerical examples and an engineering application.     

On a class of some special sets on thek-skeleton of a convex compact set

In this paper, generalizing the notion of a path we define ak-area to be the setD={g(t):t ∈J} on thek-skeleton of a convex compact setK in a Hilbert space, whereg is a continuous injection map from thek-dimensional convex compact setJ to thek-skeleton ofK. We also define anE ^k-area onK, whereE ^k is ak-dimensional subspace, to be ak-area with the propertyπ(g(t))=t,t ∈π(K), whereπ is the orthogonal projection onE ^k. This definition generalizes the notion of an increasing path on the 1-skeleton ofK. The existence of such sets is studied whenK is a subset of a Euclidean space or of a Hilbert space. Finally some conjectures are quoted for the number of such sets in some special cases.     

Some characterizations for SOC-monotone and SOC-convex functions

We provide some characterizations for SOC-monotone and SOC-convex functions by using differential analysis. From these characterizations, we particularly obtain that a continuously differentiable function defined in an open interval is SOC-monotone (SOC-convex) of order n ≥ 3 if and only if it is 2-matrix monotone (matrix convex), and furthermore, such a function is also SOC-monotone (SOC-convex) of order n ≤ 2 if it is 2-matrix monotone (matrix convex). In addition, we also prove that Conjecture 4.2 proposed in Chen (Optimization 55:363–385, 2006) does not hold in general. Some examples are included to illustrate that these characterizations open convenient ways to verify the SOC-monotonicity and the SOC-convexity of a continuously differentiable function defined on an open interval, which are often involved in the solution methods of the convex second-order cone optimization.