Least Squares Convex-Concave Data Smoothing

We consider n noisy measurements of a smooth (unknown) function, which suggest that the graph of the function consists of one convex and one concave section. Due to the noise the sequence of the second divided differences of the data exhibits more sign changes than those expected in the second derivative of the underlying function. We address the problem of smoothing the data so as to minimize the sum of squares of residuals subject to the condition that the sequence of successive second divided differences of the smoothed values changes sign at most once. It is a nonlinear problem, since the position of the sign change is also an unknown of the optimization process. We state a characterization theorem, which shows that the smoothed values can be derived by at most 2n – 2 quadratic programming calculations to subranges of data. Then, we develop an algorithm that solves the problem in about O(n ²) computer operations by employing several techniques, including B-splines, the use of active sets, quadratic programming and updating methods. A Fortran program has been written and some of its numerical results are presented. Applications of the smoothing technique may be found in scientific, economic and engineering calculations, when a potential shape for the underlying function is an S-curve. Generally, the smoothing calculation may arise from processes that show initially increasing and then decreasing rates of change.     

A robust support vector regression with a linear-log concave loss function

Support vector regression (SVR) is one of the most popular nonlinear regression techniques with the aim to approximate a nonlinear system with a good generalization capability. However, SVR has a major drawback in that it is sensitive to the presence of outliers. The ramp loss function for robust SVR has been introduced to resolve this problem, but SVR with ramp loss function has a non-differentiable and non-convex formulation, which is not easy to solve. Consequently, SVR with the ramp loss function requires smoothing and Concave-Convex Procedure techniques, which transform the non-differentiable and non-convex optimization to a differentiable and convex one. We present a robust SVR with linear-log concave loss function (RSLL), which does not require the transformation technique, where the linear-log concave loss function has a similar effect as the ramp loss function. The zero norm approximation and the difference of convex functions problem are employed for solving the optimization problem. The proposed RSLL approach is used to develop a robust and stable virtual metrology (VM) prediction model, which utilizes the status variables of process equipment to predict the process quality of wafer level in semiconductor manufacturing. We also compare the proposed approach to existing SVR-based methods in terms of the root mean squared error of prediction using both synthetic and real data sets. Our experimental results show that the proposed approach performs better than existing SVR-based methods regardless of the data set and type of outliers (ie, X-space and Y-space outliers), implying that it can be used as a useful alternative when the regression data contain outliers.     

Convergence analysis of an optimally accurate frozen multi-level projected steepest descent iteration for solving inverse problems

In this paper, we introduce a novel projected steepest descent iterative method with frozen derivative. The classical projected steepest descent iterative method involves the computation of derivative of the nonlinear operator at each iterate. The method of this paper requires the computation of derivative of the nonlinear operator only at an initial point. We exhibit the convergence analysis of our method by assuming the conditional stability of the inverse problem on a convex and compact set. Further, by assuming the conditional stability on a nested family of convex and compact subsets, we develop a multi-level method. In order to enhance the accuracy of approximation between neighboring levels, we couple it with the growth of stability constants. This along with a suitable discrepancy criterion ensures that the algorithm proceeds from level to level and terminates within finite steps. Finally, we discuss an inverse problem on which our methods are applicable.     

Duality and well-posedness in convex interpolation ∗)

We consider the problem for convex interpolation with minimal L_p norm of the second derivative, 1 < p < +α. Convergence of a class of dual methods is established and numerical results are presented. It is proved that if p 2 then the solution of the problem is locally Lipschitz with respect to the data in the uniform metric.     

Variational Interpolation of Subsets

We consider the problem of the variational interpolation of subsets of Euclidean spaces by curves such that the L² norm of the second derivative is minimized. It is well-known that the resulting curves are cubic spline curves. We study geometric boundary conditions arising for various types of subsets such as subspaces, polyhedra, and submanifolds, and we indicate how solutions can be computed in the case of convex polyhedra.     

On the problems of smoothing and near-interpolation

In the first part of this paper we apply a saddle point theorem from convex analysis to show that various constrained minimization problems are equivalent to the problem of smoothing by spline functions. In particular, we show that near-interpolants are smoothing splines with weights that arise as Lagrange multipliers corresponding to the constraints in the problem of near-interpolation. In the second part of this paper we apply certain fixed point iterations to compute these weights. A similar iteration is applied to the computation of the smoothing parameter in the problem of smoothing.

     

A Newton‐type algorithm for solving an extremal constrained interpolation problem

Given convex scattered data in R³ we consider the constrained interpolation problem of finding a smooth, minimal L _p‐norm (1 < p < ∞) interpolation network that is convex along the edges of an associated triangulation. In previous work the problem has been reduced to the solution of a nonlinear system of equations. In this paper we formulate and analyse a Newton‐type algorithm for solving the corresponding type of systems. The correctness of the application of the proposed method is proved and its superlinear (in some cases quadratic) convergence is shown. Copyright © 2000 John Wiley & Sons, Ltd.     

Quadratic Convergence of Newton's Method for Convex Interpolation and Smoothing

   Abstract. In this paper, we prove that Newton's method for convex best interpolation is locally quadratically convergent, giving an answer to a question of Irvine, Marin, and Smith [7] and strengthening a result of Andersson and Elfving [1] and our previous work [5]. A damped Newton-type method is presented which has global quadratic convergence. Analogous results are obtained for the convex smoothing problem. Numerical examples are presented.     

Quadratic Convergence of Newton's Method for Convex Interpolation and Smoothing

Abstract. In this paper, we prove that Newton's method for convex best interpolation is locally quadratically convergent, giving an answer to a question of Irvine, Marin, and Smith [7] and strengthening a result of Andersson and Elfving [1] and our previous work [5]. A damped Newton-type method is presented which has global quadratic convergence. Analogous results are obtained for the convex smoothing problem. Numerical examples are presented.     

Lower complexity bounds for interpolation algorithms

We introduce and discuss a new computational model for the Hermite-Lagrange interpolation with nonlinear classes of polynomial interpolants. We distinguish between an interpolation problem and an algorithm that solves it. Our model includes also coalescence phenomena and captures a large variety of known Hermite-Lagrange interpolation problems and algorithms. Like in traditional Hermite-Lagrange interpolation, our model is based on the execution of arithmetic operations (including divisions) in the field where the data (nodes and values) are interpreted and arithmetic operations are counted at unit cost. This leads us to a new view of rational functions and maps defined on arbitrary constructible subsets of complex affine spaces. For this purpose we have to develop new tools in algebraic geometry which themselves are mainly based on Zariski’s Main Theorem and the theory of places (or equivalently: valuations). We finish this paper by exhibiting two examples of Lagrange interpolation problems with nonlinear classes of interpolants, which do not admit efficient interpolation algorithms (one of these interpolation problems requires even an exponential quantity of arithmetic operations in terms of the number of the given nodes in order to represent some of the interpolants).In other words, classic Lagrange interpolation algorithms are asymptotically optimal for the solution of these selected interpolation problems and nothing is gained by allowing interpolation algorithms and classes of interpolants to be nonlinear. We show also that classic Lagrange interpolation algorithms are almost optimal for generic nodes and values. This generic data cannot be substantially compressed by using nonlinear techniques.We finish this paper highlighting the close connection of our complexity results in Hermite-Lagrange interpolation with a modern trend in software engineering: architecture tradeoff analysis methods (ATAM).     

On the acceleration of the double smoothing technique for unconstrained convex optimization problems

In this article, we investigate the possibilities of accelerating the double smoothing (DS) technique when solving unconstrained nondifferentiable convex optimization problems. This approach relies on the regularization in two steps of the Fenchel dual problem associated with the problem to be solved into an optimization problem having a differentiable strongly convex objective function with Lipschitz continuous gradient. The doubly regularized dual problem is then solved via a fast gradient method. The aim of this article is to show how the properties of the functions in the objective of the primal problem influence the implementation of the DS approach and its rate of convergence. The theoretical results are applied to linear inverse problems by making use of different regularization functionals.     

Simple solution methods for separable mixed linear and quadratic knapsack problem

Quadratic knapsack problem has a central role in integer and nonlinear optimization, which has been intensively studied due to its immediate applications in many fields and theoretical reasons. Although quadratic knapsack problem can be solved using traditional nonlinear optimization methods, specialized algorithms are much faster and more reliable than the nonlinear programming solvers. In this paper, we study a mixed linear and quadratic knapsack with a convex separable objective function subject to a single linear constraint and box constraints. We investigate the structural properties of the studied problem, and develop a simple method for solving the continuous version of the problem based on bi-section search, and then we present heuristics for solving the integer version of the problem. Numerical experiments are conducted to show the effectiveness of the proposed solution methods by comparing our methods with some state of the art linear and quadratic convex solvers.     

A new regularized stochastic approximation framework for stochastic inverse problems

We study the nonlinear inverse problem of estimating stochastic parameters in the fourth-order partial differential equation with random data. The primary focus is on developing a novel stochastic approximation framework for inverse problems consisting of three key components. As a first step, we reformulate the inverse problem into a stochastic convex optimization problem. The second step includes developing a new regularized stochastic extragradient framework for a nonlinear variational inequality, which subsumes the optimality conditions for the optimization formulation of the inverse problem. The third step involves modeling random variables by a Karhunen–Loève type finite-dimensional noise representation, allowing the direct and the inverse problems to be conveniently discretized. We show that the regularized extragradient methods are strongly convergent in a Hilbert space setting, and we also provide several auxiliary results for the inverse problem, including Lipschitz continuity and a derivative characterization of the solution map. We provide the outcome of computational experiments to estimate stochastic and deterministic parameters. The numerical results demonstrate the feasibility and effectiveness of the developed framework and validate stochastic approximation as an effective method for stochastic inverse problems.     

INTERPOLATION WITH RESTRICTED ARC LENGTH

AbstractFor given data (t_i,y_i),i=0, 1,…,n,0=t_0相似文献   

Second order evolution equations with time-dependent subdifferentials

We study an abstract second order nonlinear evolution equation in a real Hilbert space. We consider time-dependent convex functions and their subdifferentials operating on the first derivative of the unknown function. Introducing appropriate assumptions on the convex functions and other data, we prove the existence and uniqueness of a strong solution, and give some applications of the abstract theorem to hyperbolic variational inequalities with time-dependent constraints.      

A dual algorithm for convex-concave data smoothing by cubicC 2-splines

Summary In this paper the problem of smoothing a given data set by cubicC ²-splines is discussed. The spline may required to be convex in some parts of the domain and concave in other parts. Application of splines has the advantage that the smoothing problem is easily discretized. Moreover, the special structure of the arising finite dimensional convex program allows a dualization such that the resulting concave dual program is unconstrained. Therefore the latter program is treated numerically much more easier than the original program. Further, the validity of a return-formula is of importance by which a minimizer of the orginal program is obtained from a maximizer of the dual program.The theoretical background of this general approach is discussed and, above all, the details for applying the strategy to the present smoothing problem are elaborated. Also some numerical tests are presented.     

A smoothing Newton-type method for generalized nonlinear complementarity problem

By using a new type of smoothing function, we first reformulate the generalized nonlinear complementarity problem over a polyhedral cone as a smoothing system of equations, and then develop a smoothing Newton-type method for solving it. For the proposed method, we obtain its global convergence under milder conditions, and we further establish its local superlinear (quadratic) convergence rate under the BD-regular assumption. Preliminary numerical experiments are also reported in this paper.     

Controllability for systems governed by semilinear evolution inclusions without compactness

In this paper we study the controllability for a class of semilinear differential inclusions in Banach spaces. Since we assume the regularity of the nonlinear part with respect to the weak topology, we do not require the compactness of the evolution operator generated by the linear part. As well we are not posing any conditions on the multivalued nonlinearity expressed in terms of measures of noncompactness. We are considering the usual assumption on the controllability of the associated linear problem. Notice that, in infinite dimensional spaces, the above mentioned compactness of the evolution operator and linear controllability condition are in contradiction with each other. We suppose that the nonlinear term has convex, closed, bounded values and a weakly sequentially closed graph when restricted to its second argument. This regularity setting allows us to solve controllability problem under various growth conditions. As application, a controllability result for hyperbolic integro-differential equations and inclusions is obtained. In particular, we consider controllability of a system arising in a model of nonlocal spatial population dispersal and a system governed by the second order one-dimensional telegraph equation.     

A Geometric Programming Framework for Univariate Cubic L 1 Smoothing Splines

Univariate cubic L ₁ smoothing splines are capable of providing shape-preserving C ¹-smooth approximation of multi-scale data. The minimization principle for univariate cubic L ₁ smoothing splines results in a nondifferentiable convex optimization problem that, for theoretical treatment and algorithm design, can be formulated as a generalized geometric program. In this framework, a geometric dual with a linear objective function over a convex feasible domain is derived, and a linear system for dual to primal conversion is established. Numerical examples are given to illustrate this approach. Sensitivity analysis for data with uncertainty is presented. This work is supported by research grant #DAAG55-98-D-0003 of the Army Research Office, USA.