Minimal-energy splines: I. Plane curves with angle constraints

This is the first in a series of papers on minimal-energy splines. The paper is devoted to plane minimal-energy splines with angle constraints. We first consider minimal-energy spline segments, then general minimal-energy spline curves. We formulate problems for minimal-energy spline segments and curves, prove the existence of solutions, justify the Lagrange multiplier rules, and obtain some nice properties (e.g., the infinite smoothness). Finally, we report our computational experience on minimal-energy splines.     

Estimating arrival rate of nonhomogeneous Poisson processes with semidefinite programming

We present a summary of methods based on semidefinite programming for estimating arrival rate of nonhomogeneous Poisson processes from a finite set of observed data. Both one-dimensional time dependent, and multi-dimensional time and location dependent rates are considered. The arrival rate is a nonnegative function of time (or time and location). We also assume that it is a smooth function with continuous derivatives of up to certain order k. We estimate the rate function by one or multi-dimensional splines, with the additional condition that the underlying rate function is nonnegative. This approach results in an optimization problem over nonnegative polynomials, which can be modeled and solved using semidefinite programming. We also describe a method which requires only linear constraints. Numerical results based on e-mail arrival and highway accidents are presented.     

On p-norm linear discrimination

We consider a p-norm linear discrimination model that generalizes the model of Bennett and Mangasarian (1992) and reduces to a linear programming problem with p-order cone constraints. The proposed approach for handling linear programming problems with p-order cone constraints is based on reformulation of p-order cone optimization problems as second order cone programming (SOCP) problems when p is rational. Since such reformulations typically lead to SOCP problems with large numbers of second order cones, an “economical” representation that minimizes the number of second order cones is proposed. A case study illustrating the developed model on several popular data sets is conducted.     

Variable selection for the partial linear single-index model

In this paper, we consider the issue of variable selection in partial linear single-index models under the assumption that the vector of regression coefficients is sparse. We apply penalized spline to estimate the nonparametric function and SCAD penalty to achieve sparse estimates of regression parameters in both the linear and single-index parts of the model. Under some mild conditions, it is shown that the penalized estimators have oracle property, in the sense that it is asymptotically normal with the same mean and covariance that they would have if zero coefficients are known in advance. Our model owns a least square representation, therefore standard least square programming algorithms can be implemented without extra programming efforts. In the meantime, parametric estimation, variable selection and nonparametric estimation can be realized in one step, which incredibly increases computational stability. The finite sample performance of the penalized estimators is evaluated through Monte Carlo studies and illustrated with a real data set.     

Learning mixtures of polynomials of multidimensional probability densities from data using B-spline interpolation

Non-parametric density estimation is an important technique in probabilistic modeling and reasoning with uncertainty. We present a method for learning mixtures of polynomials (MoPs) approximations of one-dimensional and multidimensional probability densities from data. The method is based on basis spline interpolation, where a density is approximated as a linear combination of basis splines. We compute maximum likelihood estimators of the mixing coefficients of the linear combination. The Bayesian information criterion is used as the score function to select the order of the polynomials and the number of pieces of the MoP. The method is evaluated in two ways. First, we test the approximation fitting. We sample artificial datasets from known one-dimensional and multidimensional densities and learn MoP approximations from the datasets. The quality of the approximations is analyzed according to different criteria, and the new proposal is compared with MoPs learned with Lagrange interpolation and mixtures of truncated basis functions. Second, the proposed method is used as a non-parametric density estimation technique in Bayesian classifiers. Two of the most widely studied Bayesian classifiers, i.e., the naive Bayes and tree-augmented naive Bayes classifiers, are implemented and compared. Results on real datasets show that the non-parametric Bayesian classifiers using MoPs are comparable to the kernel density-based Bayesian classifiers. We provide a free R package implementing the proposed methods.     

Shape constrained smoothing using smoothing splines

Summary  In some regression settings one would like to combine the flexibility of nonparametric smoothing with some prior knowledge about the regression curve. Such prior knowledge may come from a physical or economic theory, leading to shape constraints such as the underlying regression curve being positive, monotone, convex or concave. We propose a new method for calculating smoothing splines that fulfill these kinds of constraints. Our approach leads to a quadratic programming problem and the infinite number of constraints are replaced by a finite number of constraints that are chosen adaptively. We show that the resulting problem can be solved using the algorithm of Goldfarb and Idnani (1982, 1983) and illustrate our method on several real data sets.     

Bayesian Free-Knot Monotone Cubic Spline Regression

This article proposes a new Bayesian approach for monotone curve fitting based on the isotonic regression model. The unknown monotone regression function is approximated by a cubic spline and the constraints are represented by the intersection of quadratic cones. We treat the number and locations of knots as free parameters and use reversible jump Markov chain Monte Carlo to obtain posterior samples of knot configurations. Given the number and locations of the knots, second-order cone programming is used to estimate the remaining parameters. Simulation results suggest the method performs well and we illustrate the approach using the ASA car data.     

Conic mixed-integer rounding cuts

A conic integer program is an integer programming problem with conic constraints. Many problems in finance, engineering, statistical learning, and probabilistic optimization are modeled using conic constraints. Here we study mixed-integer sets defined by second-order conic constraints. We introduce general-purpose cuts for conic mixed-integer programming based on polyhedral conic substructures of second-order conic sets. These cuts can be readily incorporated in branch-and-bound algorithms that solve either second-order conic programming or linear programming relaxations of conic integer programs at the nodes of the branch-and-bound tree. Central to our approach is a reformulation of the second-order conic constraints with polyhedral second-order conic constraints in a higher dimensional space. In this representation the cuts we develop are linear, even though they are nonlinear in the original space of variables. This feature leads to a computationally efficient implementation of nonlinear cuts for conic mixed-integer programming. The reformulation also allows the use of polyhedral methods for conic integer programming. We report computational results on solving unstructured second-order conic mixed-integer problems as well as mean–variance capital budgeting problems and least-squares estimation problems with binary inputs. Our computational experiments show that conic mixed-integer rounding cuts are very effective in reducing the integrality gap of continuous relaxations of conic mixed-integer programs and, hence, improving their solvability. This research has been supported, in part, by Grant # DMI0700203 from the National Science Foundation.     

Preprocessing and cutting for multiple allocation hub location problems

In this paper we consider formulations and solution approaches for multiple allocation hub location problems. We present a number of results, which enable us to develop preprocessing procedures and tightening constraints for existing mixed integer linear programming formulations. We employ flow cover constraints for capacitated problems to improve computation times. We present the results of our computational experience, which show that all of these steps can effectively reduce the computational effort required to obtain optimal solutions.     

Minimal conic quadratic reformulations and an optimization model

In this paper, we consider a particular form of inequalities which involves product of multiple variables with rational exponents. These inequalities can equivalently be represented by a number of conic quadratic forms called cone constraints. We propose an integer programming model and a heuristic algorithm to obtain the minimum number of cone constraints which equivalently represent the original inequality. The performance of the proposed algorithm and the computational effect of reformulations are numerically illustrated.     

Nonnegative splines, in particular of degree five

Summary. We investigate splines from a variational point of view, which have the following properties: (a) they interpolate given data, (b) they stay nonnegative, when the data are positive, (c) for a given integer they minimize the functional for all nonnegative, interpolating . We extend known results for to larger , in particular to and we find general necessary conditions for solutions of this restricted minimization problem. These conditions imply that solutions are splines in an augmented grid. In addition, we find that the solutions are in and consist of piecewise polynomials in with respect to the augmented grid. We find that for general, odd there will be no boundary arcs which means (nontrivial) subintervals in which the spline is identically zero. We show also that the occurrence of a boundary arc in an interval between two neighboring knots prohibits the existence of any further knot in that interval. For we show that between given neighboring interpolation knots, the augmented grid has at most two additional grid points. In the case of two interpolation knots (the local problem) we develop polynomial equations for the additional grid points which can be used directly for numerical computation. For the general (global) problem we propose an algorithm which is based on a Newton iteration for the additional grid points and which uses the local spline data as an initial guess. There are extensions to other types of constraints such as two-sided restrictions, also ones which vary from interval to interval. As an illustration several numerical examples including graphs of splines manufactured by MATLAB- and FORTRAN-programs are given. Received November 16, 1995 / Revised version received February 24, 1997     

A result in surrogate duality for certain integer programming problems

We consider linear programming problems with some equality constraints. For such problems, surrogate relaxation formulations relaxing equality constraints existwith zero primal-dual gap both when all variables are restricted to be integers and when no variable is required to be integer. However, for such surrogate formulations, when the variables are mixed-integer, the primal-dual gap may not be zero. We establish this latter result by a counterexample.     

Robust univariate spline models for interpolating interval data

We consider spline interpolation problems where information about the approximated function is given by means of interval estimates for the function values over ranges of x-values instead of specific knots. We propose two robust univariate spline models formulated as convex semi-infinite optimization problems. We present simplified equivalent formulations of both models as finite explicit convex optimization problems for splines of degrees up to 3. This makes it possible to use existing convex optimization algorithms and software.     

Computing ECT-B-splines recursively

ECT-spline curves for sequences of multiple knots are generated from different local ECT-systems via connection matrices. Under appropriate assumptions there is a basis of the space of ECT-splines consisting of functions having minimal compact supports, normalized to form a nonnegative partition of unity. The basic functions can be defined by generalized divided differences [24]. This definition reduces to the classical one in case of a Schoenberg space. Under suitable assumptions it leads to a recursive method for computing the ECT-B-splines that reduces to the de Boor–Mansion–Cox recursion in case of ordinary polynomial splines and to Lyche's recursion in case of Tchebycheff splines. For sequences of simple knots and connection matrices that are nonsingular, lower triangular and totally positive the spline weights are identified as Neville–Aitken weights of certain generalized interpolation problems. For multiple knots they are limits of Neville–Aitken weights. In many cases the spline weights can be computed easily by recurrence. Our approach covers the case of Bézier-ECT-splines as well. They are defined by different local ECT-systems on knot intervals of a finite partition of a compact interval [a,b] connected at inner knots all of multiplicities zero by full connection matrices A ^[i] that are nonsingular, lower triangular and totally positive. In case of ordinary polynomials of order n they reduce to the classical Bézier polynomials. We also present a recursive algorithm of de Boor type computing ECT-spline curves pointwise. Examples of polynomial and rational B-splines constructed from given knot sequences and given connection matrices are added. For some of them we give explicit formulas of the spline weights, for others we display the B-splines or the B-spline curves. *Supported in part by INTAS 03-51-6637.     

Completely positive reformulations for polynomial optimization

Polynomial optimization encompasses a very rich class of problems in which both the objective and constraints can be written in terms of polynomials on the decision variables. There is a well established body of research on quadratic polynomial optimization problems based on reformulations of the original problem as a conic program over the cone of completely positive matrices, or its conic dual, the cone of copositive matrices. As a result of this reformulation approach, novel solution schemes for quadratic polynomial optimization problems have been designed by drawing on conic programming tools, and the extensively studied cones of completely positive and of copositive matrices. In particular, this approach has been applied to solve key combinatorial optimization problems. Along this line of research, we consider polynomial optimization problems that are not necessarily quadratic. For this purpose, we use a natural extension of the cone of completely positive matrices; namely, the cone of completely positive tensors. We provide a general characterization of the class of polynomial optimization problems that can be formulated as a conic program over the cone of completely positive tensors. As a consequence of this characterization, it follows that recent related results for quadratic problems can be further strengthened and generalized to higher order polynomial optimization problems. Also, we show that the conditions underlying the characterization are conceptually the same, regardless of the degree of the polynomials defining the problem. To illustrate our results, we discuss in further detail special and relevant instances of polynomial optimization problems.     

Construction of Normalized B-Splines for a Family of Smooth Spline Spaces Over Powell–Sabin Triangulations

We construct a suitable B-spline representation for a family of bivariate spline functions with smoothness r≥1 and polynomial degree 3r?1. They are defined on a triangulation with Powell–Sabin refinement. The basis functions have a local support, they are nonnegative, and they form a partition of unity. The construction involves the determination of triangles that must contain a specific set of points. We further consider a number of CAGD applications. We show how to define control points and control polynomials (of degree 2r?1), and we provide an efficient and stable computation of the Bernstein–Bézier form of such splines.     

Optimal smoothing and interpolating splines with constraints

This paper considers the problem for designing optimal smoothing and interpolating splines with equality and/or inequality constraints. The splines are constituted by employing normalized uniform B-splines as the basis functions, namely as weighted sum of shifted B-splines of degree k. Then a central issue is to determine an optimal vector of the so-called control points. By employing such an approach, it is shown that various types of constraints are formulated as linear function of the control points, and the problems reduce to quadratic programming problems. We demonstrate the effectiveness and usefulness by numerical examples including approximation of probability density functions, approximation of discontinuous functions, and trajectory planning.     

A spline smoothing homotopy method for nonconvex nonlinear programming

Homotopy methods are globally convergent under weak conditions and robust; however, the efficiency of a homotopy method is closely related with the construction of the homotopy map and the path tracing algorithm. Different homotopies may behave very different in performance even though they are all theoretically convergent. In this paper, a spline smoothing homotopy method for nonconvex nonlinear programming is developed using cubic spline to smooth the max function of the constraints of nonlinear programming. Some properties of spline smoothing function are discussed and the global convergence of spline smoothing homotopy under the weak normal cone condition is proven. The spline smoothing technique uses a smooth constraint instead of m constraints and acts also as an active set technique. So the spline smoothing homotopy method is more efficient than previous homotopy methods like combined homotopy interior point method, aggregate constraint homotopy method and other probability one homotopy methods. Numerical tests with the comparisons to some other methods show that the new method is very efficient for nonlinear programming with large number of complicated constraints.     

Detecting copositivity of a symmetric matrix by an adaptive ellipsoid-based approximation scheme

It is co-NP-complete to decide whether a given matrix is copositive or not. In this paper, this decision problem is transformed into a quadratic programming problem, which can be approximated by solving a sequence of linear conic programming problems defined on the dual cone of the cone of nonnegative quadratic functions over the union of a collection of ellipsoids. Using linear matrix inequalities (LMI) representations, each corresponding problem in the sequence can be solved via semidefinite programming. In order to speed up the convergence of the approximation sequence and to relieve the computational effort of solving linear conic programming problems, an adaptive approximation scheme is adopted to refine the union of ellipsoids. The lower and upper bounds of the transformed quadratic programming problem are used to determine the copositivity of the given matrix.     

An Interior-Point Method for a Class of Saddle-Point Problems

We present a polynomial-time interior-point algorithm for a class of nonlinear saddle-point problems that involve semidefiniteness constraints on matrix variables. These problems originate from robust optimization formulations of convex quadratic programming problems with uncertain input parameters. As an application of our approach, we discuss a robust formulation of the Markowitz portfolio selection model.