Shape-Constrained Estimation Using Nonnegative Splines

We consider the problem of nonparametric estimation of unknown smooth functions in the presence of restrictions on the shape of the estimator and on its support using polynomial splines. We provide a general computational framework that treats these estimation problems in a unified manner, without the limitations of the existing methods. Applications of our approach include computing optimal spline estimators for regression, density estimation, and arrival rate estimation problems in the presence of various shape constraints. Our approach can also handle multiple simultaneous shape constraints. The approach is based on a characterization of nonnegative polynomials that leads to semidefinite programming (SDP) and second-order cone programming (SOCP) formulations of the problems. These formulations extend and generalize a number of previous approaches in the literature, including those with piecewise linear and B-spline estimators. We also consider a simpler approach in which nonnegative splines are approximated by splines whose pieces are polynomials with nonnegative coefficients in a nonnegative basis. A condition is presented to test whether a given nonnegative basis gives rise to a spline cone that is dense in the space of nonnegative continuous functions. The optimization models formulated in the article are solvable with minimal running time using off-the-shelf software. We provide numerical illustrations for density estimation and regression problems. These examples show that the proposed approach requires minimal computational time, and that the estimators obtained using our approach often match and frequently outperform kernel methods and spline smoothing without shape constraints. Supplementary materials for this article are provided online.     

Semidefinite diagonal directions Monte Carlo algorithms for detecting necessary linear matrix inequality constraints

Hit-and-run algorithms are Monte Carlo methods for detecting necessary constraints in convex programming including semidefinite programming. The well known of these in semidefinite programming are semidefinite coordinate directions (SCD), semidefinite hypersphere directions (SHD) and semidefinite stand-and-hit (SSH) algorithms. SCD is considered to be the best on average and hence we use it for comparison.We develop two new hit-and-run algorithms in semidefinite programming that use diagonal directions. They are the uniform semidefinite diagonal directions (uniform SDD) and the original semidefinite diagonal directions (original SDD) algorithms. We analyze the costs and benefits of this change in comparison with SCD. We also show that both uniform SDD and original SDD generate points that are asymptotically uniform in the interior of the feasible region defined by the constraints.     

Moments and sums of squares for polynomial optimization and related problems

We briefly review the duality between moment problems and sums of squares (s.o.s.) representations of positive polynomials, and compare s.o.s. versus nonnegative polynomials. We then describe how to use such results to define convergent semidefinite programming relaxations in polynomial optimization as well as for the two related problems of computing the convex envelope of a rational function and finding all zeros of a system of polynomial equations.     

非凸半定规划的广义Fakars引理及最优性条件

1引言在本文中,我们用(?),S~n,S_ ~n分别表示有限维向量空间,n阶对称矩阵空间及n阶半正定矩阵锥.我们考虑如下形式的非凸半定规划问题:     

An ϵ-sensitivity analysis for semidefinite programming

We extend the concept of ϵ-sensitivity analysis developed for linear programming to that for semidefinite programming. First, the notion of ϵ-optimality for a given semidefinite programming problem is defined, and then a generic ϵ-sensitivity analysis for semidefinite programming is introduced. Based on the definitions, we develop an implementation of the generic ϵ-sensitivity analysis under perturbations of either the cost parameters or the right-hand side.     

An alternating direction implicit scheme for discrete ordinate transport equations

This paper concerns a finite difference approximation of the discrete ordinate equations for the time-dependent linear transport equation posed in a multi-dimensional rectangular parallelepiped with partially reflecting walls. We present an unconditionally stable alternating direction implicit finite difference scheme, show how to solve the difference equations, and establish the following properties of the scheme.If a sequence of difference approximations is considered in which the time and space increments approach zero, then the corresponding sequence of solutions has a subsequence which converges continuously to a strong solution of the discrete ordinate equations. Provided that the time increment is sufficiently small, independently of the space and velocity increment sizes: the solution of the difference equations is bounded by an exponential function of time; in the subcritical case the coefficient of t in this exponential bound is zero or negative; and if the constituent functions are all nonnegative, then the solution of the difference equations will also be nonnegative. This last result implies a monotonicity principle for solutions of related difference problems.     

Lower Bounds on Matrix Factorization Ranks via Noncommutative Polynomial Optimization

We use techniques from (tracial noncommutative) polynomial optimization to formulate hierarchies of semidefinite programming lower bounds on matrix factorization ranks. In particular, we consider the nonnegative rank, the positive semidefinite rank, and their symmetric analogs: the completely positive rank and the completely positive semidefinite rank. We study convergence properties of our hierarchies, compare them extensively to known lower bounds, and provide some (numerical) examples.

     

Enhancing RLT relaxations via a new class of semidefinite cuts

In this paper, we propose a mechanism to tighten Reformulation-Linearization Technique (RLT) based relaxations for solving nonconvex programming problems by importing concepts from semidefinite programming (SDP), leading to a new class of semidefinite cutting planes. Given an RLT relaxation, the usual nonnegativity restrictions on the matrix of RLT product variables is replaced by a suitable positive semidefinite constraint. Instead of relying on specific SDP solvers, the positive semidefinite stipulation is re-written to develop a semi-infinite linear programming representation of the problem, and an approach is developed that can be implemented using traditional optimization software. Specifically, the infinite set of constraints is relaxed, and members of this set are generated as needed via a separation routine in polynomial time. In essence, this process yields an RLT relaxation that is augmented with valid inequalities, which are themselves classes of RLT constraints that we call semidefinite cuts. These semidefinite cuts comprise a relaxation of the underlying semidefinite constraint. We illustrate this strategy by applying it to the case of optimizing a nonconvex quadratic objective function over a simplex. The algorithm has been implemented in C++, using CPLEX callable routines, and two types of semidefinite restrictions are explored along with several implementation strategies. Several of the most promising lower bounding strategies have been implemented within a branch-and-bound framework. Computational results indicate that the cutting plane algorithm provides a significant tightening of the lower bound obtained by using RLT alone. Moreover, when used within a branch-and-bound framework, the proposed lower bound significantly reduces the effort required to obtain globally optimal solutions.     

广义负相依一般风险模型中有限时破产概率的估计及数值模拟

本文研究了一类带利率的重尾相依风险模型, 其中索赔额是一列上广义负相依随机变量, 索赔到达过程是一般的非负整值过程, 并且独立于索赔额序列, 保费收入过程是一个一般的非负非降随机过程. 我们考虑了两种情况, 其一是索赔额、索赔到达过程及保费收入过程相互独立, 其二是累积折现保费收入总量的尾概率可以被索赔额的尾概率高阶控制, 得到了保险公司有限时破产概率的渐近估计,并且给出了相应的数值模拟, 验证了理论结果的合理性.     

Nonnegative factorization of positive semidefinite nonnegative matrices

We give simple geometric proofs of the known results that, for n ?4, n × n nonnegative positive semidefinite matrices can be factored into n × n nonnegative factors and that, for n ? 5, these conditions are not sufficient to guarantee the existence of such a factorization.     

Exact Solutions of Some Nonconvex Quadratic Optimization Problems via SDP and SOCP Relaxations

We show that SDP (semidefinite programming) and SOCP (second order cone programming) relaxations provide exact optimal solutions for a class of nonconvex quadratic optimization problems. It is a generalization of the results by S. Zhang for a subclass of quadratic maximization problems that have nonnegative off-diagonal coefficient matrices of quadratic objective functions and diagonal coefficient matrices of quadratic constraint functions. A new SOCP relaxation is proposed for the class of nonconvex quadratic optimization problems by extracting valid quadratic inequalities for positive semidefinite cones. Its effectiveness to obtain optimal values is shown to be the same as the SDP relaxation theoretically. Numerical results are presented to demonstrate that the SOCP relaxation is much more efficient than the SDP relaxation.     

Nonlinear semidefinite programming: sensitivity, convergence, and an application in passive reduced-order modeling

We consider the solution of nonlinear programs with nonlinear semidefiniteness constraints. The need for an efficient exploitation of the cone of positive semidefinite matrices makes the solution of such nonlinear semidefinite programs more complicated than the solution of standard nonlinear programs. This paper studies a sequential semidefinite programming (SSP) method, which is a generalization of the well-known sequential quadratic programming method for standard nonlinear programs. We present a sensitivity result for nonlinear semidefinite programs, and then based on this result, we give a self-contained proof of local quadratic convergence of the SSP method. We also describe a class of nonlinear semidefinite programs that arise in passive reduced-order modeling, and we report results of some numerical experiments with the SSP method applied to problems in that class.     

On duality theory for non-convex semidefinite programming

In this paper, with the help of convex-like function, we discuss the duality theory for nonconvex semidefinite programming. Our contributions are: duality theory for the general nonconvex semidefinite programming when Slater’s condition holds; perfect duality for a special case of the nonconvex semidefinite programming for which Slater’s condition fails. We point out that the results of Fan (Appl. Math. Lett. 18:1068–1073, 2005) can be regarded as a special case of our result.     

Characterization of symmetric M-matrices as resistive inverses

We aim here at characterizing those nonnegative matrices whose inverse is an irreducible Stieltjes matrix. Specifically, we prove that any irreducible Stieltjes matrix is a resistive inverse. To do this we consider the network defined by the off-diagonal entries of the matrix and we identify the matrix with a positive definite Schrödinger operator whose ground state is determined by the lowest eigenvalue of the matrix and the corresponding positive eigenvector. We also analyze the case in which the operator is positive semidefinite which corresponds to the study of singular irreducible symmetric M-matrices.     

A nonconvex ADMM for a class of sparse inverse semidefinite quadratic programming problems

ABSTRACT

In this paper, we consider a class of sparse inverse semidefinite quadratic programming problems, in which a nonconvex alternating direction method of multiplier is investigated. Under mild conditions, we establish convergence results of our algorithm and the corresponding non-ergodic iteration-complexity is also considered under the assumption that the potential function satisfies the famous Kurdyka–?ojasiewicz property. Numerical results show that our algorithm is suitable to solve the given sparse inverse semidefinite quadratic programming problems.     

非线性半定规划一个全局收敛的无罚无滤子SSDP算法

提出了一个求解非线性半定规划的无罚函数无滤子序列二次半定规划(SSDP)算法. 算法每次迭代只需求解一个二次半定规划子问题确定搜索方向; 非单调线搜索保证目标函数或约束违反度函数的充分下降, 从而产生新的迭代点. 在适当的假设条件下, 证明了算法的全局收敛性. 最后给出了初步的数值实验结果.     

Solving the bi-objective multi-dimensional knapsack problem exploiting the concept of core

This paper deals with the bi-objective multi-dimensional knapsack problem. We propose the adaptation of the core concept that is effectively used in single-objective multi-dimensional knapsack problems. The main idea of the core concept is based on the “divide and conquer” principle. Namely, instead of solving one problem with n variables we solve several sub-problems with a fraction of n variables (core variables). The quality of the obtained solution can be adjusted according to the size of the core and there is always a trade off between the solution time and the quality of solution. In the specific study we define the core problem for the multi-objective multi-dimensional knapsack problem. After defining the core we solve the bi-objective integer programming that comprises only the core variables using the Multicriteria Branch and Bound algorithm that can generate the complete Pareto set in small and medium size multi-objective integer programming problems. A small example is used to illustrate the method while computational and economy issues are also discussed. Computational experiments are also presented using available or appropriately modified benchmarks in order to examine the quality of Pareto set approximation with respect to the solution time. Extensions to the general multi-objective case as well as to the computation of the exact solution are also mentioned.     

On convex quadratic programs with linear complementarity constraints

The paper shows that the global resolution of a general convex quadratic program with complementarity constraints (QPCC), possibly infeasible or unbounded, can be accomplished in finite time. The method constructs a minmax mixed integer formulation by introducing finitely many binary variables, one for each complementarity constraint. Based on the primal-dual relationship of a pair of convex quadratic programs and on a logical Benders scheme, an extreme ray/point generation procedure is developed, which relies on valid satisfiability constraints for the integer program. To improve this scheme, we propose a two-stage approach wherein the first stage solves the mixed integer quadratic program with pre-set upper bounds on the complementarity variables, and the second stage solves the program outside this bounded region by the Benders scheme. We report computational results with our method. We also investigate the addition of a penalty term y ^T Dw to the objective function, where y and w are the complementary variables and D is a nonnegative diagonal matrix. The matrix D can be chosen effectively by solving a semidefinite program, ensuring that the objective function remains convex. The addition of the penalty term can often reduce the overall runtime by at least 50 %. We report preliminary computational testing on a QP relaxation method which can be used to obtain better lower bounds from infeasible points; this method could be incorporated into a branching scheme. By combining the penalty method and the QP relaxation method, more than 90 % of the gap can be closed for some QPCC problems.     

UTA-poly and UTA-splines: Additive value functions with polynomial marginals

Additive utility function models are widely used in multiple criteria decision analysis. In such models, a numerical value is associated to each alternative involved in the decision problem. It is computed by aggregating the scores of the alternative on the different criteria of the decision problem. The score of an alternative is determined by a marginal value function that evolves monotonically as a function of the performance of the alternative on this criterion. Determining the shape of the marginals is not easy for a decision maker. It is easier for him/her to make statements such as “alternative a is preferred to b”. In order to help the decision maker, UTA disaggregation procedures use linear programming to approximate the marginals by piecewise linear functions based only on such statements. In this paper, we propose to infer polynomials and splines instead of piecewise linear functions for the marginals. In this aim, we use semidefinite programming instead of linear programming. We illustrate this new elicitation method and present some experimental results.     

An Augmented Lagrangian Method for a Class of Inverse Quadratic Programming Problems

We consider an inverse quadratic programming (QP) problem in which the parameters in the objective function of a given QP problem are adjusted as little as possible so that a known feasible solution becomes the optimal one. We formulate this problem as a minimization problem with a positive semidefinite cone constraint and its dual is a linearly constrained semismoothly differentiable (SC¹) convex programming problem with fewer variables than the original one. We demonstrate the global convergence of the augmented Lagrangian method for the dual problem and prove that the convergence rate of primal iterates, generated by the augmented Lagrange method, is proportional to 1/r, and the rate of multiplier iterates is proportional to  $1/\sqrt{r}$ , where r is the penalty parameter in the augmented Lagrangian. As the objective function of the dual problem is a SC¹ function involving the projection operator onto the cone of symmetrically semi-definite matrices, the analysis requires extensive tools such as the singular value decomposition of matrices, an implicit function theorem for semismooth functions, and properties of the projection operator in the symmetric-matrix space. Furthermore, the semismooth Newton method with Armijo line search is applied to solve the subproblems in the augmented Lagrange approach, which is proven to have global convergence and local quadratic rate. Finally numerical results, implemented by the augmented Lagrangian method, are reported.