高维约束矩阵回归问题

高维约束矩阵回归是指高维情况下带非凸约束的多响应多预测统计回归问题,其数学模型是一个NP-难的矩阵优化,它在机器学习与人工智能、医学影像疾病诊疗、基因表达分析、脑神经网络、风险管理等领域有广泛应用.从高维约束矩阵回归的优化理论和算法两方面总结和评述这些新成果,同时,列出了相应的重要文献.     

Iterative reweighted methods for $$\ell _1-\ell _p$$ minimization

In this paper, we focus on the \(\ell _1-\ell _p\) minimization problem with \(0<p<1\), which is challenging due to the \(\ell _p\) norm being non-Lipschizian. In theory, we derive computable lower bounds for nonzero entries of the generalized first-order stationary points of \(\ell _1-\ell _p\) minimization, and hence of its local minimizers. In algorithms, based on three locally Lipschitz continuous \(\epsilon \)-approximation to \(\ell _p\) norm, we design several iterative reweighted \(\ell _1\) and \(\ell _2\) methods to solve those approximation problems. Furthermore, we show that any accumulation point of the sequence generated by these methods is a generalized first-order stationary point of \(\ell _1-\ell _p\) minimization. This result, in particular, applies to the iterative reweighted \(\ell _1\) methods based on the new Lipschitz continuous \(\epsilon \)-approximation introduced by Lu (Math Program 147(1–2):277–307, 2014), provided that the approximation parameter \(\epsilon \) is below a threshold value. Numerical results are also reported to demonstrate the efficiency of the proposed methods.     

Correlation stress testing for value-at-risk: an unconstrained convex optimization approach

Correlation stress testing is employed in several financial models for determining the value-at-risk (VaR) of a financial institution’s portfolio. The possible lack of mathematical consistence in the target correlation matrix, which must be positive semidefinite, often causes breakdown of these models. The target matrix is obtained by fixing some of the correlations (often contained in blocks of submatrices) in the current correlation matrix while stressing the remaining to a certain level to reflect various stressing scenarios. The combination of fixing and stressing effects often leads to mathematical inconsistence of the target matrix. It is then naturally to find the nearest correlation matrix to the target matrix with the fixed correlations unaltered. However, the number of fixed correlations could be potentially very large, posing a computational challenge to existing methods. In this paper, we propose an unconstrained convex optimization approach by solving one or a sequence of continuously differentiable (but not twice continuously differentiable) convex optimization problems, depending on different stress patterns. This research fully takes advantage of the recently developed theory of strongly semismooth matrix valued functions, which makes fast convergent numerical methods applicable to the underlying unconstrained optimization problem. Promising numerical results on practical data (RiskMetrics database) and randomly generated problems of larger sizes are reported.     

A feasible semismooth asymptotically Newton method for mixed complementarity problems

 Semismooth Newton methods constitute a major research area for solving mixed complementarity problems (MCPs). Early research on semismooth Newton methods is mainly on infeasible methods. However, some MCPs are not well defined outside the feasible region or the equivalent unconstrained reformulations of other MCPs contain local minimizers outside the feasible region. As both these problems could make the corresponding infeasible methods fail, more recent attention is on feasible methods. In this paper we propose a new feasible semismooth method for MCPs, in which the search direction asymptotically converges to the Newton direction. The new method overcomes the possible non-convergence of the projected semismooth Newton method, which is widely used in various numerical implementations, by minimizing a one-dimensional quadratic convex problem prior to doing (curved) line searches. As with other semismooth Newton methods, the proposed method only solves one linear system of equations at each iteration. The sparsity of the Jacobian of the reformulated system can be exploited, often reducing the size of the system that must be solved. The reason for this is that the projection onto the feasible set increases the likelihood of components of iterates being active. The global and superlinear/quadratic convergence of the proposed method is proved under mild conditions. Numerical results are reported on all problems from the MCPLIB collection [8]. Received: December 1999 / Accepted: March 2002 Published online: September 5, 2002 RID="★" ID="★" This work was supported in part by the Australian Research Council. Key Words. mixed complementarity problems – semismooth equations – projected Newton method – convergence AMS subject classifications. 90C33, 90C30, 65H10     

Stability Analysis of Gradient-Based Neural Networks for Optimization Problems

The paper introduces a new approach to analyze the stability of neural network models without using any Lyapunov function. With the new approach, we investigate the stability properties of the general gradient-based neural network model for optimization problems. Our discussion includes both isolated equilibrium points and connected equilibrium sets which could be unbounded. For a general optimization problem, if the objective function is bounded below and its gradient is Lipschitz continuous, we prove that (a) any trajectory of the gradient-based neural network converges to an equilibrium point, and (b) the Lyapunov stability is equivalent to the asymptotical stability in the gradient-based neural networks. For a convex optimization problem, under the same assumptions, we show that any trajectory of gradient-based neural networks will converge to an asymptotically stable equilibrium point of the neural networks. For a general nonlinear objective function, we propose a refined gradient-based neural network, whose trajectory with any arbitrary initial point will converge to an equilibrium point, which satisfies the second order necessary optimality conditions for optimization problems. Promising simulation results of a refined gradient-based neural network on some problems are also reported.     

A Lagrange–Newton algorithm for sparse nonlinear programming

Mathematical Programming - The sparse nonlinear programming (SNP) problem has wide applications in signal and image processing, machine learning and finance, etc. However, the computational...     

Lagrangian duality and saddle points for sparse linear programming

The sparse linear programming(SLP) is a linear programming problem equipped with a sparsity constraint, which is nonconvex, discontinuous and generally NP-hard due to the combinatorial property involved.In this paper, by rewriting the sparsity constraint into a disjunctive form, we present an explicit formula of the Lagrangian dual problem for the SLP, in terms of an unconstrained piecewise-linear convex programming problem which admits a strong duality under bi-dual sparsity consistency. Furthermore, we show a saddle point theorem based on the strong duality and analyze two classes of stationary points for the saddle point problem. At last,we extend these results to SLP with the lower bound zero replaced by a certain negative constant.     

Regularity and well-posedness of a dual program for convex best <Emphasis Type="Italic">C</Emphasis><Superscript>1</Superscript>-spline interpolation

An efficient approach to computing the convex best C ¹-spline interpolant to a given set of data is to solve an associated dual program by standard numerical methods (e.g., Newton’s method). We study regularity and well-posedness of the dual program: two important issues that have been not yet well-addressed in the literature. Our regularity results characterize the case when the generalized Hessian of the objective function is positive definite. We also give sufficient conditions for the coerciveness of the objective function. These results together specify conditions when the dual program is well-posed and hence justify why Newton’s method is likely to be successful in practice. Examples are given to illustrate the obtained results. The work was supported by EPSRC grant EP/D502535/1 for the first author and by a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China (Project No. PolyU 5141/01E) for the second author.     

Cartesian P-property and Its Applications to the Semidefinite Linear Complementarity Problem

We introduce a Cartesian P-property for linear transformations between the space of symmetric matrices and present its applications to the semidefinite linear complementarity problem (SDLCP). With this Cartesian P-property, we show that the SDLCP has GUS-property (i.e., globally unique solvability), and the solution map of the SDLCP is locally Lipschitzian with respect to input data. Our Cartesian P-property strengthens the corresponding P-properties of Gowda and Song [15] and allows us to extend several numerical approaches for monotone SDLCPs to solve more general SDLCPs, namely SDLCPs with the Cartesian P-property. In particular, we address important theoretical issues encountered in those numerical approaches, such as issues related to the stationary points in the merit function approach, and the existence of Newton directions and boundedness of iterates in the non-interior continuation method of Chen and Tseng [6]. This work is supported by the annual grant A2004/23 of University of Southampton.     

Neurodynamical Optimization

Dynamical (or ode) system and neural network approaches for optimization have been co-existed for two decades. The main feature of the two approaches is that a continuous path starting from the initial point can be generated and eventually the path will converge to the solution. This feature is quite different from conventional optimization methods where a sequence of points, or a discrete path, is generated. Even dynamical system and neural network approaches share many common features and structures, yet a complete comparison for the two approaches has not been available. In this paper, based on a detailed study on the two approaches, a new approach, termed neurodynamical approach, is introduced. The new neurodynamical approach combines the attractive features in both dynamical (or ode) system and neural network approaches. In addition, the new approach suggests a systematic procedure and framework on how to construct a neurodynamical system for both unconstrained and constrained problems. In analyzing the stability issues of the underlying dynamical (or ode) system, the neurodynamical approach adopts a new strategy, which avoids the Lyapunov function. Under the framework of this neurodynamical approach, strong theoretical results as well as promising numerical results are obtained.