基于D.C.分解的一类箱型约束的非凸二次规划的新型分支定界算法

提出了一类求解带有箱约束的非凸二次规划的新型分支定界算法.首先,把原问题目标函数进行D.C.分解(分解为两个凸函数之差),利用次梯度方法,求出其线性下界逼近函数的一个最优值,也即原问题的一个下界.然后,利用全局椭球算法获得原问题的一个上界,并根据分支定界方法把原问题的求解转化为一系列子问题的求解.最后,理论上证明了算法的收敛性,数值算例表明算法是有效可行的.     

一种基于匈牙利算法的二次分配问题求解方法

二次分配问题(Quadratic assignment problem,QAP)属于NP-hard组合优化难题.二次分配问题的线性化及下界计算方法,是求解二次分配问题的重要途径.以Frieze-Yadegar线性化模型和Gilmore-Lawler下界为基础,详细论述了二次分配问题线性化模型的结构特征,并分析了Gilmore-Lawler下界值往往远离目标函数最优值的原因.在此基础上,提出一种基于匈牙利算法的二次分配问题对偶上升下界求解法.通过求解QAPLIB中的部分实例,说明了方法的有效和可行性.     

单纯形上二次型函数的最值搜索算法

由于单纯形域上的二次型函数往往是多峰函数,当函数形式较为复杂时难以求得全局最值.构造了两类适用于单纯形上二次型函数优化的算法,分别是单纯形上的Newton-Raphson算法与随机搜索算法.经过实例验证,这两种算法都是有效的.     

二项线性随机效应模型的拟蒙特卡罗估计

讨论了如何运用拟蒙特卡罗方法对二项线性随机效应模型进行参数估计.首先写出观测数据的边缘对数似然函数,然后用拟蒙特卡罗方法将函数中的积分写成求和的形式,接着利用Newton-Raphson算法计算参数的极大似然估计.以一组种子数据为例,说明该方法是简单可行的.     

不等式约束优化基于新型积极识别集的SQCQP算法

本文提出一个新的求解非线性不等式约束优化问题的罚函数型序列二次约束二次规划(SQCQP)算法.算法每次迭代只需求解一个凸二次约束二次规划(QCQP)子问题,且通过引入新型积极识别集技术,QCQP子问题的规模显著减小,从而降低计算成本.在不需要函数凸性等较弱假设下,算法具有全局收敛性.初步的数值试验表明算法是稳定有效的.     

非线性半定规划一个全局收敛的无罚无滤子SSDP算法

提出了一个求解非线性半定规划的无罚函数无滤子序列二次半定规划(SSDP)算法. 算法每次迭代只需求解一个二次半定规划子问题确定搜索方向; 非单调线搜索保证目标函数或约束违反度函数的充分下降, 从而产生新的迭代点. 在适当的假设条件下, 证明了算法的全局收敛性. 最后给出了初步的数值实验结果.     

缺失数据下含几何分布的对数线性模型的EM算法

本文研究缺失数据下对数线性模型参数的极大似然估计问题.通过Monte-Carlo EM算法去拟合所提出的模型.其中,在期望步中利用Metropolis-Hastings算法产生一个缺失数据的样本,在最大化步中利用Newton-Raphson迭代使似然函数最大化.最后,利用观测数据的Fisher信息得到参数极大似然估计的渐近方差和标准误差.     

Monte Carlo EM加速算法

EM算法是近年来常用的求后验众数的估计的一种数据增广算法, 但由于求出其E步中积分的显示表达式有时很困难, 甚至不可能, 限制了其应用的广泛性. 而Monte Carlo EM算法很好地解决了这个问题, 将EM算法中E步的积分用Monte Carlo模拟来有效实现, 使其适用性大大增强. 但无论是EM算法, 还是Monte Carlo EM算法, 其收敛速度都是线性的, 被缺损信息的倒数所控制, 当缺损数据的比例很高时, 收敛速度就非常缓慢. 而Newton-Raphson算法在后验众数的附近具有二次收敛速率. 本文提出Monte Carlo EM加速算法, 将Monte Carlo EM算法与Newton-Raphson算法结合, 既使得EM算法中的E步用Monte Carlo模拟得以实现, 又证明了该算法在后验众数附近具有二次收敛速度. 从而使其保留了Monte Carlo EM算法的优点, 并改进了Monte Carlo EM算法的收敛速度. 本文通过数值例子, 将Monte Carlo EM加速算法的结果与EM算法、Monte Carlo EM算法的结果进行比较, 进一步说明了Monte Carlo EM加速算法的优良性.     

基于MM算法的LAD回归的影响分析

基于Hunter and Lange(2000)提出的MM迭代算法,构造了一个代替L1目标函数的新的目标函数Qε(ββk);在此基础上研究了非线性LAD回归影响分析的若干问题.基于新的目标函数和MM迭代算法,证明了LAD回归模型中数据删除模型和均值漂移模型参数估计的等价性定理,并提出了一种新的影响度量.最后,几个数据实例说明了方法的有效性.     

求解稀疏逻辑回归问题的嵌套BB算法的分裂增广拉格朗日算法

逻辑回归是经典的分类方法,广泛应用于数据挖掘、机器学习和计算机视觉.现研究带有程。模约束的逻辑回归问题.这类问题广泛用于分类问题中的特征提取,且一般是NP-难的.为了求解这类问题,提出了嵌套BB(Barzilai and Borwein)算法的分裂增广拉格朗日算法(SALM-BB).该算法在迭代中交替地求解一个无约束凸优化问题和一个带程。模约束的二次优化问题.然后借助BB算法求解无约束凸优化问题.通过简单的等价变形直接得到带程。模约束二次优化问题的精确解,并且给出了算法的收敛性定理.最后通过数值实验来测试SALM-BB算法对稀疏逻辑回归问题的计算精确性.数据来源包括真实的UCI数据和模拟数据.数值实验表明,相对于一阶算法SLEP,SALM-BB能够得到更低的平均逻辑损失和错分率.     

Lower Bound Improvement and Forcing Rule for Quadratic Binary Programming

In this paper several equivalent formulations for the quadratic binary programming problem are presented. Based on these formulations we describe four different kinds of strategies for estimating lower bounds of the objective function, which can be integrated into a branch and bound algorithm for solving the quadratic binary programming problem. We also give a theoretical explanation for forcing rules used to branch the variables efficiently, and explore several properties related to obtained subproblems. From the viewpoint of the number of subproblems solved, new strategies for estimating lower bounds are better than those used before. A variant of a depth-first branch and bound algorithm is described and its numerical performance is presented.     

Monotonicity of quadratic-approximation algorithms

It is desirable that a numerical maximization algorithm monotonically increase its objective function for the sake of its stability of convergence. It is here shown how one can adjust the Newton-Raphson procedure to attain monotonicity by the use of simple bounds on the curvature of the objective function. The fundamental tool in the analysis is the geometric insight one gains by interpreting quadratic-approximation algorithms as a form of area approximation. The statistical examples discussed include maximum likelihood estimation in mixture models, logistic regression and Cox's proportional hazards regression.The second author's research was partially supported by the National Science Foundation under Grant DMS-8402735.     

求解带二次约束的非凸二次规划的一种分支定界算法

本文提出了一种求解带二次约束和线性约束的二次规划的分支定界算法.在算法中,我们运用Lipschitz条件来确定目标函数和约束函数的在每个n矩形上的上下界,对于n矩形的分割,我们采用选择n矩形最长边的二分法,同时我们采用了一些矩形删除技术,在不大幅增加计算量的前提下,起到了加速算法收敛的效果.从理论上我们证明了算法的收敛性,同时数值实验表明该算法是有效的.     

An improved lower bound and approximation algorithm for binary constrained quadratic programming problem

This paper presents an improved lower bound and an approximation algorithm based on spectral decomposition for the binary constrained quadratic programming problem. To decompose spectrally the quadratic matrix in the objective function, we construct a low rank problem that provides a lower bound. Then an approximation algorithm for the binary quadratic programming problem together with a worst case performance analysis for the algorithm is provided.     

A scalable FETI-DP algorithm with non-penetration mortar conditions on contact interface

By combining FETI algorithms of dual-primal type with recent results for bound constrained quadratic programming problems, we develop an optimal algorithm for the numerical solution of coercive variational inequalities. The model problem is discretized using non-penetration conditions of mortar type across the potential contact interface, and a FETI-DP algorithm is formulated. The resulting quadratic programming problem with bound constraints is solved by a scalable algorithm with a known rate of convergence given in terms of the spectral condition number of the quadratic problem. Numerical experiments for non-matching meshes across the contact interface confirm the theoretical scalability of the algorithm.     

A working set SQCQP algorithm with simple nonmonotone penalty parameters

In this paper, we present a new sequential quadratically constrained quadratic programming (SQCQP) algorithm, in which a simple updating strategy of the penalty parameter is adopted. This strategy generates nonmonotone penalty parameters at early iterations and only uses the multiplier corresponding to the bound constraint of the quadratically constrained quadratic programming (QCQP) subproblem instead of the multipliers of the quadratic constraints, which will bring some numerical advantages. Furthermore, by using the working set technique, we remove the constraints of the QCQP subproblem that are locally irrelevant, and thus the computational cost could be reduced. Without assuming the convexity of the objective function or the constraints, the algorithm is proved to be globally, superlinearly and quadratically convergent. Preliminary numerical results show that the proposed algorithm is very promising when compared with the tested SQP algorithms.     

A working set SQCQP algorithm with simple nonmonotone penalty parameters

In this paper, we present a new sequential quadratically constrained quadratic programming (SQCQP) algorithm, in which a simple updating strategy of the penalty parameter is adopted. This strategy generates nonmonotone penalty parameters at early iterations and only uses the multiplier corresponding to the bound constraint of the quadratically constrained quadratic programming (QCQP) subproblem instead of the multipliers of the quadratic constraints, which will bring some numerical advantages. Furthermore, by using the working set technique, we remove the constraints of the QCQP subproblem that are locally irrelevant, and thus the computational cost could be reduced. Without assuming the convexity of the objective function or the constraints, the algorithm is proved to be globally, superlinearly and quadratically convergent. Preliminary numerical results show that the proposed algorithm is very promising when compared with the tested SQP algorithms.     

A Smoothing Projected Levenberg-Marquardt Type Algorithm for Solving Constrained Equations

In this article, we consider the problem of finding a solution of a nonsmooth constrained (and not necessarily square) system of equations. We first reformulate the original problem as an equivalent system of equations with nonnegative constraints, and then present a smoothing projected Levenberg-Marquardt type algorithm to solve the reformulated system, which solves a strictly convex quadratic program at each iteration. We show that this algorithm not only converges globally, but also converges locally superlinearly under an error bound assumption that is much weaker than the standard nonsingularity condition. Some numerical results for the presented algorithm indicate that the algorithm works quite well in practice.