Quadratic and Superlinear Convergence of the Huschens Method for Nonlinear Least Squares Problems

This paper is concerned with quadratic and superlinear convergence of structured quasi-Newton methods for solving nonlinear least squares problems. These methods make use of a special structure of the Hessian matrix of the objective function. Recently, Huschens proposed a new kind of structured quasi-Newton methods and dealt with the convex class of the structured Broyden family, and showed its quadratic and superlinear convergence properties for zero and nonzero residual problems, respectively. In this paper, we extend the results by Huschens to a wider class of the structured Broyden family. We prove local convergence properties of the method in a way different from the proof by Huschens.     

用神经网络解非线性最小二乘问题

提出了一种求解非线性最小二乘问题的神经网络方法 ,并证明了该神经网络方法的稳定性和收敛性     

A Structured Secant Method Based on a New Quasi-Newton Equation for Nonlinear Least Squares Problems

In this paper, a new quasi-Newton equation is applied to the structured secant methods for nonlinear least squares problems. We show that the new equation is better than the original quasi-Newton equation as it provides a more accurate approximation to the second order information. Furthermore, combining the new quasi-Newton equation with a product structure, a new algorithm is established. It is shown that the resulting algorithm is quadratically convergent for the zero-residual case and superlinearly convergent for the nonzero-residual case. In order to compare the new algorithm with some related methods, our preliminary numerical experiments are also reported.     

Global Convergence of the Broyden‘’s Class of Quasi—Newton Methods with Nonmonotone Linesearch

In this paper,the Broyden class of quasi-Newton methods for unconstrained optimization is inves-tigated.Non-monotone linesearch procedure is introduced,which is combined with the Broyden‘s class.Under the convexity assumption on objective function,the global convergence of the Broyden‘s class is proved.     

剔除相关性的最小二乘研究

本文提出了一种新的回归模型,剔除相关性的最小二乘,它有效的克服了变量间的相关性,兼顾到变量的筛选。并与最小二乘、向后删除变量法、偏最小二乘比较分析。发现剔除相关性的最小二乘能很好的处理自变量间多重相关性,对变量进行有效的筛选,克服了回归系数反常的现象。     

A Hybrid Method for Nonlinear Least Squares Problems

A negative curvature method is applied to nonlinear least squares problems with indefinite Hessian approximation matrices. With the special structure of the method, a new switch is proposed to form a hybrid method. Numerical experiments show that this method is feasible and effective for zero-residual, small-residual and large-residual problems.     

非线性互补问题光滑化拟牛顿算法的收敛性分析

给出求解p_0函数非线性互补问题光滑化拟牛顿算法,在p_0函数非线性互补问题有非空有界解集且F'是Lipschitz连续的条件下,证明了算法的全局收敛性.全局收敛性的主要特征是不需要提前假设水平集是有界的.     

On Least Squares Estimation for Stable Nonlinear AR Processes

Following a Markov chain approach, this paper establishes asymptotic properties of the least squares estimator in nonlinear autoregressive (NAR) models. Based on conditions ensuring the stability of the model and allowing the use of a strong law of large number for a wide class of functions, our approach improves some known results on strong consistency and asymptotic normality of the estimator. The exact convergence rate is established by a law of the iterated logarithm. Based on this law and a generalized Akaike's information criterion, we build a strongly consistent procedure for selection of NAR models. Detailed results are given for familiar nonlinear AR models like exponential AR models, threshold models or multilayer feedforward perceptions.     

求解加权线性最小二乘问题的预处理迭代方法

给出了求解一类加权线性最小二乘问题的预处理迭代方法,也就是预处理的广义加速超松弛方法（GAOR）,得到了一些收敛和比较结果．比较结果表明当原来的迭代方法收敛时,预处理迭代方法会比原来的方法具有更好的收敛率．而且,通过数值算例也验证了新预处理迭代方法的有效性．     

Hybrid Methods for Nonlinear Least Squares

The conjecture is considered that the hybrid GN-BFGS methodof Al-Baali and Fletcher [1] is superlinearly convergent. Acounterexample is constructed for which the method is only linearlyconvergent whereas the BFGS method alone would be superlinearlyconvergent, thus disproving the conjecture. Two new hybrids methods are suggested, and it is shown thatthese are superlinearly convergent under mild conditions. Awide range of numerical experience is reported and is seen tofavour the new methods. Some interesting theoretical propertiesof a variational Hessian error measure are given.     

Meshfree First-order System Least Squares

We prove convergence for a meshfree first-order system least squares （FOSLS） partition of unity finite element method （PUFEM）. Essentially, by virtue of the partition of unity, local approximation gives rise to global approximation in H（div）∩H（curl）. The FOSLS formulation yields local a posteriori error estimates to guide the judicious allotment of new degrees of freedom to enrich the initial point set in a meshfree discretization. Preliminary numerical results are provided and remaining challenges are discussed.     

Backward Error Bounds for Constrained Least Squares Problems

We derive an upper bound on the normwise backward error of an approximate solution to the equality constrained least squares problem min _Bx=d b – Ax₂. Instead of minimizing over the four perturbations to A, b, B and d, we fix those to B and d and minimize over the remaining two; we obtain an explicit solution of this simplified minimization problem. Our experiments show that backward error bounds of practical use are obtained when B and d are chosen as the optimal normwise relative backward perturbations to the constraint system, and we find that when the bounds are weak they can be improved by direct search optimization. We also derive upper and lower backward error bounds for the problem of least squares minimization over a sphere: .     

基于Pena距离的加权最小二乘估计的影响分析

利用Pena距离对加权线性最小二乘估计的影响问题进行讨论,得到加权最小二乘估计的Pena距离的表达式,对其性质进行讨论,从而得到高杠异常点的判别方法.文中对Pena距离与Cook距离的性能进行了对比,得到在一定条件下Pena距离优于Cook距离的结论.并通过数值实验对此方法的有效性进行验证.     

偏最小二乘回归的应用效果分析

本文介绍了偏最小二乘回归 (PLS)的建模方法 ,比较了PLS与普通最小二乘回归 (OLS)及主成分回归的应用效果 ,并总结了PLS回归的基本特点 .     

一类推广的拟牛顿多进程异步并行方法

本文通过近似雅可比矩阵Bk代替雅可比矩阵F′(xk),运用多进程异步并行方法求解非线性方程组。该方法在保持解的精度的情况下,缩短了运行时间和迭代步数。文中给出了算法收敛性的证明及八个非线性方程组的数值测试结果,表明该算法是可行的和快速的。     

拟线性回归预测模型的稳定最小二乘解

可线性化回归预测模型通过换元进行线性回归,换元前后的因变量具有异方差性,致使拟线性回归参数的精度较低.运用GL算法给出了此类模型的稳定最小二乘解,提高了参数的估计精度,最后给出了一个应用实例.     

求矩阵方程组的反对称最小二乘解的递推算法

讨论了矩阵方程组A_1XB_1=D_1,A_2XB_2=D_2反对称最小二乘解的递推算法,该算法不仅能够用于计算反对称最小二乘解,而且在选取特殊的初始矩阵时,算法能够求出矩阵方程组的极小范数反对称最小二乘解,以及对给定的矩阵进行最佳逼近的反对称解.     

Implicit Scaling of Linear Least Squares Problems

We consider the solution of weighted linear least squares problems by Householder transformations with implicit scaling, that is, with the weights stored separately. By holding inverse weights, the constrained case can be accommodated. The error analysis of the weighted and unconstrained case is readily extended and we show that iterative refinement may be applied.     

Combined and Least Squares Empirical Likelihood

In conventional empirical likelihood, there is exactly one structural constraint for every parameter. In some circumstances, additional constraints are imposed to reflect additional and sought-after features of statistical analysis. Such an augmented scheme uses the implicit power of empirical likelihood to produce very natural adaptive statistical methods, free of arbitrary tuning parameter choices, and does have good asymptotic properties. The price to be paid for such good properties is in extra computational difficulty. To overcome the computational difficulty, we propose a least-squares version of the empirical likelihood. The method is illustrated by application to the case of combined empirical likelihood for the mean and the median in one sample location inference.     

关于广义压缩最小二乘估计的注记

本文研究了广义压缩最小二乘估计（ＧＳＬＳＥ）的一些性质，给出了它的均方误差（ＭＳＥ）的一个无偏估计量（ＵＥ），采用极小该ＵＥ的方法确定了ＧＳＬＳＥ的参数选取公式，并把这个统一化的方法应用于广义岭估计，岭估计、Ｍａｓｓｙ主成分估计、Ｓｔｅｉｎ型压缩估计以及根方有偏估计等，从而得到了它们的一种选取参数的方法，最后，结合Ｈａｌｄ实例进行比较分析，结果表明，本文的方法是实用的，有效的。