A Hybrid Method for Nonlinear Least Squares Problems

A negative curvature method is applied to nonlinear least squares problems with indefinite Hessian approximation matrices. With the special structure of the method, a new switch is proposed to form a hybrid method. Numerical experiments show that this method is feasible and effective for zero-residual, small-residual and large-residual problems.     

Quasi-Newton Methods for Nonlinear Least Squares Focusing on Curvatures

A new quasi-Newton method for nonlinear least squares problems is proposed. Two advantages of the method are accomplished by utilizing special geometrical properties in the problem class. First, fast convergence is established for well-conditioned problems by interpolating both the current and the previous step in each iteration. Second, high accuracy is achieved for certain difficult problems, such as ill-conditioned problems and problems with large curvatures in the tangent space. Numerical results for artificial problems and standard test problems are presented and discussed.     

用神经网络解非线性最小二乘问题

提出了一种求解非线性最小二乘问题的神经网络方法 ,并证明了该神经网络方法的稳定性和收敛性     

On Least Squares Estimation for Stable Nonlinear AR Processes

Following a Markov chain approach, this paper establishes asymptotic properties of the least squares estimator in nonlinear autoregressive (NAR) models. Based on conditions ensuring the stability of the model and allowing the use of a strong law of large number for a wide class of functions, our approach improves some known results on strong consistency and asymptotic normality of the estimator. The exact convergence rate is established by a law of the iterated logarithm. Based on this law and a generalized Akaike's information criterion, we build a strongly consistent procedure for selection of NAR models. Detailed results are given for familiar nonlinear AR models like exponential AR models, threshold models or multilayer feedforward perceptions.     

Nonlinear Least Squares Estimation of Log-ACD Models

This paper studies a nonlinear least squares estimation method for the logarithmic autoregressive conditional duration (Log-ACD) model. We establish the strong consistency and asymptotic normality for our estimator under weak moment conditions suitable for applications involving heavy-tailed distributions. We also discuss inference for the Log-ACD model and Log-ACD models with exogenous variables. Our results can be easily translated to study Log-GARCH models. Both simulation study and real data analysis are conducted to show the usefulness of our results.     

Methods with Successive and Parallel Approximations of Inverse Operator for the Nonlinear Least Squares Problem

For solving the nonlinear least-square problem, we propose iterative methods that use successive and parallel approximations of the inverse operator instead of solving a linear system of equations. The convergence order as well as the convergence radius of the proposed methods are studied. Finally, we carry out numerical experiments on a set of test problems. (© 2015 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim)     

Superlinearly Convergent Exact Penalty Methods with Projected Structured Secant Updates for Constrained Nonlinear Least Squares

We present a superlinearly convergent exact penalty method for solving constrained nonlinear least squares problems, in which the projected exact penalty Hessian is approximated by using a structured secant updating scheme. We give general conditions for the two-step superlinear convergence of the algorithm and prove that the projected structured Broyden–Fletcher–Goldfarb–Shanno (BFGS), Powell-symmetric-Broyden (PSB), and Davidon–Fletcher–Powell (DFP) update formulas satisfy these conditions. Then we extend the results to the projected structured convex Broyden family update formulas. Extensive testing results obtained by an implementation of our algorithms, as compared to the results obtained by several other competent algorithms, demonstrate the efficiency and robustness of the proposed approach.     

求解加权线性最小二乘问题的预处理迭代方法

给出了求解一类加权线性最小二乘问题的预处理迭代方法,也就是预处理的广义加速超松弛方法（GAOR）,得到了一些收敛和比较结果．比较结果表明当原来的迭代方法收敛时,预处理迭代方法会比原来的方法具有更好的收敛率．而且,通过数值算例也验证了新预处理迭代方法的有效性．     

非线性最小二乘问题收敛性的证明

主要讨论了无约束最优化中非线性最小二乘问题的收敛性.侧重于收敛的速率和整体、局部分析.改变了Gauss—Newton方法收敛性定理的条件,分两种情况证明了:(1)目标函数的海赛矩阵正定(函数严格凸)时为强整体二阶收敛;(2)目标函数不保证严格凸性,但海赛矩阵的逆存在时为局部收敛,敛速仍为二阶,同时给出了J(X)~(-1)和Q(X)~(-1)之间存在、有界性的等价条件.     

Quadratic and Superlinear Convergence of the Huschens Method for Nonlinear Least Squares Problems

This paper is concerned with quadratic and superlinear convergence of structured quasi-Newton methods for solving nonlinear least squares problems. These methods make use of a special structure of the Hessian matrix of the objective function. Recently, Huschens proposed a new kind of structured quasi-Newton methods and dealt with the convex class of the structured Broyden family, and showed its quadratic and superlinear convergence properties for zero and nonzero residual problems, respectively. In this paper, we extend the results by Huschens to a wider class of the structured Broyden family. We prove local convergence properties of the method in a way different from the proof by Huschens.     

基于Mori-Zwanzig格式和偏最小二乘的非线性模型降阶

本征正交分解及Galerkin投影是解决复杂非线性系统模型降阶问题常用的方法.然而,该方法在构造降阶系统过程中只截取基函数的部分模态,这通常会使得降阶系统不准确.针对该问题,提出了对降阶系统误差进行快速校正的方法.首先应用Mori-Zwanzig格式对降阶系统的误差进行分析,理论上得到误差模型的形式和有效预测变量.再通过偏最小二乘方法构造预测变量和系统误差的多元回归模型,建立误差预测模型.将所构造的误差预测模型直接嵌入到原降阶系统,得到新的降阶系统在形式上等价于对原模型的右端采用Petrov-Galerkin投影.最后给出了新的降阶系统的误差估计.数值结果进一步说明了所提方法能有效地提高降阶系统的稳定性和准确性,且具有较高计算效率.     

总体最小二乘准则下非线性EV模型的参数拟合方法

在非线性回归模型参数拟合问题中,当数据中的每个变量都存在不可忽略的误差时,在普通的最小二乘准则下拟合出的参数不是最优的.按照总体最小二乘准则,以观测点到拟合曲线或拟合曲面垂直距离平方和为目标函数,然后用最优化方法搜索出使目标函数值取最小值的参数和数据点估计,从而给出求最优模型参数的算法,最后,通过计算机仿真和与文献比较,验证了提出方法的正确性.     

Least Squares Successive Relaxation

The author has been interested in preserving the sparsity oflinear systems having rectangular operators. By investigatingthe important least squares problem he has extended SuccessiveRelaxation methods to cope with such operators.     

A Structured Secant Method Based on a New Quasi-Newton Equation for Nonlinear Least Squares Problems

In this paper, a new quasi-Newton equation is applied to the structured secant methods for nonlinear least squares problems. We show that the new equation is better than the original quasi-Newton equation as it provides a more accurate approximation to the second order information. Furthermore, combining the new quasi-Newton equation with a product structure, a new algorithm is established. It is shown that the resulting algorithm is quadratically convergent for the zero-residual case and superlinearly convergent for the nonzero-residual case. In order to compare the new algorithm with some related methods, our preliminary numerical experiments are also reported.     

Condition Numbers for Structured Least Squares Problems

This paper studies the normwise perturbation theory for structured least squares problems. The structures under investigation are symmetric, persymmetric, skewsymmetric, Toeplitz and Hankel. We present the condition numbers for structured least squares. AMS subject classification (2000) 15A18, 65F20, 65F25, 65F50     

m0相依误差下非线性回归模型LS估计的大偏差

该文获得了m0相依误差下非线性回归模型LS估计的大偏差，推广了Sieders和Dzhaparidze在独立误差下的相应结果     

Meshfree First-order System Least Squares

We prove convergence for a meshfree first-order system least squares （FOSLS） partition of unity finite element method （PUFEM）. Essentially, by virtue of the partition of unity, local approximation gives rise to global approximation in H（div）∩H（curl）. The FOSLS formulation yields local a posteriori error estimates to guide the judicious allotment of new degrees of freedom to enrich the initial point set in a meshfree discretization. Preliminary numerical results are provided and remaining challenges are discussed.     

剔除相关性的最小二乘研究

本文提出了一种新的回归模型,剔除相关性的最小二乘,它有效的克服了变量间的相关性,兼顾到变量的筛选。并与最小二乘、向后删除变量法、偏最小二乘比较分析。发现剔除相关性的最小二乘能很好的处理自变量间多重相关性,对变量进行有效的筛选,克服了回归系数反常的现象。