Scattered data fitting of Hermite type by a weighted meshless method

In many practical problems, it is often desirable to interpolate not only the function values but also the values of derivatives up to certain order, as in the Hermite interpolation. The Hermite interpolation method by radial basis functions is used widely for solving scattered Hermite data approximation problems. However, sometimes it makes more sense to approximate the solution by a least squares fit. This is particularly true when the data are contaminated with noise. In this paper, a weighted meshless method is presented to solve least squares problems with noise. The weighted meshless method by Gaussian radial basis functions is proposed to fit scattered Hermite data with noise in certain local regions of the problem’s domain. Existence and uniqueness of the solution is proved. This approach has one parameter which can adjust the accuracy according to the size of the noise. Another advantage of the weighted meshless method is that it can be used for problems in high dimensions with nonregular domains. The numerical experiments show that our weighted meshless method has better performance than the traditional least squares method in the case of noisy Hermite data.     

A weighted least squares method for scattered data fitting

In this paper, we present a weighted least squares method to fit scattered data with noise. Existence and uniqueness of a solution are proved and an error bound is derived. The numerical experiments illustrate that our weighted least squares method has better performance than the traditional least squares method in case of noisy data.     

A meshless local Galerkin method for the numerical solution of Hammerstein integral equations based on the moving least squares technique

In this paper, a computational scheme is proposed to estimate the solution of one- and two-dimensional Fredholm-Hammerstein integral equations of the second kind. The method approximates the solution using the discrete Galerkin method based on the moving least squares (MLS) approach as a locally weighted least squares polynomial fitting. The discrete Galerkin technique for integral equations results from the numerical integration of all integrals in the system corresponding to the Galerkin method. Since the proposed method is constructed on a set of scattered points, it does not require any background meshes and so we can call it as the meshless local discrete Galerkin method. The implication of the scheme for solving two-dimensional integral equations is independent of the geometry of the domain. The new method is simple, efficient and more flexible for most classes of nonlinear integral equations. The error analysis of the method is provided. The convergence accuracy of the new technique is tested over several Hammerstein integral equations and obtained results confirm the theoretical error estimates.     

Spectral collocation and radial basis function methods for one-dimensional interface problems

Differential equations with singular sources or discontinuous coefficients yield non-smooth or even discontinuous solutions. This problem is known as the interface problem. High-order numerical solutions suffer from the Gibbs phenomenon in that the accuracy deteriorates if the discontinuity is not properly treated. In this work, we use the spectral and radial basis function methods and present a least squares collocation method to solve the interface problem for one-dimensional elliptic equations. The domain is decomposed into multiple sub-domains; in each sub-domain, the collocation solution is sought. The solution should satisfy more conditions than the given conditions associated with the differential equations, which makes the problem over-determined. To solve the over-determined system, the least squares method is adopted. For the spectral method, the weighted norm method with different scaling factors and the mixed formulation are used. For the radial basis function method, the weighted shape parameter method is presented. Numerical results show that the least squares collocation method provides an accurate solution with high efficacy and that better accuracy is obtained with the spectral method.     

Semiparametric Estimation and Selection for Nonstationary Spatial Covariance Functions

We propose a method for estimating nonstationary spatial covariance functions by representing a spatial process as a linear combination of some local basis functions with uncorrelated random coefficients and some stationary processes, based on spatial data sampled in space with repeated measurements. By incorporating a large collection of local basis functions with various scales at various locations and stationary processes with various degrees of smoothness, the model is flexible enough to represent a wide variety of nonstationary spatial features. The covariance estimation and model selection are formulated as a regression problem with the sample covariances as the response and the covariances corresponding to the local basis functions and the stationary processes as the predictors. A constrained least squares approach is applied to select appropriate basis functions and stationary processes as well as estimate parameters simultaneously. In addition, a constrained generalized least squares approach is proposed to further account for the dependencies among the response variables. A simulation experiment shows that our method performs well in both covariance function estimation and spatial prediction. The methodology is applied to a U.S. precipitation dataset for illustration. Supplemental materials relating to the application are available online.     

Bivariate Free Knot Splines

We consider the least squares approximation of gridded 2D data by tensor product splines with free knots. The smoothing functional to be minimized—a generalization of the univariate Schoenberg functional—is chosen in such a way that the solution of the bivariate problem separates into the solution of a sequence of univariate problems in case of fixed knots. The resulting optimization problem is a constrained separable least squares problem with tensor product structure. Based on some ideas developed by the authors for the univariate case, an efficient method for solving the specially structured 2D problem is proposed, analyzed and tested on hand of some examples from the literature.     

An efficient meshfree point collocation moving least squares method to solve the interface problems with nonhomogeneous jump conditions

We are going to study a simple and effective method for the numerical solution of the closed interface boundary value problem with both discontinuities in the solution and its derivatives. It uses a strong‐form meshfree method based on the moving least squares (MLS) approximation. In this method, for the solution of elliptic equation, the second‐order derivatives of the shape functions are needed in constructing the global stiffness matrix. It is well‐known that the calculation of full derivatives of the MLS approximation, especially in high dimensions, is quite costly. In the current work, we apply the diffuse derivatives using an efficient technique. In this technique, we calculate the higher‐order derivatives using the approximation of lower‐order derivatives, instead of calculating directly derivatives. This technique can improve the accuracy of meshfree point collocation method for interface problems with nonhomogeneous jump conditions and can efficiently estimate diffuse derivatives of second‐ and higher‐orders using only linear basis functions. To introduce the appropriate discontinuous shape functions in the vicinity of interface, we choose the visibility criterion method that modifies the support of weight function in MLS approximation and leads to an efficient computational procedure for the solution of closed interface problems. The proposed method is applied for elliptic and biharmonic interface problems. For the biharmonic equation, we use a mixed scheme, which replaces this equation by a coupled elliptic system. Also the application of the present method to elasticity equation with discontinuities in the coefficients across a closed interface has been provided. Representative numerical examples demonstrate the accuracy and robustness of the proposed methodology for the closed interface problems. © 2014 Wiley Periodicals, Inc. Numer Methods Partial Differential Eq 31: 1031–1053, 2015     

基于偏最小二乘变换与Kriging模型的全局优化方法

针对传统Kriging模型在多变量(高维)输入全局优化中因超参数过多而引发收敛速度慢,精度低,建模效率不高问题,提出了基于偏最小二乘变换技术和Kriging模型的有效全局优化方法.首先,构造偏最小二乘高斯核函数;其次,借助差分进化算法寻找满足期望改进准则最大化条件的新样本点;然后,将不同核函数和期望改进准则组合,构建四种有效全局优化算法并进行比较;最后,数值算例结果表明,基于偏最小二乘变换的Kriging全局优化方法在解决高维全局优化问题方面相比于标准的全局优化算法在收敛精度及收敛速度方面更具优势.     

An iterative penalty method for the least squares solution of boundary value problems

This article is concerned with iterative techniques for linear systems of equations arising from a least squares formulation of boundary value problems. In its classical form, the solution of the least squares method is obtained by solving the traditional normal equation. However, for nonsmooth boundary conditions or in the case of refinement at a selected set of interior points, the matrix associated with the normal equation tends to be ill-conditioned. In this case, the least squares method may be formulated as a Powell multiplier method and the equations solved iteratively. Therein we use and compare two different iterative algorithms. The first algorithm is the preconditioned conjugate gradient method applied to the normal equation, while the second is a new algorithm based on the Powell method and formulated on the stabilized dual problem. The two algorithms are first compared on a one-dimensional problem with poorly conditioned matrices. Results show that, for such problems, the new algorithm gives more accurate results. The new algorithm is then applied to a two-dimensional steady state diffusion problem and a boundary layer problem. A comparison between the least squares method of Bramble and Schatz and the new algorithm demonstrates the ability of the new method to give highly accurate results on the boundary, or at a set of given interior collocation points without the deterioration of the condition number of the matrix. Conditions for convergence of the proposed algorithm are discussed. © 1997 John Wiley & Sons, Inc.     

On choosing “optimal” shape parameters for RBF approximation

Many radial basis function (RBF) methods contain a free shape parameter that plays an important role for the accuracy of the method. In most papers the authors end up choosing this shape parameter by trial and error or some other ad hoc means. The method of cross validation has long been used in the statistics literature, and the special case of leave-one-out cross validation forms the basis of the algorithm for choosing an optimal value of the shape parameter proposed by Rippa in the setting of scattered data interpolation with RBFs. We discuss extensions of this approach that can be applied in the setting of iterated approximate moving least squares approximation of function value data and for RBF pseudo-spectral methods for the solution of partial differential equations. The former method can be viewed as an efficient alternative to ridge regression or smoothing spline approximation, while the latter forms an extension of the classical polynomial pseudo-spectral approach. Numerical experiments illustrating the use of our algorithms are included.     

基于RLS算法的多项式预测模型及其应用研究

为了提高经济领域统计数据的预测精度,代数多项式预测模型的建模方法应运而生.该方法使用代数多项式模型拟合给定的经济统计数据,并使用递推最小二乘法(RLS)对多项式拟合模型的加权系数进行递推计算以获得最优模型参数,然后通过获得的最优多项式模型计算未来预测数据.文章以实际统计的经济数据为例进行了仿真计算,研究结果表明,该方法不仅能实现统计数据的高精度拟合,而且具有很好的预测能力,在经济领域具有广阔的应用前景.     

最小二乘法相关的一类特殊矩阵的代数运算

§ 1　ＩｎｔｒｏｄｕｃｔｉｏｎａｎｄｓｉｍｐｌｅｅｘａｍｐｌｅｓＴｈｅｍｅｔｈｏｄｏｆｌｅａｓｔｓｑｕａｒｅｓｉｓｗｉｄｅｌｙｕｓｅｄｎｏｗａｄａｙｓｉｎｖａｒｉｏｕｓｆｉｅｌｄｓｏｆｓｃｉｅｎｃｅａｎｄｔｅｃｈ ｎｏｌｏｇｙ[1] .Ｆｒｏｍｔｈｅｐｏｉｎｔｖｉｅｗｏｆａｌｇｅｂｒａ ,ｔｈｉｓｍｅｔｈｏｄｃａｎｂｅｓｔａｔｅｄａｓｆｏｌｌｏｗｓ .ＳｕｐｐｏｓｅｔｈａｔＡｉｓａｎｍ×ｎｍａｔｒｉｘｏｖｅｒａｎｕｍｂｅｒｆｉｅｌｄ ,Ｘ ,ｂａｒｅｃｏｌｕｍｎｖｅｃｔｏｒｓｏｆｄｉｍｅｎｓｉｏｎｓｎ ,ｍｒｅ…     

A Hybrid Method for Nonlinear Least Squares Problems

A negative curvature method is applied to nonlinear least squares problems with indefinite Hessian approximation matrices. With the special structure of the method, a new switch is proposed to form a hybrid method. Numerical experiments show that this method is feasible and effective for zero-residual, small-residual and large-residual problems.     

A semiparametric method for estimating nonlinear autoregressive model with dependent errors

The first-order nonlinear autoregressive model is considered and a semiparametric method is proposed to estimate regression function. In the presented model, dependent errors are defined as first-order autoregressive AR(1). The conditional least squares method is used for parametric estimation and the nonparametric kernel approach is applied to estimate regression adjustment. In this case, some asymptotic behaviors and simulated results for the semiparametric method are presented. Furthermore, the method is applied for the financial data in Iran’s Tejarat-Bank.     

EXTENDED OBLIQUE PROJECTION METHOD FOR GENERALIZED LEAST SQUARES PROBLEM

We extend the oblique projection method given by Y.Saad to solve the generalized least squares problem. The corresponding oblique projection operator is presented and the convergence theorems are proved. Some necessary and sufficient conditions for computing the solution or the minimum N-norm solution of the min || A x- b ||M2 have been proposed as well.     

Moving least squares based sensitivity analysis for models with dependent variables

For models with dependent input variables, sensitivity analysis is often a troublesome work and only a few methods are available. Mara and Tarantola in their paper (“Variance-based sensitivity indices for models with dependent inputs”) defined a set of variance-based sensitivity indices for models with dependent inputs. We in this paper propose a method based on moving least squares approximation to calculate these sensitivity indices. The new proposed method is adaptable to both linear and nonlinear models since the moving least squares approximation can capture severe change in scattered data. Both linear and nonlinear numerical examples are employed in this paper to demonstrate the ability of the proposed method. Then the new sensitivity analysis method is applied to a cantilever beam structure and from the results the most efficient method that can decrease the variance of model output can be determined, and the efficiency is demonstrated by exploring the dependence of output variance on the variation coefficients of input variables. At last, we apply the new method to a headless rivet model and the sensitivity indices of all inputs are calculated, and some significant conclusions are obtained from the results.     

Least squares approximation by splines with free knots

Suppose we are given noisy data which are considered to be perturbed values of a smooth, univariate function. In order to approximate these data in the least squares sense, a linear combination of B-splines is used where the tradeoff between smoothness and closeness of the fit is controlled by a smoothing term which regularizes the least squares problem and guarantees unique solvability independent of the position of knots. Moreover, a subset of the knot sequence which defines the B-splines, the so-calledfree knots, is included in the optimization process.The resulting constrained least squares problem which is linear in the spline coefficients but nonlinear in the free knots is reduced to a problem that has only the free knots as variables. The reduced problem is solved by a generalized Gauss-Newton method. The method developed can be combined with a knot removal strategy in order to obtain an approximating spline with as few parameters as possible.Dedicated to Professor Dr.-Ing. habil. Dr. h.c. Helmut Heinrich on the occasion of his 90th birthdayResearch of the second author was partly supported by Deutsche Forschungsgemeinschaft under grant Schm 968/2-1.     

Quasi-Newton Methods for Nonlinear Least Squares Focusing on Curvatures

A new quasi-Newton method for nonlinear least squares problems is proposed. Two advantages of the method are accomplished by utilizing special geometrical properties in the problem class. First, fast convergence is established for well-conditioned problems by interpolating both the current and the previous step in each iteration. Second, high accuracy is achieved for certain difficult problems, such as ill-conditioned problems and problems with large curvatures in the tangent space. Numerical results for artificial problems and standard test problems are presented and discussed.     

Multidomain Legendre-Galerkin Least-Squares Method for Linear Differential Equations with Variable Coefficients

The multidomain Legendre-Galerkin least-squares method is developed for solving linear differential problems with variable coefficients. By introducing a flux, the original differential equation is rewritten into an equivalent first-order system, and the Legendre Galerkin is applied to the discrete form of the corresponding least squares function. The proposed scheme is based on the Legendre-Galerkin method, and the Legendre/Chebyshev-Gauss-Lobatto collocation method is used to deal with the variable coefficients and the right hand side terms. The coercivity and continuity of the method are proved and the optimal error estimate in $H^1$-norm is obtained. Numerical examples are given to validate the efficiency and spectral accuracy of our scheme. Our scheme is also applied to the numerical solutions of the parabolic problems with discontinuous coefficients and the two-dimensional elliptic problems with piecewise constant coefficients, respectively.     

Quadratic approximation of penalty functions for solving large-scale linear programs

Using the least squares, modified Lagrangian function, and some other methods as examples, the capabilities of the new optimization technique based on the quadratic approximation of penalty functions that has been recently proposed by O. Mangasarian for a special class of linear programming problems are demonstrated. The application of this technique makes it possible to use unified matrix operations and standard linear algebra packages (including parallel ones) for solving large-scale problems with sparse strongly structured constraint matrices. With this technique, the computational schemes of some well-known algorithms can take an unexpected form.