Application of least square method to arbitrary-order problems with separated boundary conditions

In this paper, differential equations of arbitrary order with separated boundary conditions are converted into an optimal control problem. Then a convergent approximate solution is constructed such that the exact boundary conditions are satisfied. In fact it will be shown that for every ε>0$\epsilon >0$, there exists an approximate solution _vε$v_{\epsilon }$ of B-spline functions such that the corresponding least square error is less than ε>0$\epsilon >0$, and also _vε$v_{\epsilon }$ satisfies the exact boundary conditions. Some examples are given and the optimal errors are obtained for the sake of comparison.     

Learning rates of least-square regularized regression with polynomial kernels

This paper presents learning rates for the least-square regularized regression algorithms with polynomial kernels. The target is the error analysis for the regression problem in learning theory. A regularization scheme is given, which yields sharp learning rates. The rates depend on the dimension of polynomial space and polynomial reproducing kernel Hilbert space measured by covering numbers. Meanwhile, we also establish the direct approximation theorem by Bernstein-Durrmeyer operators in with Borel probability measure.      

Optimal learning rates for least squares regularized regression with unbounded sampling

A standard assumption in theoretical study of learning algorithms for regression is uniform boundedness of output sample values. This excludes the common case with Gaussian noise. In this paper we investigate the learning algorithm for regression generated by the least squares regularization scheme in reproducing kernel Hilbert spaces without the assumption of uniform boundedness for sampling. By imposing some incremental conditions on moments of the output variable, we derive learning rates in terms of regularity of the regression function and capacity of the hypothesis space. The novelty of our analysis is a new covering number argument for bounding the sample error.     

The least square nucleolus is a general nucleolus

This short note proves that the least square nucleolus (Ruiz et al. (1996)) and the lexicographical solution (Sakawa and Nishizaki (1994)) select the same imputation in each game with nonempty imputation set. As a consequence the least square nucleolus is a general nucleolus (Maschler et al. (1992)). Received: December 1998/Revised version: July 1999     

Error estimates for moving least square approximations

In this paper we obtain error estimates for moving least square approximations for the function and its derivatives. We introduce, at every point of the domain, condition numbers of the star of nodes in the normal equation, which are practically computable and are closely related to the approximating power of the method.     

Coordinate and subspace optimization methods for linear least squares with non-quadratic regularization

This work addresses the problem of regularized linear least squares (RLS) with non-quadratic separable regularization. Despite being frequently deployed in many applications, the RLS problem is often hard to solve using standard iterative methods. In a recent work [M. Elad, Why simple shrinkage is still relevant for redundant representations? IEEE Trans. Inform. Theory 52 (12) (2006) 5559–5569], a new iterative method called parallel coordinate descent (PCD) was devised. We provide herein a convergence analysis of the PCD algorithm, and also introduce a form of the regularization function, which permits analytical solution to the coordinate optimization. Several other recent works [I. Daubechies, M. Defrise, C. De-Mol, An iterative thresholding algorithm for linear inverse problems with a sparsity constraint, Comm. Pure Appl. Math. LVII (2004) 1413–1457; M.A. Figueiredo, R.D. Nowak, An EM algorithm for wavelet-based image restoration, IEEE Trans. Image Process. 12 (8) (2003) 906–916; M.A. Figueiredo, R.D. Nowak, A bound optimization approach to wavelet-based image deconvolution, in: IEEE International Conference on Image Processing, 2005], which considered the deblurring problem in a Bayesian methodology, also obtained element-wise optimization algorithms. We show that the last three methods are essentially equivalent, and the unified method is termed separable surrogate functionals (SSF). We also provide a convergence analysis for SSF. To further accelerate PCD and SSF, we merge them into a recently developed sequential subspace optimization technique (SESOP), with almost no additional complexity. A thorough numerical comparison of the denoising application is presented, using the basis pursuit denoising (BPDN) objective function, which leads all of the above algorithms to an iterated shrinkage format. Both with synthetic data and with real images, the advantage of the combined PCD-SESOP method is demonstrated.     

Integrating spectral clustering with wavelet based kernel partial least square regressions for financial modeling and forecasting

Traditional forecasting models are not very effective in most financial time series. To address the problem, this study proposes a novel system for financial modeling and forecasting. In the first stage, wavelet analysis transforms the input space of raw data to a time-scale feature space suitable for financial modeling and forecasting. A spectral clustering algorithm is then used to partition the feature space into several disjointed regions according to their time series dynamics. In the second stage, multiple kernel partial least square regressors ideally suited to each partitioned region are constructed for final forecasting. The proposed model outperforms neural networks, SVMs, and traditional GARCH models, significantly reducing root-mean-squared forecasting errors.     

一个新的求解线性规划最小二乘算法

§1Introduction Currently,therearetwopopularapproachesinlinearprogramming:pivotalgorithm andinterior-pointalgorithm.Manyoftheirvariantsdevelopedbothintheoryand applicationsarestillinprogress.Thepivotmethodobtainstheoptimalsolutionviamoving consecutivelytoabettercorner-pointinthefeasibleregion,anditsmodificationstryto improvethespeedofattainingtheoptimality.Incontrast,theinterior-pointalgorithmis claimedasaninterior-pointapproach,whichgoesfromafeasiblepointtoafeasiblepoint throughtheinterioroft…     

Discrepancy principles for Tikhonov regularization of ill-posed problems leading to optimal convergence rates

We propose a class ofa posteriori parameter choice strategies for Tikhonov regularization (including variants of Morozov's and Arcangeli's methods) that lead to optimal convergence rates toward the minimal-norm, least-squares solution of an ill-posed linear operator equation in the presence of noisy data.     

A smoothing least square method for nonlinear complementarity problem

By using the smoothing functions and the least square reformulation, in this paper, we present a smoothing least square method for the nonlinear complementarity problem. The method can overcome the difficulty of the non‐smooth method and a major drawback of some existed equation‐based methods. Under the standard assumptions, we obtain the global convergence of the proposed method. Copyright © 2013 John Wiley & Sons, Ltd.     

Preconditioner based on the Sherman-Morrison formula for regularized least squares problems

We propose a sparse approximate inverse preconditioner based on the Sherman-Morrison formula for Tikhonov regularized least square problems. Theoretical analysis shows that, the factorization method can take the advantage of the symmetric property of the coefficient matrix and be implemented cheaply. Combined with dropping rules, the incomplete factorization leads to a preconditioner for Krylov iterative methods to solve regularized least squares problems. Numerical experiments show that our preconditioner is competitive compared to existing methods, especially for ill-conditioned and rank deficient least squares problems.     

Meshless Galerkin algorithms for boundary integral equations with moving least square approximations

In this paper, we first give error estimates for the moving least square (MLS) approximation in the Hk norm in two dimensions when nodes and weight functions satisfy certain conditions. This two-dimensional error results can be applied to the surface of a three-dimensional domain. Then combining boundary integral equations (BIEs) and the MLS approximation, a meshless Galerkin algorithm, the Galerkin boundary node method (GBNM), is presented. The optimal asymptotic error estimates of the GBNM for three-dimensional BIEs are derived. Finally, taking the Dirichlet problem of Laplace equation as an example, we set up a framework for error estimates of the GBNM for boundary value problems in three dimensions.     

Spectral estimates of least energy solutions to the Brezis-Nirenberg problem with a variable coefficient

Let uε be a least energy solution to the Brezis-Nirenberg problem:

     

Weighted estimates for bilinear square functions with non-smooth kernels and commutators

Under weaker conditions on the kernel functions,we discuss the boundedness of bilinear square functions associated with non-smooth kernels on the product of weighted Lebesgue spaces.Moreover,we investigate the weighted boundedness of the commutators of bilinear square functions(with symbols which are BMO functions and their weighted version,respectively)on the product of Lebesgue spaces.As an application,we deduce the corresponding boundedness of bilinear Marcinkiewicz integrals and bilinear Littlewood-Paley^-functions.     

线性过程误差下概率密度函数核估计的均方相合性

设{Xt,t≥1}为一单边线性平稳过程序列,具有共同的未知密度函数f(x),本文定义通常的f(x)的核估计,在适当条件下,证明了其均方相合性.     

The polynomial Fourier transform with minimum mean square error for noisy data

In 2006, Naoki Saito proposed a Polyharmonic Local Fourier Transform (PHLFT) to decompose a signal f∈L²(Ω) into the sum of a polyharmonic componentu and a residualv, where Ω is a bounded and open domain in R^d. The solution presented in PHLFT in general does not have an error with minimal energy. In resolving this issue, we propose the least squares approximant to a given signal in L²([−1,1]) using the combination of a set of algebraic polynomials and a set of trigonometric polynomials. The maximum degree of the algebraic polynomials is chosen to be small and fixed. We show in this paper that the least squares approximant converges uniformly for a Hölder continuous function. Therefore Gibbs phenomenon will not occur around the boundary for such a function. We also show that the PHLFT converges uniformly and is a near least squares approximation in the sense that it is arbitrarily close to the least squares approximant in L² norm as the dimension of the approximation space increases. Our experiments show that the proposed method is robust in approximating a highly oscillating signal. Even when the signal is corrupted by noise, the method is still robust. The experiments also reveal that an optimum degree of the trigonometric polynomial is needed in order to attain the minimal l² error of the approximation when there is noise present in the data set. This optimum degree is shown to be determined by the intrinsic frequency of the signal. We also discuss the energy compaction of the solution vector and give an explanation to it.     

不等式约束条件下部分线性回归模型的参数估计

本文研究了不等式约束条件下部分线性回归模型的参数估计问题，利用最优化方法和贝叶斯方法，给出了不等式约束条件下部分线性回归模型的最小二乘核估计和最佳贝叶斯估计，并且证明了在一定条件下，带约束条件的最小二乘核估计在均方误差意义下要优于无约束条件的最小二乘核估计。     

不定最小二乘问题的改进的不完全双曲Gram-Schmidt预处理算法

应用改进的不完全双曲Gram-Schmidt(IHMGS)方法预处理不定最小二乘问题的共轭梯度法(CGILS)、正交分解法(ILSQR)与广义的最小剩余法(GMRES)等迭代算法来求解大型稀疏的不定最小二乘问题.数值实验表明,IHMGS预处理方法可有效提高相应算法的迭代速度,且当矩阵的条件数比较大时,效果更加显著.     

An iterative Bregman regularization method for optimal control problems with inequality constraints

We study an iterative regularization method of optimal control problems with control constraints. The regularization method is based on generalized Bregman distances. We provide convergence results under a combination of a source condition and a regularity condition on the active sets. We do not assume attainability of the desired state. Furthermore, a priori regularization error estimates are obtained.