Moderate deviations and hypothesis testing for signal detection problem

We use moderate deviations to study the signal detection problem for a diffusion model.We establish a moderate deviation principle for the log-likelihood function of the diffusion model.Then applying the moderate deviation estimates to hypothesis testing for signal detection problem we give a decision region such that its error probability of the second kind tends to zero with faster speed than the error probability of the first kind when the error probability of the first kind is approximated by e-αr(T),where α>0,r(T)=o(T) and r(T) →∞ as the observation time T goes to infinity.     

CONVERGENCE OF ONLINE GRADIENT METHOD WITH A PENALTY TERM FOR FEEDFORWARD NEURAL NETWORKS WITH STOCHASTIC INPUTS

Online gradient algorithm has been widely used as a learning algorithm for feedforward neural network training. In this paper, we prove a weak convergence theorem of an online gradient algorithm with a penalty term, assuming that the training examples are input in a stochastic way. The monotonicity of the error function in the iteration and the boundedness of the weight are both guaranteed. We also present a numerical experiment to support our results.     

The convergence rates of Shannon sampling learning algorithms

In the present paper,we provide an error bound for the learning rates of the regularized Shannon sampling learning scheme when the hypothesis space is a reproducing kernel Hilbert space(RKHS) derived by a Mercer kernel and a determined net.We show that if the sample is taken according to the determined set,then,the sample error can be bounded by the Mercer matrix with respect to the samples and the determined net.The regularization error may be bounded by the approximation order of the reproducing kernel Hilbert space interpolation operator.The paper is an investigation on a remark provided by Smale and Zhou.     

OPTIMAL ERROR ESTIMATES FOR NEDELEC EDGE ELEMENTS FOR TIME-HARMONIC MAXWELL'S EQUATIONS

In this paper, we obtain optimal error estimates in both L^2-norm and H（curl）-norm for the Nedelec edge finite element approximation of the time-harmonic Maxwell＇s equations on a general Lipschitz domain discretized on quasi-uniform meshes. One key to our proof is to transform the L^2 error estimates into the L^2 estimate of a discrete divergence-free function which belongs to the edge finite element spaces, and then use the approximation of the discrete divergence-free function by the continuous divergence-free function and a duality argument for the continuous divergence-free function. For Nedelec＇s second type elements, we present an optimal convergence estimate which improves the best results available in the literature.     

Optimal Delaunay Triangulations

The Delaunay triangulation, in both classic and more generalized sense, is studied in this paper for minimizing the linear interpolation error (measure in L^P-norm) for a given function. The classic Delaunay triangulation can then be characterized as an optimal triangulation that minimizes the interpolation error for the isotropic function ‖x‖^2 among all the triangulations with a given set of vertices. For a more general function, a functiondependent Delaunay triangulation is then defined to be an optimal triangulation that minimizes the interpolation error for this function and its construction can be obtained by a simple lifting and projection procedure. The optimal Delaunay triangulation is the one that minimizes the interpolation error among all triangulations with the same number of vertices, i.e. the distribution of vertices are optimized in order to minimize the interpolation error. Such a function-depend entoptimal Delaunay triangulation is proved to exist for any given convex continuous function.On an optimal Delaunay triangulation associated with f, it is proved that △↓f at the interior vertices can be exactly recovered by the function values on its neighboring vertices.Since the optimal Delaunay triangulation is difficult to obtain in practice, the concept of nearly optimal triangulation is introduced and two sufficient conditions are presented for a triangulation to be nearly optimal.     

Convergence of Online Gradient Method with Penalty for BP Neural Networks

Online gradient method has been widely used as a learning algorithm for training feedforward neural networks. Penalty is often introduced into the training procedure to improve the generalization performance and to decrease the magnitude of network weights. In this paper, some weight boundedness and deterministic con- vergence theorems are proved for the online gradient method with penalty for BP neural network with a hidden layer, assuming that the training samples are supplied with the network in a fixed order within each epoch. The monotonicity of the error function with penalty is also guaranteed in the training iteration. Simulation results for a 3-bits parity problem are presented to support our theoretical results.     

Unconstrained optimization reformulations of equilibrium problems

We generalize the D-gap function developed in the literature for variational inequalities to a general equilibrium problem （EP）. Through the D-gap function, the equilibrium problem is cast as an unconstrained minimization problem. We give conditions under which any stationary point of the D-gap function is a solution of EP and conditions under which it provides a global error bound for EP. Finally, these results are applied to box-constrained EP and then weaker conditions are established to obtain the desired results for box-constrained EP.     

Tong-type identity and the mean square of the error term for an extended Selberg class

In 1956, Tong established an asymptotic formula for the mean square of the error term of the summatory function of the Piltz divisor function d3(n). The aim of this paper is to generalize Tong's method to a class of Dirichlet series L(s) which satisfies a functional equation. Let a(n) be an arithmetical function related to a Dirichlet series L(s), and let E(x) be the error term of ′n xa(n). In this paper, after introducing a class of Diriclet series with a general functional equation(which contains the well-known Selberg class), we establish a Tong-type identity and a Tong-type truncated formula for the error term of the Riesz mean of the coefficients of this Dirichlet series L(s). This kind of Tong-type truncated formula could be used to study the mean square of E(x) under a certain assumption. In other words, we reduce the mean square of E(x) to the problem of finding a suitable constant σ*which is related to the mean square estimate of L(s). We shall represent some results of functions in the Selberg class of degrees 2–4.     

A dual-relax penalty function approach for solving nonlinear bilevel programming with linear lower level problem

The penalty function method, presented many years ago, is an important numerical method for the mathematical programming problems. In this article, we propose a dual-relax penalty function approach, which is significantly different from penalty function approach existing for solving the bilevel programming, to solve the nonlinear bilevel programming with linear lower level problem. Our algorithm will redound to the error analysis for computing an approximate solution to the bilevel programming. The error estimate is obtained among the optimal objective function value of the dual-relax penalty problem and of the original bilevel programming problem. An example is illustrated to show the feasibility of the proposed approach.     

AN ERROR ANALYSIS METHOD SPP-BEAM AND A CONSTRUCTION GUIDELINE OF NONCONFORMING FINITE ELEMENTS FOR FOURTH ORDER ELLIPTIC PROBLEMS

Under two hypotheses of nonconforming finite elements of fourth order elliptic problems,we present a side-patchwise projection based error analysis method(SPP-BEAM for short).Such a method is able to avoid both the regularity condition of exact solutions in the classical error analysis method and the complicated bubble function technique in the recent medius error analysis method.In addition,it is universal enough to admit generalizations.Then,we propose a sufficient condition for these hypotheses by imposing a set of in some sense necessary degrees of freedom of the shape function spaces.As an application,we use the theory to design a P3 second order triangular H2 non-conforming element by enriching two P4 bubble functions and,another P4 second order triangular H2 nonconforming finite element,and a P3 second order tetrahedral H2 non-conforming element by enriching eight P4 bubble functions,adding some more degrees of freedom.     

Elman网络梯度学习法的收敛性

考虑有限样本集上Elman网络梯度学习法的确定性收敛性．证明了误差函数的单调递减性．给出了一个弱收敛性结果和一个强收敛结果,表明误差函数的梯度收敛于0,权值序列收敛于固定点．通过数值例子验证了理论结果的正确性．     

Learning errors of linear programming support vector regression

In this paper, we give several results of learning errors for linear programming support vector regression. The corresponding theorems are proved in the reproducing kernel Hilbert space. With the covering number, the approximation property and the capacity of the reproducing kernel Hilbert space are measured. The obtained result (Theorem 2.1) shows that the learning error can be controlled by the sample error and regularization error. The mentioned sample error is summarized by the errors of learning regression function and regularizing function in the reproducing kernel Hilbert space. After estimating the generalization error of learning regression function (Theorem 2.2), the upper bound (Theorem 2.3) of the regularized learning algorithm associated with linear programming support vector regression is estimated.     

学习理论中Rademacher随机变量的应用

利用Rademacher随机变量,本文讨论了学习函数f的亏损函数及f的样本误差的估计问题,给出了f的亏损函数及样本误差的估计,同时也给出了f的亏损函数的期望值的估计,这些估计都是O(m-1/2),这里m为样本容量.     

关于ε不敏感损失函数推广误差的界

本文研究了学习理论中推广误差的界的问题.利用ε不敏感损失函数的性质,分别获得r逼近误差和估计(样本)误差的界,并在特定的假设空间上得到了学习算法推广误差的界.     

ERM learning algorithm for multi-class classification

The multi-class classification problem is considered by an empirical risk minimization (ERM) approach. The hypothesis space for the learning algorithm is taken to be a ball of a Banach space of continuous functions. When the regression function lies in some interpolation space, satisfactory learning rates for the excess misclassification error are provided in terms of covering numbers of the unit ball of the Banach space. A comparison theorem is proved and is used to bound the excess misclassification error by means of the excess generalization error.     

Hermite learning with gradient data

The problem of learning from data involving function values and gradients is considered in a framework of least-square regularized regression in reproducing kernel Hilbert spaces. The algorithm is implemented by a linear system with the coefficient matrix involving both block matrices for generating Graph Laplacians and Hessians. The additional data for function gradients improve learning performance of the algorithm. Error analysis is done by means of sampling operators for sample error and integral operators in Sobolev spaces for approximation error.     

Learning gradients by a gradient descent algorithm

We propose a stochastic gradient descent algorithm for learning the gradient of a regression function from random samples of function values. This is a learning algorithm involving Mercer kernels. By a detailed analysis in reproducing kernel Hilbert spaces, we provide some error bounds to show that the gradient estimated by the algorithm converges to the true gradient, under some natural conditions on the regression function and suitable choices of the step size and regularization parameters.     

Shannon sampling II: Connections to learning theory

We continue our study [S. Smale, D.X. Zhou, Shannon sampling and function reconstruction from point values, Bull. Amer. Math. Soc. 41 (2004) 279–305] of Shannon sampling and function reconstruction. In this paper, the error analysis is improved. Then we show how our approach can be applied to learning theory: a functional analysis framework is presented; dimension independent probability estimates are given not only for the error in the L² spaces, but also for the error in the reproducing kernel Hilbert space where the learning algorithm is performed. Covering number arguments are replaced by estimates of integral operators.     

Learning rates for least square regressions with coefficient regularization

We analyze the learning rates for the least square regression with data dependent hypothesis spaces and coefficient regularization algorithms based on general kernels. Under a very mild regularity condition on the regression function, we obtain a bound for the approximation error by estimating the corresponding K-functional. Combining this estimate with the previous result of the sample error, we derive a dimensional free learning rate by the proper choice of the regularization parameter.     

Regularized least square regression with dependent samples

In this paper we study the learning performance of regularized least square regression with α-mixing and ϕ-mixing inputs. The capacity independent error bounds and learning rates are derived by means of an integral operator technique. Even for independent samples our learning rates improve those in the literature. The results are sharp in the sense that when the mixing conditions are strong enough the rates are shown to be close to or the same as those for learning with independent samples. They also reveal interesting phenomena of learning with dependent samples: (i) dependent samples contain less information and lead to worse error bounds than independent samples; (ii) the influence of the dependence between samples to the learning process decreases as the smoothness of the target function increases.