Continuity of Approximation by Neural Networks in Lp Spaces

Devices such as neural networks typically approximate the elements of some function space X by elements of a nontrivial finite union M of finite-dimensional spaces. It is shown that if X=L ^p () (1<p< and R ^d ), then for any positive constant and any continuous function from X to M, f–(f)>f–M+ for some f in X. Thus, no continuous finite neural network approximation can be within any positive constant of a best approximation in the L ^p -norm.