Weighted Tensor Product Algorithms for Linear Multivariate Problems

We study the ε-approximation of linear multivariate problems defined over weighted tensor product Hilbert spaces of functions f of d variables. A class of weighted tensor product (WTP) algorithms is defined which depends on a number of parameters. Two classes of permissible information are studied. Λ^all consists of all linear functionals while Λ^std consists of evaluations of f or its derivatives. We show that these multivariate problems are sometimes tractable even with a worst-case assurance. We study problem tractability by investigating when a WTP algorithm is a polynomial-time algorithm, that is, when the minimal number of information evaluations is a polynomial in 1/ε and d. For Λ^all we construct an optimal WTP algorithm and provide a necessary and sufficient condition for tractability in terms of the sequence of weights and the sequence of singular values for d=1. ForΛ^std we obtain a weaker result by constructing a WTP algorithm which is optimal only for some weight sequences.     

Tractability through increasing smoothness

We prove that some multivariate linear tensor product problems are tractable in the worst case setting if they are defined as tensor products of univariate problems with logarithmically increasing smoothness. This is demonstrated for the approximation problem defined over Korobov spaces and for the approximation problem of certain diagonal operators. For these two problems we show necessary and sufficient conditions on the smoothness parameters of the univariate problems to obtain strong polynomial tractability. We prove that polynomial tractability is equivalent to strong polynomial tractability, and that weak tractability always holds for these problems. Under a mild assumption, the Korobov space consists of periodic functions. Periodicity is crucial since the approximation problem defined over Sobolev spaces of non-periodic functions with a special choice of the norm is not polynomially tractable for all smoothness parameters no matter how fast they go to infinity. Furthermore, depending on the choice of the norm we can even lose weak tractability.     

Average case complexity of linear multivariate problems I. Theory

We study the average case complexity of linear multivariate problems, that is, the approximation of continuous linear operators on functions of d variables. The function spaces are equipped with Gaussian measures. We consider two classes of information. The first class Λ^std consists of function values, and the second class Λ^all consists of all continuous linear functionals. Tractability of a linear multivariate problem means that the average case complexity of computing an ε-approximation is O((1/)^p) with p independent of d. The smallest such p is called the exponent of the problem. Under mild assumptions, we prove that tractability in Λ^all is equivalent to tractability in Λ^std and that the difference of the exponents is at most 2. The proof of this result is not constructive. We provide a simple condition to check tractability in Λ^all. We also address the issue of how to construct optimal (or nearly optimal) sample points for linear multivariate problems. We use relations between average case and worst case settings. These relations reduce the study of the average case to the worst case for a different class of functions. In this way we show how optimal sample points from the worst case setting can be used in the average case. In Part II we shall apply the theoretical results to obtain optimal or almost optimal sample points, optimal algorithms, and average case complexity functions for linear multivariate problems equipped with the folded Wiener sheet measure. Of particular interest will be the multivariate function approximation problem.     

Tractability of quasilinear problems I: General results

The tractability of multivariate problems has usually been studied only for the approximation of linear operators. In this paper we study the tractability of quasilinear multivariate problems. That is, we wish to approximate nonlinear operators S_d(·,·) that depend linearly on the first argument and satisfy a Lipschitz condition with respect to both arguments. Here, both arguments are functions of d variables. Many computational problems of practical importance have this form. Examples include the solution of specific Dirichlet, Neumann, and Schrödinger problems. We show, under appropriate assumptions, that quasilinear problems, whose domain spaces are equipped with product or finite-order weights, are tractable or strongly tractable in the worst case setting.This paper is the first part in a series of papers. Here, we present tractability results for quasilinear problems under general assumptions on quasilinear operators and weights. In future papers, we shall verify these assumptions for quasilinear problems such as the solution of specific Dirichlet, Neumann, and Schrödinger problems.     

Tractability of Multivariate Integration for Periodic Functions

We study multivariate integration in the worst case setting for weighted Korobov spaces of smooth periodic functions of d variables. We wish to reduce the initial error by a factor ε for functions from the unit ball of the weighted Korobov space. Tractability means that the minimal number of function samples needed to solve the problem is polynomial in ε⁻¹ and d. Strong tractability means that we have only a polynomial dependence in ε⁻¹. This problem has been recently studied for quasi-Monte Carlo quadrature rules and for quadrature rules with non-negative coefficients. In this paper we study arbitrary quadrature rules. We show that tractability and strong tractability in the worst case setting hold under the same assumptions on the weights of the Korobov space as for the restricted classes of quadrature rules. More precisely, let γ_j moderate the behavior of functions with respect to the jth variable in the weighted Korobov space. Then strong tractability holds iff ∑^∞_j=1 γ_j<∞, whereas tractability holds iff lim sup_d→∞ ∑^d_j=1 γ_j/ln d<∞. We obtain necessary conditions on tractability and strong tractability by showing that multivariate integration for the weighted Korobov space is no easier than multivariate integration for the corresponding weighted Sobolev space of smooth functions with boundary conditions. For the weighted Sobolev space we apply general results from E. Novak and H. Woźniakowski (J. Complexity17 (2001), 388–441) concerning decomposable kernels.     

Explicit Cost Bounds of Algorithms for Multivariate Tensor Product Problems

We study multivariate tenser product problems in the worst case and average case settings. They are defined on functions of d variables. For arbitrary d, we provide explicit upper bounds on the costs of algorithms which compute an ϵ-approximation to the solution. The cost bounds are of the form (c(d) + 2)β₁(β₂ + β₃(ln 1/ϵ)/(d − 1))^{β₄(d − 1)}(1/ϵ)^β₅. Here c(d) is the cost of one function evaluation (or one linear functional evaluation), and β_i′s do not depend on d; they are determined by the properties of the problem for d = 1. For certain tensor product problems, these cost bounds do not exceed c(d)Kϵ^−p for some numbers K and p, both independent of d. However, the exponents p which we obtain are too large. We apply these general estimates to certain integration and approximation problems in the worst and average case settings. We also obtain an upper bound, which is independent of d, for the number, n(ϵ, d), of points for which discrepancy (with unequal weights) is at most ϵ, n(ϵ, d) ≤ 7.26ϵ^−2.454, ∀d, ϵ ≤ 1.     

A Constructive Approach to Strong Tractability Using Quasi-Monte Carlo Algorithms

We prove in a constructive way that multivariate integration in appropriate weighted Sobolev classes is strongly tractable and the ε-exponent of strong tractability is 1 (which is the best-possible value) under a stronger assumption than Sloan and Woźniakowski's assumption. We show that quasi-Monte Carlo algorithms based on the Sobol sequence and Halton sequence achieve the convergence order O(n^−1+δ) for any δ>0 independent of the dimension with a worst-case deterministic guarantee (where n is the number of function evaluations). This implies that quasi-Monte Carlo algorithms based on the Sobol and Halton sequences converge faster and therefore are superior to Monte Carlo methods independent of the dimension for integrands in suitable weighted Sobolev classes.     

Tractability of Tensor Product Linear Operators

This paper deals with the worst case setting for approximating multivariate tensor product linear operators defined over Hilbert spaces. Approximations are obtained by using a number of linear functionals from a given class of information. We consider the three classes of information: the class of all linear functionals, the Fourier class of inner products with respect to given orthonormal elements, and the standard class of function values. We wish to determine which problems are tractable and which are strongly tractable. The complete analysis is provided for approximating operators of rank two or more. The problem of approximating linear functionals is fully analyzed in the first two classes of information. For the third class of standard information we show that the possibilities are very rich. We prove that tractability of linear functionals depends on the given space of functions. For some spaces all nontrivial normed linear functionals are intractable, whereas for other spaces all linear functionals are tractable. In “typical” function spaces, some linear functionals are tractable and some others are not.     

Tractability of Multivariate Integration for Weighted Korobov Classes

We study the worst-case error of multivariate integration in weighted Korobov classes of periodic functions of d coordinates. This class is defined in terms of weights γ_j which moderate the behavior of functions with respect to successive coordinates. We study two classes of quadrature rules. They are quasi-Monte Carlo rules which use n function values and in which all quadrature weights are 1/n and rules for which all quadrature weights are non-negative. Tractability for these two classes of quadrature rules means that the minimal number of function values needed to guarantee error in the worst-case setting is bounded by a polynomial in d and ⁻¹. Strong tractability means that the bound does not depend on d and depends polynomially on ⁻¹. We prove that strong tractability holds iff ∑^∞_j=1 γ_j<∞, and tractability holds iff lim sup_d→∞ ∑^d_j=1 γ_j/log d<∞. Furthermore, strong tractability or tractability results are achieved by the relatively small class of lattice rules. We also prove that if ∑^∞_j=1 γ^1/α_j<∞, where α measures the decay of Fourier coefficients in the weighted Korobov class, then for d1, n prime and δ>0 there exist lattice rules that satisfy an error bound independent of d and of order n^−α/2+δ. This is almost the best possible result, since the order n^−α/2 cannot be improved upon even for d=1. A corresponding result is deduced for weighted non-periodic Sobolev spaces: if ∑^∞_j=1 γ^1/2_j<∞, then for d1, n prime and δ>0 there exist shifted lattice rules that satisfy an error bound independent of d and of order n^−1+δ. We also check how the randomized error of the (classical) Monte Carlo algorithm depends on d for weighted Korobov classes. It turns out that Monte Carlo is strongly tractable iff ∑^∞_j=1 log γ_j<∞ and tractable iff lim sup_d→∞ ∑^d_j=1 log γ_j/log d<∞. Hence, in particular, for γ_j=1 we have the usual Korobov space in which integration is intractable for the two classes of quadrature rules in the worst-case setting, whereas Monte Carlo is strongly tractable in the randomized setting.     

A Class of Nonlinear Averaging Integral Operators

LetHbe the class of analytic functions defined in the unit discU, and let coEdenote the convex hull of a setEinC. IfK⊂H, then an operatorI:K→His an averaging operator ifI[f](0) =f(0) andI[f](U) ⊂ cof(U), for allf∈K. The authors show that the operatorI_β,γ[f](z) ≡ [γz^−γ∫^z₀f^β(t)t^γ−1dt]^1/βis an averaging operator on certain subsets ofH.     

Tractability of linear problems defined over Hilbert spaces

We study d$d$-variate approximation problems in the worst and average case settings. We consider algorithms that use finitely many evaluations of arbitrary linear functionals. In the worst case setting, we obtain necessary and sufficient conditions for quasi-polynomial tractability and uniform weak tractability. Furthermore, we give an estimate of the exponent of quasi-polynomial tractability which cannot be improved in general. In the average case setting, we obtain necessary and sufficient conditions for uniform weak tractability. As applications we discuss some examples.     

Fast construction of irreducible polynomials over finite fields

We present a randomized algorithm that on inputting a finite field K with q elements and a positive integer d outputs a degree d irreducible polynomial in K[x]. The running time is d ^1+?(d)×(log q)^5+?(q) elementary operations. The function ? in this expression is a real positive function belonging to the class o(1), especially, the complexity is quasi-linear in the degree d. Once given such an irreducible polynomial of degree d, we can compute random irreducible polynomials of degree d at the expense of d ^1+?(d) × (log q)^1+?(q) elementary operations only.     

n-Widths,Faber Expansion,and Computation of Analytic Functions

LetBH^∞(Ω) be the space of analytic functionsfin the region Ω for which |f(z)| ≤ 1,z∈ Ω, and letKbe a compact subset of Ω. How can we compute the values of any functionf∈BH^∞(Ω) at an arbitrary pointz∈K? One of the approaches to this problem applies the results concerning then-widths and ϵ-entrophy of classBH^∞(Ω) in the metricC(K). In the case whenKhas a simply connected complement inC and Ω is a canonical neighbourhood ofK, the classical tools for approximation off∈BH^∞(Ω) inC(K) give the Faber series. This work is concerned with the following: the exact values of Kolmogorov and othern-widths of Hardy spacesH^p, then-widths and ϵ-entrophy of classBH^∞(Ω), the optimality of Faber approximations, and computing values of analytic functions with the help of Faber series.     

Non-linear Approximations Using Sets of Finite Cardinality or Finite Pseudo-dimension

We investigate optimal non-linear approximations of multivariate periodic functions with mixed smoothness. In particular, we study optimal approximation using sets of finite cardinality (as measured by the classical entropy number), as well as sets of finite pseudo-dimension (as measured by the non-linear widths introduced by Ratsaby and Maiorov). Approximation error is measured in the L_q(T^d)-sense, where T^d is the d-dimensional torus. The functions to be approximated are in the unit ball SB^r_p, θ of the mixed smoothness Besov space or in the unit ball SW^r_p of the mixed smoothness Sobolev space. For 1<p, q<∞, 0<θ⩽∞ and r>0 satisfying some restrictions, we establish asymptotic orders of these quantities, as well as construct asymptotically optimal approximation algorithms. We particularly prove that for either r>1/p and θ⩾p or r>(1/p−1/q)₊ and θ⩾min{q, 2}, the asymptotic orders of these quantities for the Besov class SB^r_p, θ are both n^−r(log n)^{(d−1)(r+1/2−1/θ)}.     

Integration and approximation in arbitrary dimensions

We study multivariate integration and approximation for various classes of functions of d variables with arbitrary d. We consider algorithms that use function evaluations as the information about the function. We are mainly interested in verifying when integration and approximation are tractable and strongly tractable. Tractability means that the minimal number of function evaluations needed to reduce the initial error by a factor of ɛ is bounded by C(d)ɛ^−p for some exponent p independent of d and some function C(d). Strong tractability means that C(d) can be made independent of d. The ‐exponents of tractability and strong tractability are defined as the smallest powers of ɛ{^-1} in these bounds. We prove that integration is strongly tractable for some weighted Korobov and Sobolev spaces as well as for the Hilbert space whose reproducing kernel corresponds to the covariance function of the isotropic Wiener measure. We obtain bounds on the ‐exponents, and for some cases we find their exact values. For some weighted Korobov and Sobolev spaces, the strong ‐exponent is the same as the ‐exponent for d=1, whereas for the third space it is 2. For approximation we also consider algorithms that use general evaluations given by arbitrary continuous linear functionals as the information about the function. Our main result is that the ‐exponents are the same for general and function evaluations. This holds under the assumption that the orthonormal eigenfunctions of the covariance operator have uniformly bounded L∞ norms. This assumption holds for spaces with shift-invariant kernels. Examples of such spaces include weighted Korobov spaces. For a space with non‐shift‐invariant kernel, we construct the corresponding space with shift-invariant kernel and show that integration and approximation for the non-shift-invariant kernel are no harder than the corresponding problems with the shift-invariant kernel. If we apply this construction to a weighted Sobolev space, whose kernel is non-shift-invariant, then we obtain the corresponding Korobov space. This enables us to derive the results for weighted Sobolev spaces. This revised version was published online in June 2006 with corrections to the Cover Date.     

When Are Quasi-Monte Carlo Algorithms Efficient for High Dimensional Integrals?

Recently, quasi-Monte Carlo algorithms have been successfully used for multivariate integration of high dimensiond, and were significantly more efficient than Monte Carlo algorithms. The existing theory of the worst case error bounds of quasi-Monte Carlo algorithms does not explain this phenomenon. This paper presents a partial answer to why quasi-Monte Carlo algorithms can work well for arbitrarily larged. It is done by identifying classes of functions for which the effect of the dimensiondis negligible. These areweightedclasses in which the behavior in the successive dimensions is moderated by a sequence of weights. We prove that the minimalworst caseerror of quasi-Monte Carlo algorithms does not depend on the dimensiondiff the sum of the weights is finite. We also prove that the minimal number of function values in the worst case setting needed to reduce the initial error by ε is bounded byCε^−p, where the exponentp∈ [1, 2], andCdepends exponentially on the sum of weights. Hence, the relatively small sum of the weights makes some quasi-Monte Carlo algorithms strongly tractable. We show in a nonconstructive way that many quasi-Monte Carlo algorithms are strongly tractable. Even random selection of sample points (done once for the whole weighted class of functions and then the worst case error is established for that particular selection, in contrast to Monte Carlo where random selection of sample points is carried out for a fixed function) leads to strong tractable quasi-Monte Carlo algorithms. In this case the minimal number of function values in theworst casesetting is of order ε^−pwith the exponentp= 2. The deterministic construction of strongly tractable quasi-Monte Carlo algorithms as well as the minimal exponentpis open.     

Tractability results for weighted Banach spaces of smooth functions

We study the L_∞-approximation problem for weighted Banach spaces of smooth d-variate functions, where d can be arbitrarily large. We consider the worst case error for algorithms that use finitely many pieces of information from different classes. Adaptive algorithms are also allowed. For a scale of Banach spaces we prove necessary and sufficient conditions for tractability in the case of product weights. Furthermore, we show the equivalence of weak tractability with the fact that the problem does not suffer from the curse of dimensionality.     

Optimal Order of Convergence and (In)Tractability of Multivariate Approximation of Smooth Functions

We study the approximation problem for C ^∞ functions f:[0,1] ^d →? with respect to a W _p ^m -norm. Here, m=[m,m,…,m], d times, with the norm of the target space defined in terms of up to m partial derivatives with respect to all d variables. The optimal order of convergence is infinite, hence excellent, but the problem is still intractable and suffers from the curse of dimensionality if m≥1. This means that the order of convergence supplies incomplete information concerning the computational difficulty of a problem. For m=0 and p=2, we prove that the problem is not polynomially tractable, but that it is weakly tractable.     

The Complexity of Two-Point Boundary-Value Problems with Analytic Data

Previous work on the complexity of elliptic boundary-value problems Lu = f assumed that class F of problem elements f was the unit ball of a Sobolev space. In this paper, we assume that F consists of analytic functions. To be specific, we consider the ϵ-complexity of a model two-point boundary-value problem −u″ + u = f in I = (−1, 1) with natural boundary conditions u′(−1) = u′(1) = 0, and the class F consists of analytic functions f bounded by 1 on a disk of radius ρ ≥ 1 centered at the origin. We find that if ρ > 1, then the ϵ-complexity is Θ(ln(ϵ⁻¹)) as ϵ → 0, and there is a finite element p-method (in the sense of Babuška) whose cost is optimal to within a constant factor. If ρ = 1, we find that the ϵ-complexity is Θ(ln²(ϵ⁻¹)) as ϵ → 0, and there is a finite element (h, p)-method whose cost is optimal to within a constant factor.     

Power series solutions to Volterra integral equations

Power series type solutions are given for a wide class of linear and q-dimensional nonlinear Volterra equations on R^p. The basic assumption on the kernel K(x, y) is that K(x, xt) has a power series in x. For example, this holds for any analytic kernel.The kernel may be strongly singular, provided certain constants are finite. One and only one such power series solution exists. Its coefficients are given by a simple iterative formula. In many cases this may be solved explicitly. In particular an explicit formula for the resolvent is given.