Approximation in Learning Theory

This paper addresses some problems of supervised learning in the setting formulated by Cucker and Smale. Supervised learning, or learning-from-examples, refers to a process that builds on the base of available data of inputs x_i and outputs y_i, i = 1,...,m, a function that best represents the relation between the inputs x ∈ X and the corresponding outputs y ∈ Y. The goal is to find an estimator f_z on the base of given data z := ((x₁,y₁),...,(x_m,y_m)) that approximates well the regression function f_ρ (or its projection) of an unknown Borel probability measure ρ defined on Z = X × Y. We assume that (x_i,y_i), i = 1,...,m, are independent and distributed according to ρ. We discuss the following two problems: I. the projection learning problem (improper function learning problem); II. universal (adaptive) estimators in the proper function learning problem. In the first problem we do not impose any restrictions on a Borel measure ρ except our standard assumption that |y|≤ M a.e. with respect to ρ. In this case we use the data z to estimate (approximate) the L₂(ρ_X) projection (f_ρ)_W of f_ρ onto a function class W of our choice. Here, ρ_X is the marginal probability measure. In [KT1,2] this problem has been studied for W satisfying the decay condition ε_n(W,B) ≤ Dn^-r of the entropy numbers ε_n(W,B) of W in a Banach space B in the case B = C(X) or B = L₂(\rho_X). In this paper we obtain the upper estimates in the case ε_n(W,L₁(ρ_X)) ≤ Dn^-r with an extra assumption that W is convex. In the second problem we assume that an unknown measure ρ satisfies some conditions. Following the standard way from nonparametric statistics we formulate these conditions of the form f_ρ ∈ Θ. Next, we assume that the only a priori information available is that f_ρ belongs to a class Θ (unknown) from a known collection {Θ} of classes. We want to build an estimator that provides approximation of f_ρ close to the optimal for the class Θ. Along with standard penalized least squares estimators we consider a new method of construction of universal estimators. This method is based on a combination of two powerful ideas in building universal estimators. The first one is the use of penalized least squares estimators. This idea works well in the case of general setting with rather abstract methods of approximation. The second one is the idea of thresholding that works very well when we use wavelets expansions as an approximation tool. A new estimator that we call the big jump estimator uses the least squares estimators and chooses a right model by a thresholding criteria instead of the penalization. In this paper we illustrate how ideas and methods of approximation theory can be used in learning theory both in formulating a problem and in solving it.     

Improving on the minimum risk equivariant estimator of a location parameter which is constrained to an interval or a half-interval

For location families with densitiesf ₀(x−θ), we study the problem of estimating θ for location invariant lossL(θ,d)=ρ(d−θ), and under a lower-bound constraint of the form θ≥a. We show, that for quite general (f ₀, ρ), the Bayes estimator δ _U with respect to a uniform prior on (a, ∞) is a minimax estimator which dominates the benchmark minimum risk equivariant (MRE) estimator. In extending some previous dominance results due to Katz and Farrell, we make use of Kubokawa'sIERD (Integral Expression of Risk Difference) method, and actually obtain classes of dominating estimators which include, and are characterized in terms of δ _U . Implications are also given and, finally, the above dominance phenomenon is studied and extended to an interval constraint of the form θ∈[a, b]. Research supported by NSERC of Canada.     

The Discrete and Classical Dirichlet Problem

Let W ì \mathbbR^d{\Omega \subset \mathbb{R}^d} be some bounded domain with reasonable boundary and let f be a continuous function on the complement Ω ^c . We can construct an unique continuous function u that is harmonique on Ω and u = f on Ω ^c . Similarly, u _d is the unique function on the lattice points such that for each lattice point of Ω satisfies the “average” property with respect to its nearest neighbours and u _d = f on Ω ^c . In this paper when ∂Ω is Lipschitz I give a “best possible” estimate of ||u − u _d ||_∞.     

Analysis of the behavior of a weak solution to <Emphasis Type="Italic">m</Emphasis>-Hessian equations near the boundary of a domain

We consider the Dirichlet problem for the m-Hessian equations F _m [u] = f in a domain Ω and analyze the behavior of approximate solutions at the boundary of Ω. We show that the growth rate for weak solutions towards to the boundary locally depends on the summability exponent of f or on the fact whether f belongs to a certain Morrey type space near the boundary. The result obtained can be used for estimating the H?lder constant for weak solutions in the closed domain. Bibliography: 11 titles.     

Estimating the ratio of two scale parameters: a simple approach

We describe a simple approach for estimating the ratio ρ = σ ₂/σ ₁ of the scale parameters of two populations from a decision theoretic point of view. We show that if the loss function satisfies a certain condition, then the estimation of ρ reduces to separately estimating σ ₂ and 1/σ ₁. This implies that the standard estimator of ρ can be improved by just employing an improved estimator of σ ₂ or 1/σ ₁. Moreover, in the case where the loss function is convex in some function of its argument, we prove that such improved estimators of ρ are further dominated by corresponding ones that use all the available data. Using this result, we construct new classes of double-adjustment improved estimators for several well-known convex as well as non-convex loss functions. In particular, Strawderman-type estimators of ρ in general models are given whereas Shinozaki-type estimators of the ratio of two normal variances are briefly treated.     

Second-Order Optimality Conditions for Constrained Domain Optimization

This paper develops boundary integral representation formulas for the second variations of cost functionals for elliptic domain optimization problems. From the collection of all Lipschitz domains Ω which satisfy a constraint ∫ _Ω g(x) dx=1, a domain is sought which maximizes either , fixed x ₀∈Ω, or ℱ(Ω)=∫ _Ω F(x,u(x)) dx, where u solves the Dirichlet problem Δu(x)=−f(x), x∈Ω, u(x)=0, x∈∂Ω. Necessary and sufficient conditions for local optimality are presented in terms of the first and second variations of the cost functionals and ℱ. The second variations are computed with respect to domain variations which preserve the constraint. After first summarizing known facts about the first variations of u and the cost functionals, a series of formulas relating various second variations of these quantities are derived. Calculating the second variations depends on finding first variations of solutions u when the data f are permitted to depend on the domain Ω.     

Triangular finite elements of HCT type and classC ρ

Let τ be some triangulation of a planar polygonal domain Ω. Given a smooth functionu, we construct piecewise polynomial functionsv∈C ^ρ(Ω) of degreen=3 ρ for ρ odd, andn=3ρ+1 for ρ even on a subtriangulation τ₃ of τ. The latter is obtained by subdividing eachT∈ρ into three triangles, andv/T is a composite triangular finite element, generalizing the classicalC ¹ cubic Hsieh-Clough-Tocher (HCT) triangular scheme. The functionv interpolates the derivatives ofu up to order ρ at the vertices of τ. Polynomial degrees obtained in this way are minimal in the family of interpolation schemes based on finite elements of this type.     

Three-term asymptotics for the boundary layers of semilinear elliptic eigenvalue problems

We consider the nonlinear eigenvalue problem −Δu=λ f(u) in Ω u=0 on ∂Ω, where Ω is a ball or an annulus in R^N (N ≥ 2) and λ > 0 is a parameter. It is known that if λ >> 1, then the corresponding positive solution u_λ develops boundary layers under some conditions on f. We establish the asymptotic formulas for the slope of the boundary layers of u_λ with the exact second term and the ‘optimal’ estimate of the third term.     

Estimates for parametric Marcinkiewicz integrals in BMO and Campanato spaces

In this paper, the authors consider the behaviors of a class of parametric Marcinkiewicz integrals μ _Ω ^ρ , μ _Ω,λ ^*,ρ and μ _Ω,S ^ρ on BMO(ℝ ⁿ ) and Campanato spaces with complex parameter ρ and the kernel Ω in Llog⁺ L(S ⁿ⁻¹). Here μ _Ω,λ ^*,ρ and μ _Ω,S ^ρ are parametric Marcinkiewicz functions corresponding to the Littlewood-Paley g _λ *-function and the Lusin area function S, respectively. Under certain weak regularity condition on Ω, the authors prove that if f belongs to BMO(ℝ ⁿ ) or to a certain Campanato space, then [μ _Ω,λ ^*,ρ (f)]², [μ _Ω,S ^ρ (f)]² and [μ _Ω ^ρ (f)]² are either infinite everywhere or finite almost everywhere, and in the latter case, some kind of boundedness are also established.     

Universal bases and greedy algorithms for anisotropic function classes

We suggest a three-step strategy to find a good basis (dictionary) for non-linear m-term approximation. The first step consists of solving an optimization problem of finding a near best basis for a given function class F, when we optimize over a collection D of bases (dictionaries). The second step is devoted to finding a universal basis (dictionary) D _u ∈ D for a given pair (F, D) of collections: F of function classes and D of bases (dictionaries). This means that D_u provides near optimal approximation for each class F from a collection F. The third step deals with constructing a theoretical algorithm that realizes near best m-term approximation with regard to D _u for function classes from F. In this paper we work this strategy out in the model case of anisotropic function classes and the set of orthogonal bases. The results are positive. We construct a natural tensor-product-wavelet-type basis and prove that it is universal. Moreover, we prove that a greedy algorithm realizes near best m-term approximation with regard to this basis for all anisotropic function classes.     

Asymptotic behavior and uniqueness of blow-up solutions of quasilinear elliptic equations

In this paper, we study the asymptotic behavior of solutions of the problem Δ _p u = f (u) in Ω, u = ∞ on ∂Ω, under general conditions on the function f, where Ω _p is the p-Laplace operator. We show that the technique used by the author for the special case p = 2 works in this more general setting, and that the behavior described by various authors for the case p = 2 is easily derived from this technique for the general case.     

On classical solutions of the compressible Navier-Stokes equations with nonnegative initial densities

We study the Navier-Stokes equations for compressible barotropic fluids in a bounded or unbounded domain Ω of R³. We first prove the local existence of solutions (ρ,u) in C([0,T_*]; (ρ^∞ +H³(Ω)) × under the assumption that the data satisfies a natural compatibility condition. Then deriving the smoothing effect of the velocity u in t>0, we conclude that (ρ,u) is a classical solution in (0,T_**)×Ω for some T_** ∈ (0,T_*]. For these results, the initial density needs not be bounded below away from zero and may vanish in an open subset (vacuum) of Ω.     

Failure of Plais-Smale condition and blow-up analysis for the critical exponent problem inR 2

Let Ω be a bounded smooth domain inR ². Letf:R→R be a smooth non-linearity behaving like exp{s ²} ass→∞. LetF denote the primitive off. Consider the functionalJ:H ₀ ¹ (Ω)→R given by

Some Optimization Problems for <Emphasis Type="Italic">p</Emphasis>-Laplacian Type Equations

In this paper we study some optimization problems for nonlinear elastic membranes. More precisely, we consider the problem of optimizing the cost functional over some admissible class of loads f where u is the (unique) solution to the problem −Δ _p u+|u| ^p−2 u=0 in Ω with |∇ u| ^p−2 u _ν =f on ∂Ω. Supported by Universidad de Buenos Aires under grant X078, by ANPCyT PICT No. 2006-290 and CONICET (Argentina) PIP 5478/1438. J. Fernández Bonder is a member of CONICET. Leandro M. Del Pezzo is a fellow of CONICET.     

Solution to the boundary blowup problem for k-curvature equation

We consider the boundary blowup problem for k-curvature equation, i.e., H _k [u] = f(u) g(|Du|) in an n-dimensional domain Ω, with the boundary condition u(x) → ∞ as dist (x,∂Ω) → 0. We prove the existence result under some hypotheses. We also establish the asymptotic behavior of a solution near the boundary ∂Ω. Mathematics Subject Classification (2000) 35J65, 35B40, 53C21     

On representation of the solution of the Neumann problem in a domain with peak by the harmonic simple layer potential

The problem of finding a solution of the Neumann problem for the Laplacian in the form of a simple layer potential Vρ with unknown density ρ is known to be reducible to a boundary integral equation of the second kind to be solved for density. The Neumann problem is examined in a bounded n-dimensional domain Ω⁺ (n > 2) with a cusp of an outward isolated peak either on its boundary or in its complement Ω⁻ = R ⁿ \Ω⁺. Let Γ be the common boundary of the domains Ω^±, Tr(Γ) be the space of traces on Γ of functions with finite Dirichlet integral over R ⁿ , and Tr(Γ)* be the dual space to Tr(Γ). We show that the solution of the Neumann problem for a domain Ω⁻ with a cusp of an inward peak may be represented as Vρ⁻, where ρ⁻ ∈ Tr(Γ)* is uniquely determined for all Ψ⁻ ∈ Tr(Γ)*. If Ω⁺ is a domain with an inward peak and if Ψ⁺ ∈ Tr(Γ)*, Ψ⁺ ⊥ 1, then the solution of the Neumann problem for Ω⁺ has the representation u ⁺ = Vρ⁺ for some ρ⁺ ∈ Tr(Γ)* which is unique up to an additive constant ρ₀, ρ₀ = V ⁻¹(1). These results do not hold for domains with outward peak.     

An Application of a Mountain Pass Theorem

We are concerned with the following Dirichlet problem: −Δu(x) = f(x, u), x∈Ω, u∈H ¹ ₀(Ω), (P) where f(x, t) ∈C (×ℝ), f(x, t)/t is nondecreasing in t∈ℝ and tends to an L ^∞-function q(x) uniformly in x∈Ω as t→ + ∞ (i.e., f(x, t) is asymptotically linear in t at infinity). In this case, an Ambrosetti-Rabinowitz-type condition, that is, for some θ > 2, M > 0, 0 > θF(x, s) ≤f(x, s)s, for all |s|≥M and x∈Ω, (AR) is no longer true, where F(x, s) = ∫ ^s ₀ f(x, t)dt. As is well known, (AR) is an important technical condition in applying Mountain Pass Theorem. In this paper, without assuming (AR) we prove, by using a variant version of Mountain Pass Theorem, that problem (P) has a positive solution under suitable conditions on f(x, t) and q(x). Our methods also work for the case where f(x, t) is superlinear in t at infinity, i.e., q(x) ≡ +∞. Received June 24, 1998, Accepted January 14, 2000.     

Quasilinear elliptic equations with boundary blow-up

Assume that Ω is a bounded domain in ℝ _N withN ≥2, which has aC ²-boundary. We show that forp ∃ (1, ∞) there exists a weak solutionu of the problem δ_p u(x) = f(u(x)), x ∃ Ω with boundary blow-up, wheref is a positive, increasing function which meets some natural conditions. The boundary blow-up ofu(x) is characterized in terms of the distance ofx from ∂Ω. For the Laplace operator, our results coincide with those of Bandle and Essén [1]. Finally, for a rather wide subclass of the class of the admissible functionsf, the solution is unique whenp ∃ (1, 2].     

Second fundamental forms of holomorphic isometries of the Poincaré disk into bounded symmetric domains and their boundary behavior along the unit circle

Motivated by problems arising from Arithmetic Geometry, in an earlier article one of the authors studied germs of holomorphic isometries between bounded domains with respect to the Bergman metric. In the case of a germ of holomorphic isometry f: (Δ, λ ds _Δ²;0) → (Ω, ds _Ω²;0) of the Poincaré disk Δ into a bounded symmetric domain Ω ⋐ ℂ ^N in its Harish-Chandra realization and equipped with the Bergman metric, f extends to a proper holomorphic isometric embedding F: (Δ, λ ds _Δ²;) → (Ω, ds _Ω²) and Graph(f) extends to an affine-algebraic variety V ⊂ ℂ × ℂ ^N . Examples of F which are not totally geodesic have been constructed. They arise primarily from the p-th root map ρ _p : H → H ^p and a non-standard holomorphic embedding G from the upper half-plane to the Siegel upper half-plane H ₃ of genus 3. In the current article on the one hand we examine second fundamental forms σ of these known examples, by computing explicitly φ = |σ|². On the other hand we study on the theoretical side asymptotic properties of σ for arbitrary holomorphic isometries of the Poincaré disk into polydisks. For such mappings expressing via the inverse Cayley transform in terms of the Euclidean coordinate τ = s + it on the upper half-plane H, we have φ(τ) = t ² u(τ), where u| _t=0 ≢ 0. We show that u must satisfy the first order differential equation δu/δt| _t=0 ≡ 0 on the real axis outside a finite number of points at which u is singular. As a by-product of our method of proof we show that any non-standard holomorphic isometric embedding of the Poincaré disk into the polydisk must develop singularities along the boundary circle. The equation δu/δt| _t=0 ≡ 0 along the real axis for holomorphic isometries into polydisks distinguishes the latter maps from holomorphic isometries into Siegel upper half-planes arising from G. Towards the end of the article we formulate characterization problems for holomorphic isometries suggested both by the theoretical and the computational results of the article.     

Approximation of holomorphic functions by Taylor-Abel-Poisson means

We investigate the problem of approximation of functions ƒ holomorphic in the unit disk by means A _{ρ, r} (f) as ρ → 1−. In terms of the error of approximation by these means, a constructive characteristic of classes of holomorphic functions H _p ^r Lipα is given. The problem of the saturation of A _{ρ, r} (f) in the Hardy space H _p is solved. __________ Translated from Ukrains’kyi Matematychnyi Zhurnal, Vol. 59, No. 9, pp. 1253–1260, September, 2007.