On differentiable vectors for representations of infinite dimensional Lie groups

In this paper we develop two types of tools to deal with differentiability properties of vectors in continuous representations π:G→GL(V) of an infinite dimensional Lie group G on a locally convex space V. The first class of results concerns the space V^∞ of smooth vectors. If G is a Banach-Lie group, we define a topology on the space V^∞ of smooth vectors for which the action of G on this space is smooth. If V is a Banach space, then V^∞ is a Fréchet space. This applies in particular to C^∗-dynamical systems (A,G,α), where G is a Banach-Lie group. For unitary representations we show that a vector v is smooth if the corresponding positive definite function 〈π(g)v,v〉 is smooth. The second class of results concerns criteria for Ck-vectors in terms of operators of the derived representation for a Banach-Lie group G acting on a Banach space V. In particular, we provide for each k∈N examples of continuous unitary representations for which the space of Ck+1-vectors is trivial and the space of Ck-vectors is dense.