Extreme points of the unit ball of the Fourier-Stieltjes algebra

Let be a locally compact group. Among other things, we proved in this paper that for an IN-group , the extreme points of the unit ball of the Fourier-Stieltjes algebra are not in the Fourier algebra if and only if is non-compact, or equivalently, there is no irreducible representation of which is quasi-equivalent to a subrepresentation of the left regular representation of if and only if is non-compact. This result is a non-commutative version of the following well known result: For any locally compact group , the extreme points of the unit ball of the measure algebra are not in the group algebra if and only if is non-discrete. On the other hand, we also showed that if has the RNP, then there are extreme points of the unit ball of that are in . Since it is well known there are non-compact locally compact group for which has the RNP, there exist non-compact locally compact groups where extreme points of the unit ball of can be in . This shows that the condition be an IN-group cannot be entirely removed.