While writing the chapters about compact groups in my notes, I had a few times the impression that it would be useful to use the fact that there is a basis of conjugacy-invariant neighborhoods of 1 in such a group. This thought would then fork in two subthreads, one in which I noticed that I didn’t really see how to prove this, and the second in which I realized I didn’t need it anyway.
This happened again during the last week-end, with the difference that I thought of trying to prove this property using representation theory. I failed at first, and finally tried to look it up online to make sure the property was actually true. Indeed, it is, and I found this book, which has three proofs (of a slightly more general statement). But I have to admit to having an inferiority complex with respect to general topology: any argument that goes on for more than a few lines without being transparent tends to make me uneasy and confuse me, and this is what happened with these arguments.
So I tried again to use representations, and indeed it works! Here’s the idea: the regular representation $latex \rho$ of $latex G$ on $latex H=L^2(G)$ (with respect to Haar measure) is faithful, and gives a continuous injection
$latex G\rightarrow U(H)$
where the unitary group $latex U(H)$ is given the strong operator topology (if $latex G$ is infinite, it is not continuous for the operator norm topology). Hence this map is a homeomorphism onto its image (we use here the compactness of $latex G$). What we gain from seeing $latex G$ in this way as a subgroup of the unitary group of a (typically) infinite-dimensional space, and with a “strange” topology to boot, is a concrete description of a basis of open neighborhoods of 1, which is open to further manipulations.
Indeed, from the definition of the strong topology on $latex U(H)$, any open neighborhood $latex U$ of 1 contains a finite intersection of sets of the type
$latex U_{f,\epsilon}=\{g\in G\,\mid\, \|\rho(g)f-f\|\text{\textless} \epsilon\}$
where $latex f\in H$ has norm 1 and $latex \epsilon\text{\textgreater} 0$. It is then easy to see, using unitarity, that the conjugacy-invariant subset
$latex V_{f,\epsilon}=\bigcap_{x\in G}{x^{-1}U_{f,\epsilon}x}\subset U_{f,\epsilon}$
is equal to
$latex V_{f,\epsilon}=\{g\in G\,\mid\, \|\rho(g)\varphi-\varphi\|\text{\textless} \epsilon\text{ for all }\varphi\in A_f\}$
where
$latex A_f=\{\rho(x)f\,\mid\, x\in G\}\subset H$
is the orbit of $latex f$ under $latex G$. But the strong continuity of $latex \rho$ implies that the orbit map $latex g\mapsto \rho(g)f$ is continuous for fixed $latex f$, so that $latex A_f$ is compact in $latex H$ (as image of a compact set under a continuous map; here $latex H$ has the usual norm topology).
It is then a quick application of a standard compactness argument and splitting of epsilons to check that $latex V_{f,\epsilon}$ is a neighborhood of 1 contained in $latex U_{f,\epsilon}$, and intersecting finitely many of these, we see that $latex U$ contains indeed a conjugacy invariant neighborhood of 1…
I really can’t say that it is simpler than the purely topological argument (mostly because one needs to know about Haar measure!), but I find this rather nice as an exercise. It illustrates how representation theory can be useful to study a general compact group, at a very basic level, and also shows that it may be useful to embed (or project, with the orbit map) a compact group into complicated-looking infinite-dimensional beasts… (Of course, if $latex G$ has a faithful finite-dimensional representation, this can be used instead of $latex \rho$, but the purely topological argument becomes also much simpler.)
