Fell topology and its application for some semidirect products

Abstract

Let K be a closed subgroup of the unitary group U(d) and let ${\mathbb {H}}_d$ be the $(2d+1)$-dimensional Heisenberg group. We say that the pair $(K,{\mathbb {H}}_d)$ is a Gelfand pair when the set $L_K^1({\mathbb {H}}_d)$ of integrable K-invariant functions on ${\mathbb {H}}_d$ forms an abelian algebra under convolution. In this paper, we consider the semidirect product $G=K\ltimes {\mathbb {H}}_d,$ such that $(K,{\mathbb {H}}_d)$ is a Gelfand pair. The main interest in the current work, is to give a precise description of the Fell topology on the unitary dual, ${\widehat{G}},$ of G,  which in turn gives an explicitly description of the cortex, $\mathrm{\mathrm{cor}}(G)$ of G,  which is a closed subset in ${\widehat{G}},$ introduced by Vershik and Karpushev in (USSR Sbornik 47:513–526, 1984), as the set of all $\pi \in {\widehat{G}}$ that cannot be Hausdorff separated from the trivial one-dimensional representation $1_G$ of G.