Integral and distance integral Cayley graphs over generalized dihedral groups

Abstract

A graph is said to be integral (resp. distance integral) if all the eigenvalues of its adjacency matrix (resp. distance matrix) are integers. Let H be a finite abelian group, and let ${\mathscr {H}}=\langle H,b\,|\,b^2=1,bhb=h^{-1},h\in H\rangle$ be the generalized dihedral group of H. The contribution of this paper is threefold. Firstly, based on the representation theory of finite groups, we obtain a necessary and sufficient condition for a Cayley graph over ${\mathscr {H}}$ to be integral, which naturally contains the main results obtained in Lu et al. (J Algebr Comb 47:585–601, 2018). Secondly, a closed-form decomposition formula for the distance matrix of Cayley graphs over any finite groups is derived. As applications, a necessary and sufficient condition for the distance integrality of Cayley graphs over ${\mathscr {H}}$ is displayed. Some simple sufficient (or necessary) conditions for the integrality and distance integrality of Cayley graph are exhibited, respectively, from which several infinite families of integral and distance integral Cayley graphs over ${\mathscr {H}}$ are constructed. And lastly, some necessary and sufficient conditions for the equivalence of integrity and distance integrity of Cayley graphs over generalized dihedral groups are obtained.

1 Introduction

Throughout this paper, we only consider simple graphs $\varGamma =(V_\varGamma , E_\varGamma )$ with vertex set $V_\varGamma$ and edge set $E_\varGamma$. The distance between two vertices $x,y\in V_\varGamma$, denoted by $d_\varGamma (x,y)$, is the length of a shortest path connecting them.

The adjacency matrix $A(\varGamma )$ of $\varGamma$ is a 0–1 $n\times n$ matrix whose (x, y)-entry is equal to 1 if and only if vertices x and y are adjacent, whereas the distance matrix $D(\varGamma )$ of $\varGamma$ is an $n\times n$ square matrix whose (x, y)-entry is equal to $d_\varGamma (x,y)$, where $n:=|V_\varGamma |.$ The set of all eigenvalues of $A(\varGamma )$ and $D(\varGamma )$ are called the spectrum and distance spectrum of $\varGamma$, respectively. Note that both $A(\varGamma )$ and $D(\varGamma )$ are real symmetric matrices. Hence, all the eigenvalues of $A(\varGamma )$ [resp. $D(\varGamma )$] are real. $\varGamma$ is said to be integral (resp. distance integral) if all the eigenvalues of $A(\varGamma )$ [resp, $D(\varGamma )$] are integers.

The notion of integral graphs was originally defined by Harary and Schwenk [12]; in the same paper, they proposed the following interesting problem: “Which graphs have integral spectra in Graphs and Combinatorics?” Since then, classifying and constructing integral graphs have become important research topics in algebraic graph theory. However, for a general graph, giving a systemic and complete solution to the aforementioned problem turns out to be extremely difficult, and the problem is yet far from being solved. Many researchers then tried to make some progress in solving this problem by studying the integrity of some special classes of graphs, for example, see [6, 24]. One of the most popular among them is the study on the integrity of Cayley graphs.

Let G be a finite group, and let S be a subset of G such that $1_G\notin S$ and $S^{-1}=S$, where $1_G$ denotes the identity element of G. The Cayley graph Cay(G, S) over G with respect to S is the graph with vertices given by the elements of G, and two vertices $g,h\in G$ are adjacent if and only if $gh^{-1}\in S$. A remarkable achievement for the spectrum of Cayley graphs is due to Babai [4] who gave an expression for the spectrum of a Cayley graph Cay(G, S) in terms of irreducible characters of the finite group G in 1979. Bridges and Mena [5] derived a complete characterization of integral Cayley graphs over abelian groups. So [22] characterized integral graphs among circulant graphs by using a different approach. Klotz and Sander [14, 15] classified finite abelian Cayley integral groups as finite abelian groups of exponent dividing 4 or 6; they also proposed the determination of all non-abelian Cayley integral groups. (A group G is said to be integral if the Cayley graph Cay(G, S) is integral for any $S\subseteq G$ satisfying $1_G\in S$ and $S^{-1}=S.)$ Alperin and Peterson [3] presented a necessary and sufficient condition for the integrity of Cayley graphs Cay(G, S) on abelian groups G by describing the structure of S. Ahmady, Bell and Mohar [2] classified all finite groups that have a non-trivial integral Cayley graph. Recently, Lu et al. [17] have obtained a necessary and sufficient condition for the integrality of Cayley graphs over dihedral groups $D_n$ by analyzing the irreducible characters of $D_n$. For more results on integral Cayley graphs, one may be referred to [1, 8, 10, 18, 19].

In distinction from the extensive studies on integral Cayley graphs, not much work was done on the distance integrity of Cayley graphs, which is also a very important research object in algebraic graph theory. One possible reason is that it is not easy to find the distance spectrum of graphs. Along with these directions, some special classes of distance integrity of Cayley graphs were studied. Renteln [20] showed that the distance spectrum of a Cayley graph over a real reflection group with respect to the set of all reflections is integral and provided a combinatorial formula for such spectrum. Foster-Greenwood and Kriloff [9] proved that the eigenvalues and distance eigenvalues of Cayley graphs over complex reflection groups with connection sets consisting of all reflections are integers.

The relationships between the integrity and distance integrity of Cayley graphs have also been considered in the literature. Ilić [13] proved that all the distance eigenvalues of integral Cayley graphs over cyclic groups are integers. Two years later, Klotz and Sander [16] extended the above result from cyclic groups to abelian groups. All these conclusions suggest that there are some intrinsic connections between the integrity and distance integrity of Cayley graphs, although it maybe not clear to find these connections just by their definitions. A very natural problem is that whether or not the integrity and the distance integrity of Cayley graphs are equivalent; if possible, under what conditions they may be equivalent.

The aforementioned works [3, 9, 13, 16, 17, 20] lead us to the study on the integrity and distance integrity of Cayley graphs over generalized dihedral groups in this paper. Given a finite abelian group H, the generalized dihedral group of H, written as ${\mathscr {H}}$, is the semidirect product of H and ${\mathbb {Z}}_2$ with ${\mathbb {Z}}_2$ acting on H by inverting elements. Specifically, let H be a finite abelian group of order n with $H=\{h_0=1_H,h_1,h_2,\ldots ,h_{n-1}\}$; then, the generalized dihedral group of H is defined as

$$\begin{aligned} {\mathscr {H}}=\big \langle H,b\,\big |\,b^2=1,bhb=h^{-1}, h\in H\big \rangle =\big \{h_0=1_H,h_1,\ldots ,h_{n-1},b,bh_1,\ldots ,bh_{n-1}\big \}. \end{aligned}$$

(1.1)

It is obvious that ${\mathscr {H}}$ is reduced to the so-called dihedral group when H is a cyclic group.

The contribution of this paper is threefold. Firstly, in Sect. 3, we present a necessary and sufficient condition for the integrality of $Cay({\mathscr {H}},S)$, where S is any subset of ${\mathscr {H}}$ satisfying $1_H\notin S$ and $S=S^{-1}$; actually, each eigenvalue of the adjacency matrix $A(Cay({\mathscr {H}},S))$ can be presented explicitly in terms of the irreducible representations of H. Our method is quite different from those given in [17]. Consequently, several new infinite families of integral Cayley graphs over generalized dihedral groups are constructed and some previously known results in [17] may be extended naturally. Secondly, in Sect. 4, we use the Fourier transform of finite groups to obtain the decomposition formula of the distance matrix, which is used to give a necessary and sufficient condition for the distance integrity of $Cay({\mathscr {H}},S)$. Several infinite classes of distance integral Cayley graphs over generalized dihedral groups are constructed. Finally, in Sect. 5, we present some necessary and sufficient conditions for the equivalence of the integrity and distance integrity of Cayley graphs over generalized dihedral groups.

2 Preliminary results

We first restate some basic results from representation theory of finite groups, e.g., see [21, 23]. We follow the notation and terminologies in [23] except if otherwise stated. Let G be a finite group, and let V be a finite-dimensional vector space over the complex field ${\mathbb {C}}$. Denote by GL(V) the group of all bijective linear maps $T:V\rightarrow V$. A representation of G on V is a group homomorphism $\rho :G\rightarrow GL(V)$. The degree of $\rho$ is the dimension of V. Suppose that V is a unitary space; that is, it is endowed with a Hermitian scalar product $\langle \cdot ,\cdot \rangle _V$. A representation $\rho : G\mapsto GL(V)$ is unitary provided that $\rho (g)$ is a unitary operator for all $g\in G$, which means that $\langle \rho (g)v_1,\rho (g)v_2\rangle _V=\langle v_1,v_2\rangle _V$ for all $g\in G$ and $v_1,v_2\in V$. It is well known that any finite-dimensional representation of a finite group can be unitarizable. Therefore, we consider only unitary representations.

Fix an orthonormal basis of V over ${\mathbb {C}}$. For each $g\in G$, the matrix ${\mathfrak {X}}(g)$ of $\rho (g)$ with respect to the orthonormal basis is a unitary matrix, and ${\mathfrak {X}}:g\mapsto {\mathfrak {X}}(g)$ defines a matrix representation of G called a matrix representation afforded by $\rho$. For every finite group G, we define the trivial (matrix) representation as the one-dimensional representation which sends every element $g\in G$ to $1\in {\mathbb {C}}$. Two unitary representations $\rho _1:G\rightarrow GL(V_1)$ and $\rho _2:G\rightarrow GL(V_2)$ of G are said to be unitarily equivalent, denoted by $\rho _1\sim \rho _2$, if there exists a unitary isomorphism of vector spaces $T:V_1\rightarrow V_2$ such that $\rho _1(g)=T^{-1}\rho _2(g)T$ for all $g\in G$. Let ${\widehat{G}}$ denote the set of irreducible pairwise inequivalent unitary representations of G. The character $\chi _\rho :G\rightarrow {\mathbb {C}}$ of $\rho$ is defined as $\chi _\rho (g)=Tr(\rho (g))$ for $g\in G$, where $Tr(\rho (g))$ is the trace of the representation $\rho (g)$ with respect to some orthonormal basis of V. A subspace $W\leqslant V$ is G-invariant if $\rho (g)w\in W$ for all $g\in G$ and $w\in W$. The trivial subspaces V and $\{0\}$ are always invariant. We say that a representation $\rho : G\rightarrow GL(V)$ is irreducible if V has no non-trivial invariant subspaces; otherwise, we say that it is reducible.

Regular representations of finite groups are useful in algebraic graph theory, and it plays an important role in our proofs of the main results. To introduce the regular representation of a finite group G, we need to recall the notion of group algebra ${\mathbb {C}}[G]$. Let ${\mathbb {C}}[G]$ denote the set of formal sums $\sum _{g\in G}a_gg$, where $a_g\in {\mathbb {C}}$ and G is any (not necessarily abelian) finite group; that is,

$$\begin{aligned} {\mathbb {C}}[G]=\left\{ \sum _{g\in G}a_gg\,\Big |\, a_g\in {\mathbb {C}}\right\} . \end{aligned}$$

Obviously, ${\mathbb {C}}[G]$ is a complex algebra having a basis consisting of the set of group elements. The addition, scalar product and multiplication on ${\mathbb {C}}[G]$ are defined as

$$\begin{aligned} \sum _{g\in G}a_gg+\sum _{g\in G}b_gg=\sum _{g\in G}(a_g+b_g)g,\ \ \lambda \cdot \left( \sum _{g\in G}a_gg\right) =\sum _{g\in G}(\lambda a_g)g \end{aligned}$$

and

$$\begin{aligned} \left( \sum _{g\in G}a_gg\right) \cdot \left( \sum _{g\in G}b_gg\right) =\sum _{g\in G}\left( \sum _{h\in G}a_hb_{h^{-1}g}\right) g, \end{aligned}$$

respectively, where $\lambda \in {\mathbb {C}}$. If $D=\sum _{g\in G}a_gg\in {\mathbb {C}}[G]$, define $D^{-1}=\sum _{g\in G}a_gg^{-1}$; if D is a subset of G, we identify D with $\sum _{d\in D}d\in {\mathbb {C}}[G].$ The regular representation $\rho _{\mathrm{reg}}$ of G on the vector space ${\mathbb {C}}[G]$ is defined by setting

$$\begin{aligned} \rho _{\mathrm{reg}}(g)\left( \sum _{s\in G}a_ss\right) =\sum _{s\in G}a_sgs=\sum \limits _{t\in G}a_{g^{-1}t}t \end{aligned}$$

for all $g\in G$ and $\sum \nolimits _{s\in G}a_ss\in {\mathbb {C}}[G]$. The following result is well known, e.g., see [21] or [23].

Lemma 2.1

Let G be a finite group with ${\widehat{G}}=\{\rho _1,\rho _2,\ldots ,\rho _h\}$, and let $\rho _{\mathrm{reg}}$ be its regular representation. Then

$$\begin{aligned} \rho _{\mathrm{reg}}(g)\sim d_1\rho _1(g)\bigoplus d_2\rho _2(g)\bigoplus \cdots \bigoplus d_h\rho _h(g), \end{aligned}$$

for each $g\in G$, where $d_i$ is the degree of $\rho _i,i=1,2,\ldots ,h$.

Babai [4] noticed that the adjacency matrix of a Cayley graph over any finite groups can be expressed by the regular representation, as we reproduce below.

Lemma 2.2

[4] Let G be a finite group with $S\subseteq G$ such that $1_G\notin S$ and $S^{-1}=S$. Then

$$\begin{aligned} A(Cay(G,S))=\sum _{s\in S}R(s), \end{aligned}$$

where R is the matrix representation corresponding to $\rho _{\mathrm{reg}}$ with respect to the basis $\{g\,|\,g\in G\}$ of ${\mathbb {C}}[G]$.

For convenience, let $S_A(Cay(G,S))$ [resp. $S_D(Cay(G,S))$] be the set of all eigenvalues of A(Cay(G, S)) [resp. D(Cay(G, S))] and let $[\lambda ]^{k}$ denote the eigenvalue $\lambda$ with multiplicity k. Let $\rho _1,\rho _2,\ldots ,\rho _k$ be all inequivalent unitary irreducible representations of G with $d_1,d_2,\ldots ,d_k$ as their degrees, respectively. Denote by $R_i(g)$ the matrix representation corresponding to $\rho _i(g)$ for $g\in G$ and $1\leqslant i\leqslant k$. Together with Lemmas 2.1–2.2, there exists an invertible matrix P such that

$$\begin{aligned} PA(Cay(G,S))P^{-1}= & {} P\left( \sum _{s\in S}R(s)\right) P^{-1} \nonumber \\= & {} d_1\sum _{s\in S}R_1(s)\bigoplus d_2\sum _{s\in S}R_2(s)\bigoplus \cdots \bigoplus d_k\sum _{s\in S}R_k(s).\nonumber \\ \end{aligned}$$

(2.1)

Consequently,

$$\begin{aligned} S_A(Cay(G,S))=\left\{ [\lambda _{1,1}]^{d_1},\ldots ,[\lambda _{1,d_1}]^{d_1},\ldots ,[\lambda _{k,1}]^{d_k},\ldots ,[\lambda _{k,d_k}]^{d_k} \right\} , \end{aligned}$$

(2.2)

where $\lambda _{i,1},\ldots ,\lambda _{i,d_i}$ are all eigenvalues of the matrix $\sum _{s\in S}R_i(s)$ for $1\leqslant i\leqslant k$.

We end this section with an elegant result due to Alperin and Peterson [3], which gave a necessary and sufficient condition for the integrality of Cayley graphs over finite abelian groups. Let G be a finite abelian group, and let ${\mathcal {F}}_G$ be the set consisting of all subgroups of G. Then the Boolean algebra B(G) is the set whose elements are obtained by arbitrary finite intersections, unions and complements of the elements in ${\mathcal {F}}_G.$ The minimal elements of B(G) are called atoms. Denote by ${\widetilde{B}}(G)$ the set of all different atoms. A multi-subset S of G is called integral if $\chi (S)=\sum _{s\in S}\chi (s)$ is an integer for every irreducible character $\chi$ of G. Alperin and Peterson [3] not only showed that each element of B(G) is the union of some atoms and each atom of B(G) has the form $[g]=\{x|\langle x\rangle =\langle g\rangle , x\in G\}$ but also determined the integrality of Cayley graphs over abelian groups, which is list in the following.

Lemma 2.3

[3] Let G be a finite abelian group and $S\subseteq G$. Then S is integral if and only if $S\in B(G)$.

Lu et al. [17] used an approach similar to those given in [3] to extend the above lemma to multi-sets.

Lemma 2.4

[17] Let G be a finite abelian group, and let T be a multi-subset of G. Then T is integral if and only if $T\in C(G)$, where $C(G)=\left\{ \bigcup _{[g]\in {\widetilde{B}}(G)}m_g[g]\,|\,m_g\in {\mathbb {N}}\right\}$ with ${\mathbb {N}}=\{0,1,2,\ldots \}$ being the set of natural numbers.

3 Integral Cayley graphs over generalized dihedral groups

The purpose of this section is to obtain a necessary and sufficient condition for the integrality of $Cay({\mathscr {H}},S)$, where ${\mathscr {H}}$ is the generalized dihedral group given as in (1.1) and S is a non-empty subset of ${\mathscr {H}}$ satisfying $1_H\notin S$ and $S=S^{-1}$. We first recall the irreducible representations of ${\mathscr {H}}$. One may be referred to [11] for details. Let $H^2$ be the set of squares in H; that is, $H^2=\{h^2\,|\,h\in H\}$. Denote by $H/H^2=\{hH^2\,|\,h\in H\}$ the quotient group with identity element $H^2$. Note that $(hH^2)^2=h^2H^2=H^2$ for any $h\in H$. Therefore, $H/H^2$ is an elementary abelian 2-group. By Sylow theorem, the order of $H/H^2$ is a power of 2, say $2^c$ for some nonnegative integer c. Now we list all the irreducible representations of ${\mathscr {H}}$ in the following.

$\bullet$ One-dimensional representations of ${\mathscr {H}}$: Since $H/H^2$ is an elementary abelian 2-group of order $2^c$, it has $2^c$ one-dimensional representations. Each of these gives rise to a one-dimensional representation of H. For each such representation $\rho$ of H, there are two corresponding one-dimensional representations of ${\mathscr {H}}$ whose restriction to H is $\rho$: One representation sends b to 1, and the other representation sends b to $-1$. Thus, we get a total of $2^{c+1}$ one-dimensional representations of ${\mathscr {H}}$.

$\bullet$ Two-dimensional irreducible representations of ${\mathscr {H}}$: There are $n-2^{c}$ of these, described as follows. These two-dimensional representations arise from all the irreducible representations of H that do not contain $H^2$ in its kernel. Start with any representation $\chi$ of H that satisfies $\chi (h^2)\ne 1$ for some $h\in H$. Consider the following matrix representation R of ${\mathscr {H}}$ corresponding to the induced representation of $\chi$ of H:

$$\begin{aligned} R(b)= \left( \begin{array}{cc} 0 &{} \quad 1 \\ 1 &{} \quad 0 \\ \end{array} \right) ,\ \ R(h)=\left( \begin{array}{cc} \chi (h) &{}\quad 0\\ 0&{}\quad \chi (h^{-1}) \\ \end{array} \right) , \end{aligned}$$

(3.1)

for each $h\in H$. Then this induced representation is irreducible. What’s more, such two induced representations are equivalent if and only if they are complex conjugates of each other. Since these representations do not descend to $H^2$, it is not equal to its complex conjugate. Consequently, we obtain $\frac{n-2^c}{2}$ inequivalent two-dimensional irreducible representations.

Now we are ready to state and prove our main result in this section, which gives a necessary and sufficient condition for the integrality of Cayley graphs over generalized dihedral groups.

Theorem 3.1

Let H be a finite abelian group and let ${\mathscr {H}}$ be its generalized dihedral group as given in (1.1). Denote by $S=S_1\cup bS_2\subseteq {\mathscr {H}}$ such that $1_H\notin S$ and $S^{-1}=S$, where $S_1,S_2$ are subsets of H. Then $Cay({\mathscr {H}},S)$ is integral if and only if $S_1\in B(H)$ and $\chi (S_2S_2^{-1})$ is a square number for all irreducible representations of H except for the ones satisfying $\chi (h^2)=1$ for all $h\in H$.

Proof

From $(bh_j)^2=h_j^{-1}h_j=1_H$ for $h_j\in S_2$, we get $(bS_2)^{-1}=bS_2$. Thus, $S=S^{-1}$ if and only if $S_1=S_1^{-1}$. By Lemma 2.2, one knows that the problem of computing the eigenvalues of the adjacency matrix $A(Cay({\mathscr {H}},S))$ can be fully converted into those of computing the eigenvalues of the matrix $\sum _{s\in S}R(s)$, where R(s) is the representation matrix corresponding to $\rho _{\mathrm{reg}}(s)$. It follows from representation theory that the matrix $\sum _{s\in S}R(s)$ is similar to a block diagonal matrix, as shown in (2.1). This suggests that for our purpose of determining the integrality of $Cay({\mathscr {H}},S)$ it is enough to ensure that the eigenvalues of the matrix $\sum _{s\in S}R_i(s)$ are integers, where $R_i$ ranges over all the irreducible matrix representations of ${\mathscr {H}}$. Note, from the irreducible representation of ${\mathscr {H}}$ (as exhibited before), that if $R_i$ is a one-dimensional representation of ${\mathscr {H}}$, then the eigenvalues of the matrix $\sum _{s\in S}R_i(s)$ are integers. Therefore, only the matrices $\sum _{s\in S}R_i(s)$ where $R_i$ are two-dimensional irreducible representation of ${\mathscr {H}}$ require further attentions.

Now assume that R is a fixed two-dimensional irreducible representation of ${\mathscr {H}}$ given as in (3.1). We then have

$$\begin{aligned} \sum _{s\in S}R(s)= & {} \sum _{s\in S_1}R(s)+\sum _{s\in S_2}R(bs)\nonumber \\= & {} \sum _{s\in S_1}R(s)+\sum _{s\in S_2}R(b)R(s)\nonumber \\= & {} \left( \begin{array}{cc} \chi (S_1) &{}\quad \chi \big (S_2^{-1}\big ) \\ \chi (S_2) &{}\quad \chi \big (S_1^{-1}\big ) \\ \end{array} \right) \nonumber \\= & {} \left( \begin{array}{cc} \chi (S_1) &{}\quad \chi \big (S_2^{-1}\big ) \\ \chi (S_2) &{}\quad \chi (S_1) \\ \end{array} \right) , \end{aligned}$$

(3.2)

where the last equality in (3.2) follows from the fact that $S_1=S_1^{-1}$. Therefore,

$$\begin{aligned} \det \left( xI_2-\sum _{s\in S}R(s)\right)= & {} \left| \begin{array}{cc} x-\chi (S_1) &{}\quad -\chi \big (S_2^{-1}\big ) \\ -\chi (S_2) &{}\quad x-\chi (S_1) \\ \end{array} \right| \\= & {} (x-\chi (S_1))^2-\chi (S_2)\chi \big (S_2^{-1}\big )\\= & {} x^2-2\chi (S_1)x+\left( \chi (S_1)\right) ^2-\chi \big (S_2S_2^{-1}\big ). \end{aligned}$$

Consequently, the eigenvalues of $\sum _{s\in S}R(s)$ are

$$\begin{aligned} x_{1}=\chi (S_1)+\sqrt{\chi \big (S_2S_2^{-1}\big )},\ \ x_{2}=\chi (S_1)-\sqrt{\chi \big (S_2S_2^{-1}\big )}. \end{aligned}$$

(3.3)

If both $x_{1}$ and $x_{2}$ are integers, then by (3.3), $\chi (S_1)=\frac{x_{1}+x_{2}}{2}$ is a rational number. Since $\chi (S_1)$ is an algebraic number, $\chi (S_1)$ is thus forced to be an integer and so does $\sqrt{\chi (S_2S_2^{-1})}$, which means $\chi (S_2S_2^{-1})$ is a square number. Conversely, it is obvious that if $\chi (S_1)$ is an integer and $\chi (S_2S_2^{-1})$ is a square number, then both $x_{1}$ and $x_{2}$ are integers. Consequently, it follows from Lemma 2.3 and (2.2) that $Cay({\mathscr {H}},S)$ is integral if and only if $S_1\in B(H)$ and $\chi \left( S_2S_2^{-1}\right)$ is a square number, where $\chi$ ranges over all irreducible representations of H except for the ones satisfying $\chi (h^2)=1$ for all $h\in H$. This completes the proof. $\square$

As a first corollary of Theorem 3.1, we reobtain a necessary and sufficient condition for the integrality of Cayley graphs over dihedral groups, which is the main result of [17]. Note that [17] used the irreducible characters of the dihedral groups $D_n$; we list them below. Let $H=\langle a\rangle$ be a cyclic group of order n; thus, ${\mathscr {H}}$ is reduced to the dihedral group $D_n=\langle a,b\,|\,a^n=b^2=1,bab=a^{-1}\rangle$. The irreducible characters of $D_n$ have been completely characterized, e.g., see [21].

Lemma 3.2

[21] The irreducible characters of $D_n$ are given in Table 1 if n is odd and in Table 2 otherwise, where $\psi _i$ and $\phi _j$ are, respectively, the irreducible characters of degree one and two for $1\leqslant j\leqslant \lfloor \frac{n-1}{2}\rfloor$.

Table 1 Character table of $D_n$ for odd n

Table 2 Character table of $D_n$ for even n

Let $S'=S_1'\cup S_2'\subseteq D_n{\setminus }\{1_{D_n}\}$ be such that $S'=S'^{-1}$, where $S_1'\subseteq \langle a\rangle$ and $S_2'\subseteq b\langle a\rangle$. Denote $\phi$ and $\chi$ by the irreducible characters of $D_n$ and $\langle a\rangle$, respectively. Assume that $S_2'=\{ba^{l_1},ba^{l_2},\ldots ,ba^{l_t}\}$, where $\{l_1,l_2,\ldots ,l_t\}\subseteq \{1,2,\ldots ,n\}$. Then it is routine to check that $S_2'^2=\left\{ a^{l_i-l_j}\,|\,i,j\in \{1,2,\ldots ,t\}\right\}$. On the one hand, after direct calculations, we have

$$\begin{aligned} \phi _h\big (S_2'^2\big )= & {} 2\sum _{i,j\in \{1,2,\ldots ,t\}}\cos \frac{2\pi h(l_i-l_j)}{n}\nonumber \\= & {} 4\sum _{1\leqslant i<j\leqslant t}\cos \frac{2\pi h(l_j-l_i)}{n}+2t, \end{aligned}$$

(3.4)

where the first equality in (3.4) follows by Lemma 3.2. On the other hand, note that $\chi _h(a)=\omega ^{h},$ where $\omega$ denotes a primitive nth root of unity. Then

$$\begin{aligned} \chi _h\big (S_2'^2\big )= & {} \sum _{i,j\in \{1,2,\ldots ,t\}}\omega ^{ h(l_i-l_j)}\\= & {} \sum _{\begin{array}{c} i\ne j\\ {i,j\in \{1,2,\ldots ,t\}} \end{array}}\omega ^{ h(l_i-l_j)}+t\\= & {} 2\sum _{1\leqslant i<j\leqslant t}\cos \frac{2\pi h(l_j-l_i)}{n}+t \end{aligned}$$

for $1\leqslant h\leqslant \lfloor \frac{n-1}{2}\rfloor$. Therefore,

$$\begin{aligned} \phi _h\big (S_2'^2\big )=2\chi _h\big (S_2'^2\big ). \end{aligned}$$

(3.5)

By virtue of Theorem 3.1 and (3.5), we can immediately get the following corollary about the integrality of Cayley graphs over dihedral groups.

Corollary 3.3

[17] Let $D_n=\langle a,b\,|\,a^n=b^2=1,bab=a^{-1}\rangle$ be the dihedral group, and let $S'=S_1'\cup S_2'\subseteq D_n$ such that $1_{D_n}\notin S'$ and $S'^{-1}=S'$, where $S_1'\subseteq \langle a\rangle$ and $S_2'\subseteq b\langle a\rangle$. Then $Cay(D_n,S')$ is integral if and only if $S_1'\in B(\langle a\rangle )$ and $2\phi _h(S_2'^2)$ is a square number for all $1\leqslant h\leqslant \lfloor \frac{n-1}{2}\rfloor .$

The following three corollaries are immediate consequences of Lemma 2.3 and Theorem 3.1, which can be used to generate infinite families of integral Cayley graphs over ${\mathscr {H}}$.

Corollary 3.4

Let H be a finite abelian group, and let ${\mathscr {H}}$ be its generalized dihedral group. Let $S=S_1\cup bS_2\subseteq {\mathscr {H}}$ such that $1_H\notin S$ and $S^{-1}=S$, where $S_1,S_2\subseteq H$ with $|S_2|=1$. Then $Cay({\mathscr {H}},S)$ is integral if and only if $S_1\in B(H)$.

Proof

We just note that $\chi (S_2S_2^{-1})=\chi (1)=1$ holds for any irreducible representation $\chi$ of H. $\square$

Corollary 3.5

Let H be a finite abelian group, and let ${\mathscr {H}}$ be its generalized dihedral group. Denote $S=S_1\cup bS_2\subseteq {\mathscr {H}}$ such that $1_H\notin S$ and $S_i^{-1}=S_i$ for $i=1,2$, where $S_1,S_2\subseteq H$. Then $Cay({\mathscr {H}},S)$ is integral if and only if $S_1,S_2\in B(H)$.

Proof

In this case, we note that $\chi (S_2S_2^{-1})=\chi (S_2)\chi (S_2^{-1})=\chi (S_2)^2$. Then the desired result follows from Lemma 2.3 and Theorem 3.1. $\square$

For all irreducible representations of H except for the ones satisfying $\chi (h^2)=1$ for all $h\in H$, one hopes to characterize those subsets $S_2$ of H such that $\chi (S_2S_2^{-1})$ is a square number, which seems not to be an easy task. Here, we consider the simplest case. Suppose that H is a finite abelian group of odd order n and that $\chi (S_2S_2^{-1})$ is equal to a fixed square number for all non-trivial representation $\chi$ of H, say $\mu ^2$. Assume further that the value of $\mu ^2$ is less than the size of $S_2$, i.e., $\mu ^2<|S_2|$. It follows that

$$\begin{aligned} S_2S_2^{-1}=|S_2|1_H+\big (|S_2|-\mu ^2\big )\big (H-1_H\big ). \end{aligned}$$

This reveals that $S_2$ is a difference set in H with parameters $(n,|S_2|, |S_2|-\mu ^2)$. We have thus arrived at the following corollary.

Corollary 3.6

Let H be a finite abelian group of odd order n, and let ${\mathscr {H}}$ be its generalized dihedral group. Suppose $S_1$ and $S_2$ are two subsets of H satisfying $1_H\notin S_1$, $S_1\in B(H)$ and $S_2$ is an $(n,|S_2|,\lambda )$-difference set of H. If $|S_2|-\lambda$ is a square number, then $Cay({\mathscr {H}},S)$ is integral.

We mention that there are two well-known classes of difference sets meeting the requirements of Corollary 3.6. One is a particular class of the famous singer difference set: Let q be an even power of a prime number. Then there exists a $\left( q^2+q+1,q+1,1\right)$-difference set in the additive group of integers modulo $q^2+q+1$. The second class is the so-called twin prime power difference set: Let q and $q+2$ be odd prime powers. Then there exists a difference set in the group $({\mathbb {F}}_q, +)\oplus ({\mathbb {F}}_{q+2}, +)$ with parameters $\left( q^2+2q, \frac{q^2+2q-1}{2}, \frac{q^2+2q-3}{4}\right)$. Please refer to [7] for the constructions of such difference sets.

We conclude this section with the following corollary, which gives a necessary condition for the integrality of $Cay({\mathscr {H}},S)$.

Corollary 3.7

Let H be a finite abelian group, and let ${\mathscr {H}}$ be its generalized dihedral group. Denote by $S=S_1\cup bS_2\subseteq {\mathscr {H}}$ such that $1_H\notin S$ and $S^{-1}=S$, where $S_1,S_2\subseteq H$. If $Cay({\mathscr {H}},S)$ is integral, then $S_1\in B(H)$ and $S_2S_2^{-1}\in C(H)$.

4 Distance integral Cayley graphs over generalized dihedral groups

In this section, we study the distance integrality of $Cay({\mathscr {H}},S)$, where S is a non-empty subset of ${\mathscr {H}}$ satisfying $1_H\notin S, S=S^{-1}$ and $\langle S\rangle ={\mathscr {H}}$. A necessary and sufficient condition for the distance integrality of $Cay({\mathscr {H}},S)$ is derived, and some infinite families of distance integral Cayley graphs over generalized dihedral groups are constructed. To do this, we firstly establish a closed-form formula for the distance matrix of Cayley graphs over any finite groups by using the Fourier transform of finite groups.

For convenience, we begin to restate the main results about the Fourier transform of finite groups (see also [23]). We adopt the notation in [23]. Let G be a finite group, and let L(G) be the vector space of all complex-valued functions defined on G. Then the set $\{\delta _g\,|\,g\in G\}$ is an orthogonal basis of L(G), where $\delta _g(x)=\left\{ \begin{array}{ll} 1, &{}\quad {\text {if }}g=x; \\ 0, &{}\quad {\text {if }}g\ne x. \end{array} \right.$ Define a representation $\rho _{\mathrm{reg}}$ of G on L(G) by setting

$$\begin{aligned}{}[\rho _{\mathrm{reg}}(g)f](g_0)=f\left( g^{-1}g_0\right) ~\hbox {for all } g,g_0\in G\hbox { and }f\in L(G). \end{aligned}$$

This is indeed a representation:

$$\begin{aligned}{}[\rho _{\mathrm{reg}}(g_1g_2)f](g_0)=f\left( g_2^{-1}g_1^{-1}x\right) =[\rho _{\mathrm{reg}}(g_2)f]\left( g_1^{-1}g_0\right) =\rho _{\mathrm{reg}}(g_1)[\rho _{\mathrm{reg}}(g_2)f](g_0). \end{aligned}$$

That is, $\rho _{\mathrm{reg}}(g_1g_2)=\rho _{\mathrm{reg}}(g_1)\rho _{\mathrm{reg}}(g_2).$ The above representation $\rho _{\mathrm{reg}}$ is called the regular representation. Let ${\widehat{G}}$ be a fixed set of irreducible unitary pairwise inequivalent representation of G. Given two finite-dimensional vector spaces V and W over the complex field ${\mathbb {C}}$, we denote by Hom(V, W) the space of all linear maps from V to W.

In what follows, set ${\mathcal {A}}(G)=\bigoplus _{\rho \in {\widehat{G}}}Hom(W_\rho ,W_\rho )$, where $W_\rho$ denotes the vector space corresponding to the representation $\rho$. For every $\rho \in {\widehat{G}}$, fix an orthonormal basis $\{v_1^\rho ,v_2^\rho ,\ldots ,v_{d_\rho }^\rho \}$ in the representation space $W_\rho$. Define the element $T_{i,j}^\rho \in {\mathcal {A}}(G)$ by $T_{i,j}^\rho (w)=\delta _{\rho ,\sigma }\langle w,v_j^\rho \rangle _{W_\rho }v_i^\rho$ for $w\in W_\sigma , \sigma \in {\widehat{G}}$ and $i,j=1,2,\ldots ,d_\rho$, where $\delta _{\rho ,\sigma }$ is the indicator function $\delta _{\rho ,\sigma }=\left\{ \begin{array}{ll} 1, &{}\quad {\text {if }}\rho =\sigma ; \\ 0, &{} \quad {\text {if }}\rho \ne \sigma . \end{array} \right.$ It has been proved that the set $\{T_{i,j}^\rho \,|\,\rho \in {\widehat{G}},i,j=1,2,\ldots ,d_\rho \}$ is an orthogonal basis of ${\mathcal {A}}(G)$.

The Fourier transform is the linear map ${\mathcal {F}}:L(G)\rightarrow {\mathcal {A}}(G)$ which satisfies ${\mathcal {F}}(f)=\bigoplus _{\rho \in {\widehat{G}}}\rho (f)$ for $f\in L(G)$. In correspondence with the orthonormal basis $\{v_1^\rho ,v_2^\rho ,\ldots ,v_{d_\rho }^\rho \}$, we define $\varphi _{i,j}^\rho (g)=\langle \rho (g)v_j^\rho ,v_i^\rho \rangle _{W_\rho }$. With the above notations, it has been showed that the Fourier transform has the following property.

Lemma 4.1

[23] Let ${\mathcal {F}}$ be the Fourier transform from L(G) to ${\mathcal {A}}(G)$. Then

$$\begin{aligned} {\mathcal {F}}\overline{\varphi _{i,j}^\rho }=\frac{|G|}{d_\rho }T_{i,j}^\rho , \end{aligned}$$

where $\overline{\varphi _{i,j}^\rho }$ is the conjugate of $\varphi _{i,j}^\rho .$

Given a Cayley graph Cay(G, S) with $1_G\notin S=S^{-1}$ and $\langle S\rangle =G$, define a function $\ell _S:G\rightarrow {\mathbb {C}}$ such that

$$\begin{aligned} \ell _S(g)=\left\{ \begin{array}{ll} \min \{k\,|\,g=s_1s_2\ldots s_k, s_i\in S\}, &{} \quad {\text {if }}g\ne 1; \\ 0, &{} \quad {\text {if }}g=1. \end{array} \right. \end{aligned}$$

Then it is obvious that $d_{Cay(G,S)}(g,h)=\ell _S(gh^{-1})$ for $g,h\in G$. Note that

$$\begin{aligned} \left( \rho _{\mathrm{reg}}(\ell _S)\right) (\delta _h)=\sum _{g\in G}\ell _S(g)\rho _{\mathrm{reg}}(g)(\delta _h)=\sum _{g\in G}\ell _S(g)\delta _{gh}=\sum _{g\in G}\ell _S(gh^{-1})\delta _g. \end{aligned}$$

As given in [9], the matrix of $\rho _{\mathrm{reg}}(\ell _S)$ with respect to the basis $\{\delta _g\,|\,g\in G\}$ of L(G) is exactly the distance matrix D(Cay(G, S)).

Fig. 1

Composition of transformations from ${\mathcal {A}}(G)$ to ${\mathcal {A}}(G)$

In the following, we use the composition of transformations as depicted in Fig. 1 to find the eigenvalues of the distance matrix D(Cay(G, S)). By Lemma 4.1, we have

$$\begin{aligned} \left[ {\mathcal {F}}\rho _{\mathrm{reg}}(\ell _S){\mathcal {F}}^{-1}\right] T_{i,j}^\rho= & {} \frac{d_\rho }{|G|}{\mathcal {F}}\rho _{\mathrm{reg}}(\ell _S)\overline{\varphi _{i,j}^\rho }\nonumber \\= & {} \frac{d_\rho }{|G|}{\mathcal {F}}\left( \sum _{g\in G}\ell _S(g)\rho _{\mathrm{reg}}(g)\overline{\varphi _{i,j}^\rho }\right) \nonumber \\= & {} \frac{d_\rho }{|G|}{\mathcal {F}}\left( \sum _{g\in G}\ell _S(g) \sum _{k=1}^{d_\rho }\varphi _{k,i}^\rho (g)\overline{\varphi _{k,j}^\rho }\right) \end{aligned}$$

(4.1)

$$\begin{aligned}= & {} \sum _{g\in G}\ell _S(g)\left( \sum _{k=1}^{d_\rho }\varphi _{k,i}^\rho (g)T_{k,j}^\rho \right) \nonumber \\= & {} \sum _{k=1}^{d_\rho }\left( \sum _{g\in G}\ell _S(g)\varphi _{k,i}^\rho (g)\right) T_{k,j}^\rho , \end{aligned}$$

(4.2)

where the third equality in (4.1) follows from the fact that

$$\begin{aligned} \rho _{\mathrm{reg}}(g)\overline{\varphi _{i,j}^\rho }(h)= & {} \overline{\varphi _{i,j}^\rho }(g^{-1}h)\\= & {} \overline{\langle \rho (g^{-1}h)v_j^\rho ,v_i^\rho \rangle }\\= & {} \overline{\langle \rho (h)v_j^\rho ,\rho (g)v_i^\rho \rangle }\\= & {} \overline{\left\langle \rho (h)v_j^\rho ,\sum _{k=1}^{d_\rho }\left\langle \rho (g)v_i^\rho ,v_k^\rho \right\rangle v_k^\rho \right\rangle }\\= & {} \sum _{k=1}^{d_\rho }\left\langle \rho (g)v_i^\rho ,v_k^\rho \right\rangle \overline{\left\langle \rho (h)v_j^\rho , v_k^\rho \right\rangle }\\= & {} \sum _{k=1}^{d_\rho }\varphi _{k,i}^\rho (g)\overline{\varphi _{k,j}^\rho }(h) \end{aligned}$$

for any $h\in G$. Equality (4.2) indicates that $Hom(W_\rho ,W_\rho )$ is an invariant subspace of ${\mathcal {F}}\rho _{\mathrm{reg}}(\ell _S){\mathcal {F}}^{-1}$ for each $\rho \in {\widehat{G}}$. Therefore, we have the following decomposition formula about the distance matrix of Cayley graphs over any finite groups.

Theorem 4.2

Let G be a finite group with $S\subseteq G$ such that $1_G\notin S=S^{-1}$ and $\langle S\rangle =G$. Let $\rho _1,\rho _2,\ldots ,\rho _h$ be all inequivalent irreducible unitary representations of G with $d_1,d_2,\ldots ,d_h$ as their degrees, respectively. Then there exists an invertible matrix Q such that

$$\begin{aligned} QD(Cay(G,S))Q^{-1}=d_1\Phi (\rho _1)\bigoplus d_2\Phi (\rho _2)\bigoplus \cdots \bigoplus d_h\Phi (\rho _h), \end{aligned}$$

where $\Phi (\rho _k)$ denotes the $d_k\times d_k$ matrix whose (i, j)-entry is equal to $\sum _{g\in G}\ell _S(g)\varphi _{i,j}^{\rho _k}(g)$ for $i,j=1,2,\ldots ,d_k$ and $k=1,2,\ldots ,h$. Consequently,

$$\begin{aligned} S_D(Cay(G,S))=\left\{ [\mu _{1,1}]^{d_1},\ldots ,[\mu _{1,d_1}]^{d_1},\ldots ,[\mu _{h,1}]^{d_h},\ldots ,[\mu _{h,d_h}]^{d_h}\right\} , \end{aligned}$$

where $\mu _{i,1},\ldots ,\mu _{i,d_i}$ are all eigenvalues of the matrix $\Phi (\rho _i)$ for $1\leqslant i\leqslant h$.

We are now in a position to give a necessary and sufficient condition for the distance integrality of Cayley graphs over generalized dihedral groups.

Theorem 4.3

Let H be a finite abelian group, and let ${\mathscr {H}}$ be its generalized dihedral group as given in (1.1). Denote by $S\subseteq {\mathscr {H}}$ such that $1_H\notin S=S^{-1}$ and $\langle S\rangle ={\mathscr {H}}$. Then $Cay({\mathscr {H}},S)$ is distance integral if and only if $\sum _{h\in H}\ell _S(h)h\in C(H)$ and $\chi \left[ \sum _{h\in H}\ell _S(bh)h\cdot \sum _{h\in H}\ell _S(bh)h^{-1}\right]$ is a square number for each irreducible representation of H except for the ones satisfying $\chi (h^2)=1$ for all $h\in H$.

Proof

Given a representation $\rho \in \widehat{{\mathscr {H}}}$, then $d_\rho =1$ or $d_\rho =2.$ If $d_\rho =1$, assume that $W_\rho =\{\alpha v_1^\rho |\alpha \in {\mathbb {C}}\}.$ Then

$$\begin{aligned} \Phi (\rho )=\sum _{g\in G}\ell _S(g)\varphi _{1,1}^\rho (g)=\sum _{g\in G}\ell _S(g)\langle \rho (g)v_1^\rho ,v_1^\rho \rangle =\sum _{g\in G}\ell _S(g)\rho (g). \end{aligned}$$

Since both $\ell _S(g)$ and $\rho (g)$ are integers, $\Phi (\rho )$ is an integer. If $d_\rho =2$, assume that $W_\rho =\{\beta v_1^\rho +\gamma v_2^\rho |\beta ,\gamma \in {\mathbb {C}}\}$, where $\{v_1^\rho ,v_2^\rho \}$ is an orthonormal basis corresponding to the induced representation (3.1). It follows from Theorem 4.2 that

$$\begin{aligned} \Phi (\rho )=\left( \begin{array}{cc} {\sum }_{g\in G}\ell _S(g)\varphi _{1,1}^\rho (g) &{}\quad {\sum }_{g\in G}\ell _S(g)\varphi _{1,2}^\rho (g) \\ {\sum }_{g\in G}\ell _S(g)\varphi _{2,1}^\rho (g) &{}\quad {\sum }_{g\in G}\ell _S(g)\varphi _{2,2}^\rho (g) \\ \end{array} \right) . \end{aligned}$$

(4.3)

In view of (3.1), we have

$$\begin{aligned} \left\{ \begin{array}{l} \rho (b)v_1^\rho =v_2^\rho ,\\ \rho (b)v_2^\rho =v_1^\rho ,\\ \rho (h)v_1^\rho =\chi (h)v_1^\rho ,\\ \rho (h)v_2^\rho =\chi (h^{-1})v_2^\rho \end{array} \right. \end{aligned}$$

for each $h\in H$. Thus, we can obtain

$$\begin{aligned} \left\{ \begin{array}{l} \varphi _{1,1}^\rho (h_i)=\langle \rho (h_i)v_1^\rho ,v_1^\rho \rangle =\langle \chi (h_i)v_1^\rho ,v_1^\rho \rangle =\chi (h_i),\\ \varphi _{1,1}^\rho (bh_i)=\langle \rho (bh_i)v_1^\rho ,v_1^\rho \rangle =\langle \chi (h_i)\rho (b)v_1^\rho ,v_1^\rho \rangle =\langle \chi (h_i)v_2^\rho ,v_1^\rho \rangle =0,\\ \varphi _{1,2}^\rho (h_i)=\langle \rho (h_i)v_2^\rho ,v_1^\rho \rangle =\langle \chi (h_i^{-1})v_2^\rho ,v_1^\rho \rangle =0,\\ \varphi _{1,2}^\rho (bh_i)=\langle \rho (bh_i)v_2^\rho ,v_1^\rho \rangle =\langle \chi (h_i^{-1})\rho (b)v_2^\rho ,v_1^\rho \rangle =\langle \chi (h_i^{-1})v_1^\rho ,v_1^\rho \rangle =\chi (h_i^{-1}),\\ \varphi _{2,1}^\rho (h_i)=\langle \rho (h_i)v_1^\rho ,v_2^\rho \rangle =\langle \chi (h_i)v_1^\rho ,v_2^\rho \rangle =0,\\ \varphi _{2,1}^\rho (bh_i)=\langle \rho (bh_i)v_1^\rho ,v_2^\rho \rangle =\langle \chi (h_i)\rho (b)v_1^\rho ,v_2^\rho \rangle =\langle \chi (h_i)v_2^\rho ,v_2^\rho \rangle =\chi (h_i),\\ \varphi _{2,2}^\rho (h_i)=\langle \rho (h_i)v_2^\rho ,v_2^\rho \rangle =\langle \chi (h_i^{-1})v_2^\rho ,v_2^\rho \rangle =\chi (h_i^{-1}),\\ \varphi _{2,2}^\rho (bh_i)=\langle \rho (bh_i)v_2^\rho ,v_2^\rho \rangle =\langle \chi (h_i^{-1})\rho (b)v_2^\rho ,v_2^\rho \rangle =\langle \chi (h_i^{-1})v_1^\rho ,v_2^\rho \rangle =0 \end{array} \right. \nonumber \\ \end{aligned}$$

(4.4)

for $i=0,1,\ldots ,n-1.$ By equalities (4.3) and (4.4), one has

$$\begin{aligned} \Phi (\rho )= & {} \left( \begin{array}{ll} {\sum }_{h\in H}\ell _S(h)\chi (h)&{}\quad {\sum }_{h\in H}\ell _S(bh)\chi (h^{-1}) \\ {\sum }_{h\in H}\ell _S(bh)\chi (h) &{}\quad {\sum }_{h\in H}\ell _S(h)\chi (h^{-1}) \\ \end{array} \right) . \end{aligned}$$

Recall that $S=S^{-1}$, we get $\ell _S(h)=\ell _S(h^{-1})$ for all $h\in H$. Then

$$\begin{aligned} \sum _{h\in H}\ell _S(h)\chi (h^{-1})=\sum _{h\in H}\ell _S(h^{-1})\chi (h^{-1})=\sum _{h^{-1}\in H}\ell _S(h)\chi (h)=\sum _{h\in H}\ell _S(h)\chi (h). \end{aligned}$$

This leads to

$$\begin{aligned} \Phi (\rho )= & {} \left( \begin{array}{ll} {\sum }_{h\in H}\ell _S(h)\chi (h)&{}\quad {\sum }_{h\in H}\ell _S(bh)\chi (h^{-1}) \\ {\sum }_{h\in H}\ell _S(bh)\chi (h) &{}\quad {\sum }_{h\in H}\ell _S(h)\chi (h) \\ \end{array} \right) . \end{aligned}$$

(4.5)

Therefore,

$$\begin{aligned} \det \left( xI_2-\Phi (\rho )\right)= & {} \left| \begin{array}{cc} x-{\sum }_{h\in H}\ell _S(h)\chi (h) &{}\quad -{\sum }_{h\in H}\ell _S(bh)\chi (h^{-1}) \\ - {\sum }_{h\in H}\ell _S(bh)\chi (h) &{}\quad x-{\sum }_{h\in H}\ell _S(h)\chi (h) \\ \end{array} \right| \\= & {} \left( x-\sum _{h\in H}\ell _S(h)\chi (h)\right) ^2-\sum _{h\in H}\ell _S(bh)\chi (h)\cdot \sum _{h\in H}\ell _S(bh)\chi (h^{-1})\\= & {} \left[ x-\chi \left( \sum _{h\in H}\ell _S(h)h\right) \right] ^2-\chi \left( \sum _{h\in H}\ell _S(bh)h\cdot \sum _{h\in H}\ell _S(bh)h^{-1}\right) . \end{aligned}$$

Consequently, the eigenvalues of $\Phi (\rho )$ are

$$\begin{aligned} x_1= & {} \chi \left( \sum _{h\in H}\ell _S(h)h\right) +\sqrt{\chi \left( \sum _{h\in H}\ell _S(bh)h\cdot \sum _{h\in H}\ell _S(bh)h^{-1}\right) }, \end{aligned}$$

(4.6)

$$\begin{aligned} x_2= & {} \chi \left( \sum _{h\in H}\ell _S(h)h\right) -\sqrt{\chi \left( \sum _{h\in H}\ell _S(bh)h\cdot \sum _{h\in H}\ell _S(bh)h^{-1}\right) }. \end{aligned}$$

(4.7)

If both $x_1$ and $x_2$ are integers, then by (4.6) and (4.7), $\chi \left( \sum _{h\in H}\ell _S(h)h\right) =\frac{x_1+x_2}{2}$ is a rational number. Note that $\chi \left( \sum _{h\in H}\ell _S(h)h\right)$ is an algebraic number, and $\chi \left( \sum _{h\in H}\ell _S(h)h\right)$ is thus forced to be an integer. Then

$$\begin{aligned} \chi \left[ \sum _{h\in H}\ell _S(bh)h\cdot \sum _{h\in H}\ell _S(bh)h^{-1}\right] \end{aligned}$$

is a square number from (4.6). Conversely, if $\chi \left( \sum _{h\in H}\ell _S(h)h\right)$ is an integer and

$$\begin{aligned} \chi \left[ \sum _{h\in H}\ell _S(bh)h\cdot \sum _{h\in H}\ell _S(bh)h^{-1}\right] \end{aligned}$$

is a square number, then both $x_1$ and $x_2$ are integers from (4.6) and (4.7).

Therefore, combining Lemma 2.4 and the arbitrariness of $\rho$ yields that $Cay({\mathscr {H}},S)$ is distance integral if and only if $\sum _{h\in H}\ell _S(h)h$ is in C(H) and $\chi \left[ \sum _{h\in H}\ell _S(bh)(h)\cdot \sum _{h\in H}\ell _S(bh)h^{-1}\right]$ is a square number for each irreducible representation of H except for the ones satisfying $\chi (h^2)=1$ for all $h\in H$. This completes the proof. $\square$

By Theorem 4.3, we can obtain infinite families of distance integral Cayley graphs over generalized dihedral groups in the following two corollaries.

Corollary 4.4

Let H be a finite abelian group, and let ${\mathscr {H}}$ be its generalized dihedral group. Let $S=S_1\cup bS_2\subseteq {\mathscr {H}}$ such that $1_H\notin S$ and $S^{-1}=S$, where $S_1,S_2\subseteq H$ with $|S_2|=1$. Then $Cay({\mathscr {H}},S)$ is distance integral if and only if $\sum _{h\in H}\ell _{S_1}(h)h\in C(H)$.

Proof

Assume that $S_2=\{x_0\}$, then $S=S_1\cup \{bx_0\}$. Fix an element $h\in H$, and assume that $\ell _S(bh)=r+1$ with $bh=bx_0x_1x_2\ldots x_r$, where $x_1,x_2,\ldots ,x_r\in S_1$. Then $x_0^{-1}h=x_1x_2\ldots x_r$. Thus, $\ell _{S_1}(x_0^{-1}h)\leqslant r=\ell _S(bh)-1$. Similarly, $\ell _S(bh)\leqslant \ell _{S_1}(x_0^{-1}h)+1$. Therefore, $\ell _S(bh)=\ell _{S_1}(x_0^{-1}h)+1$ for every $h\in H$. Consequently,

$$\begin{aligned} \sum _{h\in H}\ell _S(bh)h\cdot \sum _{h\in H}\ell _S(bh)h^{-1}= & {} \sum _{h\in H}\ell _S(bh)h\cdot \sum _{h\in H}\ell _S(bh^{-1})h\nonumber \\= & {} \sum _{h\in H}(\ell _{S_1}(x_0^{-1}h)+1)h\cdot \sum _{h\in H}(\ell _{S_1}(x_0^{-1}h^{-1})+1)h\nonumber \\= & {} \left( \sum _{h\in H}\ell _{S_1}(x_0^{-1}h)h+\sum _{h\in H}h\right) \left( \sum _{h\in H}\ell _{S_1}(x_0^{-1}h^{-1})h+\sum _{h\in H}h\right) .\nonumber \\ \end{aligned}$$

(4.8)

For any non-trivial irreducible representation $\chi$ of H, there exists $h'\in H$ such that $\chi (h')\ne 1$. Then

$$\begin{aligned} \chi (h')\chi \left( \sum _{h\in H}h\right) =\chi \left( \sum _{h\in H}hh'\right) =\chi \left( \sum _{h\in H}h\right) . \end{aligned}$$

Hence, $\chi \left( \sum _{h\in H}h\right) =0$. Expanding the right-hand side of (4.8) and after a short calculation, we have, for any non-trivial irreducible representation $\chi$ of H, that

$$\begin{aligned} \chi \left( \sum _{h\in H}\ell _S(bh)h\cdot \sum _{h\in H}\ell _S(bh)h^{-1}\right)= & {} \chi \left( \sum _{h\in H}\ell _{S_1}(x_0^{-1}h)h\cdot \sum _{h\in H}\ell _{S_1}(x_0^{-1}h^{-1})h\right) \nonumber \\= & {} \chi \left( \sum _{h\in H}\ell _{S_1}(h)h\cdot \sum _{h\in H}\ell _{S_1}(h)h^{-1}\right) \nonumber \\= & {} \chi \left( \sum _{h\in H}\ell _{S_1}(h)h\cdot \sum _{h\in H}\ell _{S_1}(h^{-1})h\right) \nonumber \\= & {} \left[ \chi \left( \sum _{h\in H}\ell _{S_1}(h)h\right) \right] ^2, \end{aligned}$$

(4.9)

where the last equality in (4.9) follows from the fact that $S_1=S_1^{-1}$. Then the desired result follows from Lemma 2.4 and Theorem 4.3. $\square$

Corollary 4.5

Let H be a finite abelian group, and let ${\mathscr {H}}$ be its generalized dihedral group. Denote by $S=S_1\cup bS_2\subseteq {\mathscr {H}}$ such that $1_H\notin S$ and $S_i^{-1}=S_i$ for $i=1,2$, where $S_1,S_2\subseteq H$. Then $Cay({\mathscr {H}},S)$ is distance integral if and only if both $\sum _{h\in H}\ell _S(h)h$ and $\sum _{h\in H}\ell _S(bh)h$ are in C(H).

Proof

First we show that $\ell _S(bh)=\ell _S(bh^{-1})$ for all $h\in H$. In fact, note that bh can be expressed as $bh=x_1x_2\ldots x_k$, where $x_i\in S_1$ or $x_i=bs_i\in bS_2$ for $1\leqslant i\leqslant k$. Then

$$\begin{aligned} bh^{-1}=b(bh)b=bx_1x_2\ldots x_kb=\left\{ \begin{array}{ll} (bx_1x_2\ldots x_{k-1}b)x_k^{-1}, &{}\quad {\text {if }}x_k\in S_1; \\ (bx_1x_2\ldots x_{k-1}b)bs_k^{-1}, &{}\quad {\text {if }}x_k=bs_k\in bS_2. \end{array} \right. \end{aligned}$$

Iterating the above argument yields

$$\begin{aligned} bh^{-1}=x_1'x_2'\ldots x_k',~ \hbox {where}~~x_i'=\left\{ \begin{array}{ll} x_i^{-1}, &{} \quad {\text {if }}x_i\in S_1; \\ bs_i^{-1}, &{}\quad {\text {if }}x_i=bs_i\in bS_2 \end{array} \right. ~ \hbox {for}~~i=1,2,\ldots ,k. \end{aligned}$$

We know from $S_i^{-1}=S_i$ for $i=1,2$ that $x_i'\in S$, and thus, $\ell _S(bh^{-1})\leqslant \ell _S(bh)$. Similarly, $\ell _S(bh)\leqslant \ell _S(bh^{-1})$. Therefore, $\ell _S(bh^{-1})=\ell _S(bh)$, which implies that

$$\begin{aligned} \sum _{h\in H}\ell _S(bh)h^{-1}=\sum _{h^{-1}\in H}\ell _S(bh^{-1})h=\sum _{h\in H}\ell _S(bh)h. \end{aligned}$$

The desired result then follows from Lemma 2.4 and Theorem 4.3. $\square$

The following corollary gives a necessary condition for the distance integrality of $Cay({\mathscr {H}},S)$.

Corollary 4.6

Let H be a finite abelian group, and let ${\mathscr {H}}$ be its generalized dihedral group. Denote by $S\subseteq {\mathscr {H}}$ such that $1_H\notin S=S^{-1}$ and $\langle S\rangle ={\mathscr {H}}$. If $Cay({\mathscr {H}},S)$ is distance integral, then both $\sum _{h\in H}\ell _S(h)h$ and $\sum _{h\in H}\ell _S(bh)h\cdot \sum _{h\in H}\ell _S(bh)h^{-1}$ are in C(H).

5 Relationships between integral and distance integral Cayley graphs over generalized dihedral groups

In this section, we focus on the relations between the integral Cayley graphs and the distance integral Cayley graphs over generalized dihedral groups. Here, we first show that the integrity and distance integrity of Cayley graphs over generalized dihedral groups can be equivalent under some special conditions.

Theorem 5.1

Let H be a finite abelian group, and let ${\mathscr {H}}$ be its generalized dihedral group. Let $S=S_1\cup bS_2\subseteq {\mathscr {H}}$ such that $1_H\notin S$ and $S^{-1}=S$, where $S_1,S_2\subseteq H$ with $|S_2|=1$. Then $Cay({\mathscr {H}},S)$ is integral if and only if $Cay({\mathscr {H}},S)$ is distance integral.

Proof

In view of Corollaries 3.4 and  4.4, it suffices for us to show that $S_1\in B(H)$ if and only if $\sum _{h\in H}\ell _{S_1}(h)h\in C(H)$. Note that $S_1=\{h\in H\,|\,\ell _{S_1}(h)=1\}$. The sufficiency is thus obvious.

Suppose conversely that ${\widetilde{B}}(H)=\{[g_1],[g_2],\ldots ,[g_k]\}$ for some integer k. Let $\langle h_1\rangle =\langle h_2\rangle \in {\widetilde{B}}(H)$ with $\ell _S(h_1)=q$ and $ord(h_1)=t$. Then $h_2=h_1^l$ for some integer l, which leads to $\gcd (l,t)=1$. Recall that H is of order n, then t is a divisor of n (abbreviated $t\,|\,n$). Thus, there exists a surjective group homomorphism $f:{\mathbb {Z}}_n^*\rightarrow {\mathbb {Z}}_t^*$ such that $f(x\pmod {n})=x\pmod {t}$, where ${\mathbb {Z}}_n^*=\{n'|\gcd (n',n)=1\}$. Recall that $l\in {\mathbb {Z}}_t^*$. Then there exists $y\in {\mathbb {Z}}_n^*$ such that $f(y\pmod {n})=l\pmod {t}=y\pmod {t}$. Therefore, $t\,|\,(l-y)$, which gives $h_2=h_1^l=h_1^y$. Note that $h_1$ can be expressed as $h_1=z_1z_2\ldots z_q$, where $z_i\in S_1$ for $1\leqslant i\leqslant q$. Then $h_2=z_1^yz_2^y\ldots z_q^y$. Recall that $\gcd (y,n)=1$, we thus have $\gcd (y,ord(z_i))=1$, leading to $z_i^y\in \langle z_i\rangle \in S_1$ for all $1\leqslant i\leqslant q$. Therefore, $\ell _{S_1}(h_2)\leqslant \ell _{S_1}(h_1)$. In a similar way, $\ell _{S_1}(h_1)\leqslant \ell _{S_1}(h_2)$. Consequently, $\ell _{S_1}(h_1)=\ell _{S_1}(h_2)$ whenever $\langle h_1\rangle =\langle h_2\rangle$, and therefore, we conclude that $\sum _{h\in H}\ell _{S_1}(h)h\in C(H)$ as desired. $\square$

Theorem 5.2

Let H be a finite abelian group, and let ${\mathscr {H}}$ be its generalized dihedral group. Denote by $S=S_1\cup bS_2\subseteq {\mathscr {H}}$ such that $1_H\notin S$ and $S_i^{-1}=S_i$ for $i=1,2$, where $S_1,S_2\subseteq H$. Then $Cay({\mathscr {H}},S)$ is integral if and only if $Cay({\mathscr {H}},S)$ is distance integral.

Proof

In view of Corollaries 3.5 and  4.5, it suffices to show that $S_1,S_2\in B(H)$ if and only if $\sum _{h\in H}\ell _S(h)h\in C(H)$ and $\sum _{h\in H}\ell _S(bh)h\in C(H)$. Note that $S_1=\{h\in H\,|\,\ell _S(h)=1\}$ and $S_2=\{h\in H\,|\,\ell _S(bh)=1\}$. The sufficiency is thus obvious.

Conversely, assume that ${\widetilde{B}}(H)=\{[g_1],[g_2],\ldots ,[g_w]\}$ for some integer w. Let $\langle h_1\rangle =\langle h_2\rangle \in {\widetilde{B}}(H)$ with $\ell _S(h_1)=r$. Just as we did in the proof of Theorem 5.1, there exists an integer k with $\gcd (k,n)=1$ such that $h_2=h_1^k$. Note that $h_1$ can be expressed as $h_1=x_1x_2\ldots x_r$, where $x_i\in S_1$ or $x_i=bs_i\in bS_2$ for $1\leqslant i\leqslant r$. Furthermore, assume that $x_i\in [g_j]\in {\widetilde{B}}(H)$ if $x_i\in S_1$ and $s_i\in [g_j]\in {\widetilde{B}}(H)$ if $x_i=bs_i\in bS_2$ for some $j\in [1,w]$. Moving all b contained in $h_1$ to the leftmost according to the same operation as in Corollary 4.5 yields

$$\begin{aligned} h_1=x_1x_2'\ldots x_r',~ \hbox {where}~~x_i'\in \left\{ \begin{array}{ll} \{x_i,x_i^{-1}\}, &{}\quad {\text {if }}x_i\in S_1; \\ \{s_i,s_i^{-1}\}, &{}\quad {\text {if }}x_i=bs_i\in bS_2 \end{array} \right. ~ \hbox {for}~~i=1,2,\ldots ,r. \end{aligned}$$

Then $h_2=h_1^k=(x_1')^k(x_2')^k\ldots (x_r')^k=x_1''x_2''\ldots x_r''$, where

$$\begin{aligned} x_i''\in \left\{ \begin{array}{ll} \{x_i^k,x_i^{-k}\}, &{}\quad {\text {if }}x_i\in S_1; \\ \{bs_i^k,bs_i^{-k}\}, &{}\quad {\text {if }}x_i=bs_i\in bS_2. \end{array} \right. \end{aligned}$$

Since $\gcd (k,n)=1$, we get $\gcd (k,ord(x_i))=1$ or $\gcd (k,ord(s_i))=1$. Then $\{x_i^k,x_i^{-k}\}\subseteq [g_j]\subseteq S_1$ or $\{s_i^k,s_i^{-k}\}\subseteq [g_j]\subseteq S_2$ for $1\leqslant i\leqslant r, 1\leqslant j\leqslant w$. Consequently, $\ell _S(h_2)\leqslant \ell _S(h_1)$. Similarly, $\ell _S(h_1)\leqslant \ell _S(h_2)$. Therefore, $\ell _S(h_1)=\ell _S(h_2)$ whenever $\langle h_1\rangle =\langle h_2\rangle$, implying that $\sum _{h\in H}\ell _S(h)h\in C(H)$. In a similar way, we have $\sum _{h\in H}\ell _S(bh)h\in C(H)$ as desired. This completes the proof. $\square$

It is worth noting that the conditions $|S_2|=1$ and $S_2=S_2^{-1}$ in Theorems 5.1-5.2 cannot be omitted in general. We give a counterexample in the following.

Example 5.3

Let $D_{21}=\langle a,b\,|\,a^{21}=b^2=1,bab=a^{-1}\rangle$ be the dihedral group of order 42. Put $S_1=\left\{ a^7,a^{14}\right\} ,S_2=\left\{ a^7,a^9,a^{14},a^{15},a^{18}\right\}$ and $S=S_1\cup bS_2$. On the one hand, note that

$$\begin{aligned} {\widetilde{B}}(\langle a\rangle )= & {} \left\{ \{1\},\{a,a^2,a^4,a^5,a^8,a^{10},a^{11}, a^{13},a^{16},a^{17},a^{19},a^{20}\}\right. ,\\&\left. \{a^3,a^6,a^9,a^{12},a^{15},a^{18}\},\{a^7,a^{14}\}\right\} \end{aligned}$$

and $\{a^7,a^9,a^{14},a^{15},a^{18}\}$ is a (21, 5, 1)-difference set of $\langle a\rangle$. By Corollary 3.6, $Cay(D_{21},S)$ is integral. However, by direct calculations, we have

$$\begin{aligned} \begin{array}{l} \ell _S\left( ba^7\right) =\ell _S\left( ba^9\right) =\ell _S\left( ba^{14}\right) =\ell _S\left( ba^{15}\right) =\ell _S\left( ba^{18}\right) =1,\\ \ell _S(b)=\ell _S(ba)=\ell _S\left( ba^2\right) =\ell _S\left( ba^4\right) =\ell _S\left( ba^8\right) =\ell _S\left( ba^{11}\right) =\ell _S\left( ba^{16}\right) =2,\\ \ell _S\left( ba^3\right) =\ell _S\left( ba^5\right) =\ell _S\left( ba^6\right) =\ell _S\left( ba^{10}\right) =\ell _S\left( ba^{12}\right) =\ell _S\left( ba^{13}\right) =\ell _S\left( ba^{17}\right) \\ =\ell _S\left( ba^{19}\right) =\ell _S\left( ba^{20}\right) =3. \end{array} \end{aligned}$$

Therefore, $\chi _3\left( \sum _{k=0}^{11}\ell \left( ba^k\right) a^k\cdot \sum _{k=0}^{11}\ell _S\left( ba^k\right) a^{-k}\right) =37$ is not a square number. Consequently, $Cay(D_{21},S)$ is not distance integral by Theorem 4.3.

Although the conclusions of Theorems 5.1 and 5.2 may fail in general if we remove the assumptions $|S_2|=1$ and $S_2=S_2^{-1}$, we can still find a special kind of Cayley graphs over dihedral groups, which satisfy the integrality implies distance integrality and vice versa.

In the following, we discuss the distance integrality of Cayley graphs over dihedral groups $D_p$, where $p\geqslant 3$ is a prime. It is inspired by Lu et al. [17], who have completely determined all integral Cayley graphs over the dihedral group $D_p$. It turns out that $Cay(D_p,S)$ is integral if and only if it is distance integral.

We first establish the following result.

Theorem 5.4

Let $D_p=\langle a,b\,|\,a^p=b^2=1,bab=a^{-1}\rangle$ with $p\geqslant 3$ being a prime. Let $S=S_1\cup bS_2\subseteq D_p$ such that $1\notin S=S^{-1}$ and $\langle S\rangle =G$, where $S_1,S_2\subseteq \langle a\rangle$. Then $Cay(D_p,S)$ is distance integral if and only if either $S_1=\emptyset$ and $S_2\in \left\{ \langle a\rangle {\setminus }\{a^k\},\langle a\rangle \right\}$ for $1\leqslant k\leqslant p-1$ or $S_1=\langle a\rangle {\setminus }\{1\}$ and $S_2\in \left\{ \{a^k\},\langle a\rangle {\setminus }\{a^k\},\langle a\rangle \right\}$ for $0\leqslant k\leqslant p-1$.

Proof

Recall that $\chi _i\left( a^k\right) =\omega ^{ki}$ are irreducible representations of $\langle a\rangle$ for $0\leqslant i\leqslant p-1$, where $\omega$ is a primitive $p\hbox {th}$ root of unity in the complex field. By Theorem 4.3, we know that $Cay(D_p,S)$ is distance integral if and only if $\sum _{k=1}^{p-1}\ell _S\left( a^k\right) \omega ^{ki}$ is an integer and

$$\begin{aligned} \sum _{k=0}^{p-1}\ell _S\left( ba^k\right) \omega ^{ki}\cdot \sum _{k=0}^{p-1}\ell _S\left( ba^k\right) \omega ^{-ki} \end{aligned}$$

is a square number for all $1\leqslant i\leqslant \frac{p-1}{2}$.

Now assume that $S_1$ and $S_2$ satisfy the sufficient conditions of our result, and we aim to show that $Cay(D_p,S)$ is distance integral. If $S_1=\emptyset$ and $S_2=\langle a\rangle$, then $\ell _S\left( a^j\right) =2$ for $1\leqslant j\leqslant p-1$, $\ell _S\left( ba^{j'}\right) =1$ for $0\leqslant j'\leqslant p-1$, and thus,

$$\begin{aligned} \sum _{k=1}^{p-1}\ell _S\left( a^k\right) \omega ^{ki}=2\sum _{k=1}^{p-1}\omega ^{ki}=-2,\ \ \sum _{k=0}^{p-1}\ell _S\left( ba^k\right) \omega ^{ki}\cdot \sum _{k=0}^{p-1}\ell _S\left( ba^k\right) \omega ^{-ki}=0. \end{aligned}$$

If $S_1=\emptyset$ and $S_2=\langle a\rangle {\setminus }\left\{ a^k\right\}$ for some $1\leqslant k\leqslant p-1$, then it is routine to check that $\ell _S\left( a^j\right) =2$ for $1\leqslant j\leqslant p-1$, $\ell _S\left( ba^{j'}\right) =1$ for $~j'\ne k$ and $\ell _S\left( ba^k\right) =3$, which leads to

$$\begin{aligned}&\sum _{k=1}^{p-1}\ell _S\left( a^k\right) \omega ^{ki}=2\sum _{k=1}^{p-1}\omega ^{ki}\\&\quad =-2,\ \ \sum _{k=0}^{p-1}\ell _S\left( ba^k\right) \omega ^{ki}\cdot \sum _{k=0}^{p-1}\ell _S\left( ba^k\right) \omega ^{-ki}=2\omega ^k\cdot 2\omega ^{-k}=4. \end{aligned}$$

In a similar way, if $S_1=\langle a\rangle {\setminus }\{1\}$ and $S_2\in \left\{ \{a^k\},\langle a\rangle {\setminus }\{a^k\},\langle a\rangle \right\}$ for $0\leqslant k\leqslant p-1$, then $\sum _{k=1}^{p-1}\ell _S\left( a^k\right) \omega ^{ki}=\sum _{k=1}^{p-1}\omega ^{ki}=-1$ is an integer and $\sum _{k=0}^{p-1}\ell _S\left( ba^k\right) \omega ^{ki}\cdot \sum _{k=0}^{p-1}\ell _S\left( ba^k\right) \omega ^{-ki}\in \{0,1\}$ is a square number.

Assume conversely that $Cay(D_p,S)$ is distance integral. Note that ${\widetilde{B}}\left( \langle a\rangle \right) =\left\{ \{1\},\{a,a^2,\ldots ,a^{p-1}\}\right\}$ by the fact that p is a prime number. It follows from Corollary 4.6 that

$$\begin{aligned} \sum _{k=1}^{p-1}\ell _S(a^k)a^k\in C\left( \langle a\rangle \right) ,\ \ \sum _{k=0}^{p-1}\ell _S\left( ba^k\right) a^{k}\cdot \sum _{k=0}^{p-1}\ell _S\left( ba^k\right) a^{-k}\in C\left( \langle a\rangle \right) . \end{aligned}$$

Then we have

$$\begin{aligned} \ell _S(a)=\ell _S\left( a^2\right) =\cdots =\ell _S\left( a^{p-1}\right) \end{aligned}$$

(5.1)

and

$$\begin{aligned}&\left[ \sum _{k=0}^{p-1}\ell _S\left( ba^k\right) a^{k}\right] \cdot \left[ \sum _{k=0}^{p-1}\ell _S\left( ba^k\right) a^{-k}\right] \nonumber \\&\quad =\sum _{k=0}^{p-1}\left[ \ell _S\left( ba^k\right) \right] ^2+m\left( a+a^2+\cdots +a^{p-1}\right) , \end{aligned}$$

(5.2)

where m is a nonnegative integer. Taking the trivial representation of $\langle a\rangle$ through (5.2) yields

$$\begin{aligned} \left[ \sum _{k=0}^{p-1}\ell _S\left( ba^k\right) \right] ^2=\sum _{k=0}^{p-1}\left[ \ell _S\left( ba^k\right) \right] ^2+(p-1)m. \end{aligned}$$

(5.3)

Recall that $\chi _i\left[ \sum _{k=0}^{p-1}\ell _S\left( ba^k\right) a^{k}\cdot \sum _{k=0}^{p-1}\ell _S\left( ba^k\right) a^{-k}\right]$ is a square number, then there exists an integer t such that

$$\begin{aligned} t^2=\sum _{k=0}^{p-1}\left[ \ell _S\left( ba^k\right) \right] ^2-m. \end{aligned}$$

(5.4)

Further on, we proceed by distinguishing the following two cases to show our result.

Case 1. $|S_1|\geqslant 1.$ In this case, assume that $a^k\in S_1$ for some k. Then $\ell _S(a^k)=1$. In view of (5.1), one has $\ell _S(a)=\ell _S(a^2)=\cdots =\ell _S(a^{p-1})=1$, which implies that $S_1=\langle a\rangle {\setminus }\{1\}.$ Thus, we get

$$\begin{aligned} \ell _S\left( ba^{k'}\right) =\left\{ \begin{array}{ll} 1, &{}\quad {\text {if }}a^{k'}\in S_2; \\ 2, &{}\quad {\text {if }}a^{k'}\notin S_2. \end{array} \right. \end{aligned}$$

(5.5)

Assume that $|S_2|=x$, then $1\leqslant x\leqslant p$ (based on the fact that $\langle S\rangle =G$). Substituting (5.5) into (5.3) and (5.4) yields

$$\begin{aligned}{}[x+2(p-x)]^2=x+4(p-x)+(p-1)m,\ \ t^2=x+4(p-x)-m. \end{aligned}$$

Then we obtain $p(x-t^2)=(x+t)(x-t).$ Therefore, $p\,|\,(x+t)$ or $p\,|\,(x-t)$. Note that $t^2\leqslant 4p-3x\leqslant p^2$, we have $x+t,x-t\in \{0,p,2p\}.$ If $x+t=0$, then $x=t^2=x^2$. Thus, we have $x=1$. Similarly, all the possible cases lead to $x\in \{1,p-1,p\}$, which implies that $S_2\in \left\{ \{a^k\},\langle a\rangle {\setminus }\{a^k\},\langle a\rangle \right\}$ for $0\leqslant k\leqslant p-1$, as desired.

Case 2. $|S_1|=0.$ In this case, as $\langle S\rangle =G$, we have $|S_2|\geqslant 2$. Assume that $a^k,a^j\in S_2$ for some $0\leqslant k<j\leqslant p-1.$ Then $\ell _S(a^{j-k})=\ell _S(ba^k\cdot ba^j)=2.$ In view of (5.1), one has $\ell _S(a)=\ell _S(a^2)=\cdots =\ell _S(a^{p-1})=2$, which implies that

$$\begin{aligned} \ell _S\left( ba^{j'}\right) =\left\{ \begin{array}{ll} 1, &{}\quad {\text {if }}a^{j'}\in S_2; \\ 3, &{}\quad {\text {if }}a^{j'}\notin S_2. \end{array} \right. \end{aligned}$$

(5.6)

Assume that $|S_2|=y$, then $2\leqslant y\leqslant p$. Substituting (5.6) into (5.3) and (5.4) yields

$$\begin{aligned}{}[y+3(p-y)]^2=y+9(p-y)+(p-1)m,\ \ t^2=y+9(p-y)-m. \end{aligned}$$

Then we obtain $p(4y-t^2)=(2y+t)(2y-t).$ Therefore, $p\,|\,(2y+t)$ or $p\,|\,(2y-t)$. Note that $t^2\leqslant 9p-8y\leqslant (p+1)^2$, we have $2y+t,2y-t\in \{0,p,2p,3p\}.$ If $2y+t=0$, then $4y=t^2=4y^2$, which is impossible since $y\geqslant 2$. Similarly, all the possible cases lead to $y\in \{p-1,p\}$, implying that $S_2\in \left\{ \langle a\rangle {\setminus }\{a^k\},\langle a\rangle \right\}$ for $1\leqslant k\leqslant p-1$. We are done. $\square$

In order to compare the conditions for the integrality and distance integrality of $Cay(D_p,S)$, we restate [17, Theorem 4.2] below in our notations for convenience.

Lemma 5.5

[17] Let $D_p=\langle a,b\,|\,a^p=b^2=1,bab=a^{-1}\rangle$ with $p\geqslant 3$ being a prime. Let $S=S_1\cup bS_2\subseteq D_p$ such that $1\notin S$ and $S^{-1}=S$, where $S_1,S_2\subseteq \langle a\rangle$. Then $Cay(D_p,S)$ is integral if and only if $S_1\in \left\{ \emptyset ,\langle a\rangle {\setminus }\{1\}\right\}$ and $S_2\in \left\{ \{a^k\},\langle a\rangle {\setminus }\{a^k\},\langle a\rangle \right\}$ for $0\leqslant k\leqslant p-1$.

Comparing Theorem 5.4 with Lemma 5.5, we immediately arrive at the following result.

Corollary 5.6

Let $D_p=\langle a,b\,|\,a^p=b^2=1,bab=a^{-1}\rangle$ with $p\geqslant 3$ being a prime. Denote by $S=S_1\cup bS_2\subseteq D_p$ such that $1\notin S=S^{-1}$ and $\langle S\rangle =G$, where $S_1,S_2\subseteq \langle a\rangle$. Then $Cay(D_p,S)$ is integral if and only if $Cay(D_p,S)$ is distance integral.