1 Introduction

The \({\mathcal {Z}}_2 \times {\mathcal {Z}}_N\) flavour symmetry [1] provides a new framework for the celebrated Froggatt–Nielsen (FN) mechanism that eventually furnishes an elegant solution to the flavour problem of the standard model (SM) [2]. The flavour problem of the SM comprises a set of fundamental questions, including the origin of the mass pattern of fermions of the SM, an explanation for the observed quark-mixing, and the source of neutrino masses and oscillations. There are various approaches to address this problem in literature. For instance, it can have a solution through the hierarchy of vacuum-expectation-values (VEVs) in a technicolour-framework where VEVs are sequential chiral condensates of an extended dark-technicolour sector [3,4,5]. A possible explanation can be obtained using Abelian flavor symmetries [2, 6,7,8,9,10,11,12,13], creating loop-suppressed couplings to the Higgs [14], through wave-function localization [15,16,17,18,19] or via compositeness [20].

The central idea of the FN mechanism is based on an Abelian flavour symmetry \(U(1)_F\), which can distinguish different flavours of fermions among and within the fermionic generations in the SM. This is achieved by introducing a flavon field \(\chi \) in such a way that only top quark gets mass from a renormalized SM interaction, and masses for other fermions are obtained from appropriate non-renormalized higher dimensional operators, which are constructed using the flavon field \(\chi \). For instance, if under the \(U(1)_F\) symmetry the fermions \(\psi _i^c\) and \(\psi _j\) have charges \(\theta _i\) and \(\theta _j\) respectively and the charge of the SM Higgs field is zero then the Yukawa Lagrangian of the SM is forbidden by the \(U(1)_F\) symmetry. In this scenario, the masses of the SM fermions can be recovered by the non-renormalizable operators of the form,

$$\begin{aligned} {\mathcal {O}}= & {} y \left( \dfrac{\chi }{\Lambda }\right) ^{(\theta _i + \theta _j)} {\bar{\psi }} \varphi \psi , \nonumber \\= & {} y \epsilon ^{(\theta _i + \theta _j)} {\bar{\psi }} \varphi \psi = Y {\bar{\psi }} \varphi \psi \end{aligned}$$
(1)

where y is the dimensionless coupling constant, \(\Lambda \) is the scale at which these operators are renormalized, \(\epsilon = \dfrac{\langle \chi \rangle }{\Lambda }\), \(Y= y \epsilon ^{(\theta _i + \theta _j)}\) is the effective Yukawa coupling, and the gauge singlet flavon scalar field \(\chi \) transforms under the \(SU(3)_c \times SU(2)_L \times U(1)_Y\) symmetry of the SM as,

$$\begin{aligned} \chi :(1,1,0). \end{aligned}$$
(2)

The \(U(1)_F\) flavour symmetry is broken spontaneously when the flavon field \(\chi \) acquires a VEV. The scale \(\Lambda \) is not provided by the theory, and it can be anywhere between the weak and the Planck scale. We only require that the flavour symmetry should be broken weakly which means the ratio \( \dfrac{\langle \chi \rangle }{ \Lambda } \) should be less than unity. The flavon exchange effects in the SM phenomenology will be highly suppressed if the scale of new physics \(\Lambda \) is much larger than the weak scale. However, if the flavour symmetry is broken close to the weak scale, we can hope to see observable effects on the direct or indirect experimentally measured SM observables such as mixing and CP-violation in mesons. Therefore, we need to ask how low the flavour scale could be such that it respects the bounds on flavour-changing and CP-violating processes. Moreover, the nature of the flavour symmetry also plays an important role in the investigation of the flavour scale. For example, if the flavour symmetry is a continuous \(U(1)_F\), then we should ask whether it is a gauged or a global symmetry. In a gauged \(U(1)_F\) scenario, the phenomenology of the flavon field will be affected by the exchange of the corresponding gauge boson. If the continuous \(U(1)_F\) is global, then a massless Goldstone boson must exist.

The \({\mathcal {Z}}_2 \times {\mathcal {Z}}_N\) flavour symmetry, unlike the conventional continuous U(1) flavour symmetry that is employed to achieve the FN mechanism, is a product of two discrete symmetries which can implement the FN mechanism in a unique way such that the flavour structure of the SM including neutrino masses and mixing parameters can be parametrized in terms of a small parameter which is the ratio of the VEV of the flavon field and the flavour scale \(\Lambda \) [1]. We notice that the origin of the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_N\) flavour symmetry may be traced to an underlying Abelian or non-Abelian continuous symmetry or their products. For instance, the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_N\) symmetry may be a by-product of a spontaneous breaking of \(U (1) \times U (1)\) continuous product symmetry.

We note that the discrete \({\mathcal {Z}}_2\) symmetry is extensively used in studying the different versions of the two-Higgs-doublet model (2HDM) and the minimal supersymmetric SM (MSSM). In particular, in the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_N\) flavour symmetry, the discrete \({\mathcal {Z}}_2\) symmetry exactly behaves like the one used in the type-II 2HDM [21]. Therefore, the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_N\) flavour symmetry may also be used to implement the FN mechanism in the type-II 2HDM and the MSSM. Moreover, the discrete \({\mathcal {Z}}_2\) symmetry is also found to be useful in model building, for instance, see Refs. [3, 5, 22,23,24,25].

In this work, we investigate flavour bounds on the dynamics of the flavon field of a minimal and a non-minimal form of the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_N\) flavour symmetry that provides a simple set-up for the FN mechanism. We do not consider any ultraviolet completion of the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_N\) based FN mechanism, and present our results in a model-independent manner. The phenomenological investigations of the flavon field of the FN mechanism in the framework of a continuous U(1) symmetry and its extensions are dedicatedly performed in literature, for instance, flavour bounds are investigated in Ref. [26], the LHC phenomenology is explored in Refs. [27,28,29,30,31,32,33,34], a low flavour breaking scale is studied in Ref. [35], a study for a future high energy collider is presented in Ref. [36], flavon exchange effects in the dark matter interactions are studied in Ref. [37], the texture based investigation of the FN mechanism can be found in Ref. [38].

We shall present our phenomenological analysis along the following line: In Sect. 2, we investigate a minimal form of the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_N\) flavour symmetry that can implement the FN mechanism. An explanation to neutrino masses and mixing parameters is discussed in Sect. 2.4.1. A non-minimal form of the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_N\) flavour symmetry that implements the FN mechanism is discussed in Sect. 3. The scalar potential of our model is discussed in Sect. 4. Phenomenological bounds based on the quark flavour physics on the parameter space of a minimal and a non-minimal form of the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_N\) flavour symmetry are derived in Sect. 5. Leptonic flavour constraints are investigated in Sect. 6. A summary of the work is presented in Sect. 7.

2 A minimal \({\mathcal {Z}}_2 \times {\mathcal {Z}}_N\) flavour symmetry

We now discuss the question of a minimal form of the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_N\) flavour symmetry that can provide a simple set-up of the FN mechanism. Our guiding principle for this purpose is the observation that a minimal suppression of the effective Yukawa couplings will require a minimal form of the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_N\) flavour symmetry. For instance, we assume that the mass of the top quark originates from the tree level SM Yukawa operator, then, following the principle of minimum suppression (PMS), the mass of the bottom quark is obtained from the operator having the suppression of the order \(y \epsilon \), the mass of the charm quark from the operator having the suppression of the order \(y \epsilon ^2\), the mass of the strange quark from the operator having the suppression of the order \(y \epsilon ^3\), and the mass of the up and down quarks from the operators having at least the suppression of the order \(y \epsilon ^4\).

Additionally, we need to count the number of hierarchical energy scales needed to account for the fermionic mass hierarchy in the SM. For instance, for the quark sector, we need three energy scales to explain the mass hierarchy among the three fermionic families. We note that only the second and the third quark families have intra-generational mass hierarchies, which require only two hierarchical energy scales to achieve an explanation for the mass hierarchy within the second and third quark families. These hierarchical energy scales are created by the different non-renormalizable operators of the flavon fields as given in Eq. 1. Since the mass of the top quark is generated by the renormalized SM Yukawa operators, at least four energy scales are required to be created through the operators of the form given in Eq. 1 for providing an explanation for the hierarchical quark mass pattern.

The symmetry \( {\mathcal {Z}}_N\) in the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_N\) flavour symmetry is responsible for providing such operators. The symmetry \( {\mathcal {Z}}_2\) distinguishes between the up-type and the down-type quarks, which makes sure that identical non-renormalizable operators of the flavon fields, as given in Eq. 1, do not appear in the up and down-type quark mass matrices. For creating four energy scales, the required \( {\mathcal {Z}}_N\) symmetry, therefore, should have at least four non-trivial charges. Therefore, the size of a minimal symmetry will be determined by this requirement and through the application of the PMS.

Finally, we must note that a minimal form of the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_N\) flavour symmetry should not only produce correct pattern of the charged fermion masses, it should be capable of explaining the quark mixing pattern, neutrino masses, and more importantly, it should predict correct pattern of the neutrino mixing angles.

After taking into account above considerations, \({\mathcal {Z}}_2 \times {\mathcal {Z}}_{N}\) flavour symmetry allows us to write the following generic Lagrangian which provides masses to the charged fermions of the SM,

$$\begin{aligned} -{{\mathcal {L}}}_{\textrm{Yukawa}}= & {} \left[ \dfrac{ \chi (\chi ^{\dagger })}{\Lambda } \right] ^{n_{ij}^u} y_{ij}^u {\bar{\psi }}_{L_i}^q {\tilde{\varphi }} \psi _{R_j}^{u} \nonumber \\{} & {} + \left[ \dfrac{ \chi (\chi ^{\dagger })}{\Lambda } \right] ^{n_{ij}^d} y_{ij}^d {\bar{\psi }}_{L_i}^q \varphi \psi _{R_j}^{d} \nonumber \\{} & {} + \left[ \dfrac{ \chi (\chi ^{\dagger })}{\Lambda } \right] ^{n_{ij}^\ell } y_{ij}^\ell {\bar{\psi }}_{L_i}^\ell \varphi \psi _{R_j}^{\ell } + {\mathrm{H.c.}}, \nonumber \\= & {} Y^u_{ij} {\bar{\psi }}_{L_i}^q {\tilde{\varphi }} \psi _{R_j}^{u} + Y^d_{ij} \bar{ \psi }_{L_i}^q \varphi \psi _{R_j}^{d} \nonumber \\{} & {} + Y^\ell _{ij} \bar{ \psi }_{L_i}^\ell \varphi \psi _{R_j}^{\ell } + \text {H.c.}, \end{aligned}$$
(3)

where \(\chi \) or \(\chi ^\dagger \) may appear in the numerator of the term inside the square brackets. We note that in the above Lagrangian i and j are family indices, \( \psi _{L}^q, \psi _{L}^\ell \) are quark and leptonic doublets, \( \psi _{R}^u, \psi _{R}^d, \psi _{R}^\ell \) are right-handed up, down type singlet quarks and leptons, \(\varphi \) and \( {\tilde{\varphi }}= -i \sigma _2 \varphi ^* \) are the SM Higgs field and its conjugate and \(\sigma _2\) is the second Pauli matrix. The effective Yukawa couplings \(Y_{ij}\) are defined in terms of the expansion parameter \( \dfrac{\langle \chi \rangle }{ \Lambda } = \dfrac{f}{\sqrt{2} \Lambda }= \epsilon \ll 1\) such that \(Y_{ij} = y_{ij} \epsilon ^{n_{ij}}\).

2.1 \({\mathcal {Z}}_2 \times {\mathcal {Z}}_2\) flavour symmetry

The simplest choice is the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_2\) flavour symmetry, which turns out to be a trivial selection since the only charges of the \( {\mathcal {Z}}_2\) symmetry are \(\pm 1\), which are too trivial to provide four energy scales or equivalently non-trivial operators of the form given in Eq. 1. Hence, we conclude that this symmetry cannot be used to create a simple FN mechanism.

2.2 \({\mathcal {Z}}_2 \times {\mathcal {Z}}_3\) flavour symmetry

The first non-trivial form of the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_N\) flavour symmetry, which may provide an implementation of the FN mechanism, is \({\mathcal {Z}}_2 \times {\mathcal {Z}}_3\). The symmetry \( {\mathcal {Z}}_3\) has two non-trivial charges characterized by \(\omega \) and \(\omega ^2\), where \(\omega \) is the cube root of unity. In the first scenario, following the PMS, we assign the charges to the SM and flavon fields as given in Table 1.

Table 1 The charges of left and right-handed fermions of three families of the SM, Higgs and the flavon field under the \({\mathcal {Z}}_2\) and \({\mathcal {Z}}_3\) product symmetry, where \(\omega \) is the cube root of unity

We observe that the masses of s and b quarks can be recovered from this charge assignment, for instance, mass of the s quark is of the order \( \epsilon ^3\), and that of the b quark is of the order \( \epsilon \). However, the mass of the u and d quarks are produced by the operators \(y (\dfrac{\chi ^\dagger }{\Lambda })^{2} {\bar{\psi }} \varphi u_R\) and \(y (\dfrac{\chi ^\dagger }{\Lambda }) {\bar{\psi }} \varphi d_R\) instead of the operators \(y (\dfrac{\chi }{\Lambda })^{4} {\bar{\psi }} \varphi u_R\) and \(y (\dfrac{\chi }{\Lambda })^{5} {\bar{\psi }} \varphi d_R\). Any other charge assignment also does not reproduce masses of every quark.

As an additional check, we may assume that exactly identical diagonal operators for the u and d quarks in their mass matrices as given in Table 2. This charge assignment is against the original sprite of the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_N\) flavour symmetry, where the \({\mathcal {Z}}_2\) is exactly like the symmetry used in the type II 2HDM. It turns out that even in this case, the mass of the u quark is of the order \( \epsilon \). This conclusion does not change even if we provide different non-trivial charge assignments to the fermions and flavon fields under the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_3\) flavour symmetry.

Table 2 The charges of left and right-handed fermions of three families of the SM, Higgs and the flavon fields under the \({\mathcal {Z}}_2\) and \({\mathcal {Z}}_3\) product symmetry, where \(\omega \) is the cube root of unity

2.3 \({\mathcal {Z}}_2 \times {\mathcal {Z}}_4\) flavour symmetry

The charges of the symmetry \({\mathcal {Z}}_2 \times {\mathcal {Z}}_4\) are characterized by the fourth roots of unity, which are \(1, \omega , \omega ^2 \) and \(\omega ^3\) where \(\omega = i\), \(\omega ^3 = \omega ^*\) and \(\omega ^2 =-1\). We particularly note that minimally suppressed diagonal operator of the form \(y (\dfrac{\chi }{\Lambda })^{4} {\bar{\psi }}_L \varphi u_R\) is not the dominant operator no matter what charge we assign to the flavon and fermionic fields. This is because the tree-level SM Yukawa operator \(y {\bar{\psi }}_L \varphi u_R\) is allowed for any charge assignment for the \(y (\dfrac{\chi }{\Lambda })^{4} {\bar{\psi }}_L \varphi u_R\) operator under the \( {\mathcal {Z}}_2 \times {\mathcal {Z}}_4\) flavour symmetry.

Therefore, to produce the mass of the u-quark, we either choose a non-trivial transformation of the \(u_R\)-quark or the first family of the quarks under the \( {\mathcal {Z}}_2\) symmetry, in addition to the next to the minimal suppressed operator of the order \(\epsilon ^5\). One such charge assignment is given in Table 3. In this case, the masses of the down-type quarks are produced correctly through the operators with minimal suppression. However, in the case of up-type quarks, still non-diagonal tree-level SM operators dominate the mass of the u-quark. Other alternative charge assignments also do not work for creating an FN mechanism through the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_4\) symmetry.

2.4 \({\mathcal {Z}}_2 \times {\mathcal {Z}}_5\) flavour symmetry

We now impose the next flavour symmetry, that is, the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_5\) symmetry on the SM in a way that the various fields of the SM transform under this symmetry as given in Table 4 [1]. As discussed earlier, we need to create at least four hierarchical energy scales as an origin of the quark mass spectrum. This means, for creating these energy scales, a non-trivial and a minimal \( {\mathcal {Z}}_N\) symmetry should have at least four non-trivial charges. Thus, the symmetry \( {\mathcal {Z}}_5\) could be such a symmetry. Moreover, we note that the transformation of fields under the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_5\) symmetry is chosen such that the symmetry \({\mathcal {Z}}_2\) exactly acts like the way used in the type-II 2HDM.Footnote 1

The \({\mathcal {Z}}_2 \times {\mathcal {Z}}_5\) flavour symmetry allows us to write the following Lagrangian which provides masses to the charged fermions of the SM,

Table 3 The charges of left and right-handed fermions of three families of the SM, Higgs and the flavon fields under the \({\mathcal {Z}}_2\) and \({\mathcal {Z}}_4\) product symmetry, where \(\omega \) is the fourth root of unity
Table 4 The charges of left and right-handed fermions of three families of the SM, right-handed neutrinos, Higgs, and singlet scalar fields under \({\mathcal {Z}}_2\) and \({\mathcal {Z}}_5\) symmetries, where \(\omega \) is the fifth root of unity
$$\begin{aligned} -{{\mathcal {L}}}_{\textrm{Yukawa}}= & {} \left( \dfrac{ \chi }{\Lambda } \right) ^{4} y_{11}^u {\bar{\psi }}_{L_1}^q {\tilde{\varphi }} u_{R}+ \left( \dfrac{ \chi }{\Lambda } \right) ^{4} y_{12}^u {\bar{\psi }}_{L_1}^q {\tilde{\varphi }} c_{R} \\{} & {} + \left( \dfrac{ \chi }{\Lambda } \right) ^{4} y_{13}^u {\bar{\psi }}_{L_1}^q {\tilde{\varphi }} t_{R} + \left( \dfrac{ \chi }{\Lambda } \right) ^{2} y_{21}^u {\bar{\psi }}_{L_2}^q {\tilde{\varphi }} u_{R} \\{} & {} + \left( \dfrac{ \chi }{\Lambda } \right) ^{2} y_{22}^u {\bar{\psi }}_{L_2}^q {\tilde{\varphi }} c_{R} + \left( \dfrac{ \chi }{\Lambda } \right) ^{2} y_{23}^u {\bar{\psi }}_{L_2}^q {\tilde{\varphi }} t_{R}\\{} & {} + y_{31}^u {\bar{\psi }}_{L_3}^q {\tilde{\varphi }} u_{R} + y_{32}^u {\bar{\psi }}_{L_3}^q {\tilde{\varphi }} c_{R} + y_{33}^u {\bar{\psi }}_{L_3}^q {\tilde{\varphi }} t_{R} \\{} & {} +\left( \dfrac{ \chi }{\Lambda } \right) ^{5} y_{11}^d {\bar{\psi }}_{L_1}^q \varphi d_{R} + \left( \dfrac{ \chi }{\Lambda } \right) ^{5} y_{12}^d {\bar{\psi }}_{L_1}^q \varphi s_{R} \\{} & {} + \left( \dfrac{ \chi }{\Lambda } \right) ^{5} y_{13}^d {\bar{\psi }}_{L_1}^q \varphi b_{R} + \left( \dfrac{ \chi }{\Lambda } \right) ^{3} y_{21}^d {\bar{\psi }}_{L_2}^q \varphi d_{R} \\{} & {} + \left( \dfrac{ \chi }{\Lambda } \right) ^{3} y_{22}^d {\bar{\psi }}_{L_2}^q \varphi s_{R} + \left( \dfrac{ \chi }{\Lambda } \right) ^{3} y_{23}^d {\bar{\psi }}_{L_2}^q \varphi b_{R} \\{} & {} + \left( \dfrac{ \chi }{\Lambda } \right) y_{31}^d {\bar{\psi }}_{L_3}^q \varphi d_{R}+ \left( \dfrac{ \chi }{\Lambda } \right) y_{32}^d {\bar{\psi }}_{L_3}^q \varphi s_{R} \\{} & {} + \left( \dfrac{ \chi }{\Lambda } \right) y_{33}^d \bar{ \psi }_{L_3}^q \varphi b_{R} + \left( \dfrac{ \chi }{\Lambda } \right) ^{5} y_{11}^\ell {\bar{\psi }}_{L_1}^\ell \varphi e_{R} \\{} & {} + \left( \dfrac{ \chi }{\Lambda } \right) ^{5} y_{12}^\ell {\bar{\psi }}_{L_1}^\ell \varphi \mu _{R} + \left( \dfrac{ \chi }{\Lambda } \right) ^{5} y_{13}^\ell {\bar{\psi }}_{L_1}^\ell \varphi \tau _{R} \\{} & {} + \left( \dfrac{ \chi }{\Lambda } \right) ^{3} y_{21}^\ell {\bar{\psi }}_{L_2}^\ell \varphi e_{R} + \left( \dfrac{ \chi }{\Lambda } \right) ^{3} y_{22}^\ell {\bar{\psi }}_{L_2}^\ell \varphi \mu _{R} \\{} & {} + \left( \dfrac{ \chi }{\Lambda } \right) ^{3} y_{23}^\ell {\bar{\psi }}_{L_2}^\ell \varphi \tau _{R} + \left( \dfrac{ \chi }{\Lambda } \right) y_{31}^\ell {\bar{\psi }}_{L_3}^\ell \varphi e_{R} \\{} & {} + \left( \dfrac{ \chi }{\Lambda } \right) y_{32}^\ell \bar{ \psi }_{L_3}^\ell \varphi \mu _{R} + \left( \dfrac{ \chi }{\Lambda } \right) y_{33}^\ell {\bar{\psi }}_{L_3}^\ell \varphi \tau _{R} + \text {H.c.} \end{aligned}$$

The mass matrices for up- and down-type quarks and charged leptons can be written now in terms of the expansion parameter \(\epsilon \),

$$\begin{aligned} \mathcal{M}_u= & {} \dfrac{v}{\sqrt{2}} \begin{pmatrix} y_{11}^u \epsilon ^4 &{} y_{12}^u \epsilon ^4 &{} y_{13}^u \epsilon ^4 \\ y_{21}^u \epsilon ^2 &{} y_{22}^u \epsilon ^2 &{} y_{23}^u \epsilon ^2 \\ y_{31}^u &{} y_{32}^u &{} y_{33}^u \end{pmatrix},\nonumber \\ \mathcal{M}_d= & {} \dfrac{v}{\sqrt{2}} \begin{pmatrix} y_{11}^d \epsilon ^5 &{} y_{12}^d \epsilon ^5 &{} y_{13}^d \epsilon ^5 \\ y_{21}^d \epsilon ^3 &{} y_{22}^d \epsilon ^3 &{} y_{23}^d \epsilon ^3 \\ y_{31}^d \epsilon &{} y_{32}^d \epsilon &{} y_{33}^d \epsilon \end{pmatrix},\nonumber \\ \mathcal{M}_\ell= & {} \dfrac{v}{\sqrt{2}} \begin{pmatrix} y_{11}^\ell \epsilon ^5 &{} y_{12}^\ell \epsilon ^5 &{} y_{13}^\ell \epsilon ^5 \\ y_{21}^\ell \epsilon ^3 &{} y_{22}^\ell \epsilon ^3 &{} y_{23}^\ell \epsilon ^3 \\ y_{31}^\ell \epsilon &{} y_{32}^\ell \epsilon &{} y_{33}^\ell \epsilon \end{pmatrix}. \end{aligned}$$
(4)

The masses of quarks and charged leptons approximately are [39],

$$\begin{aligned} \&\{m_t, m_c, m_u\} \nonumber \\&\quad \simeq \left\{ |y_{33}^u| , ~ \left| y_{22}^u- \frac{y_{23}^u y_{32}^u}{|y_{33}^u|} \right| \epsilon ^2,\right. \nonumber \\&\quad ~ \left| y_{11}^u- \frac{y_{12}^u y_{21}^u}{|y_{22}^u-y_{23}^u y_{32}^u/y_{33}^u|}\right. \nonumber \\&\quad \left. \left. - \frac{y_{13}^u |y_{31}^u y_{22}^u-y_{21}^u y_{32}^u|-y_{31}^u y_{12}^u y_{23}^u}{|y_{22}^u- y_{23}^u y_{32}^u/y_{33}^u| |y_{33}^u|} \right| \epsilon ^4\right\} v/\sqrt{2} , \end{aligned}$$
(5)
$$\begin{aligned}&\{m_b, m_s, m_d\} \nonumber \\&\quad \simeq \left\{ |y_{33}^d| \epsilon , ~ \left| y_{22}^d- \frac{y_{23}^d y_{32}^d}{|y_{33}^d|} \right| \epsilon ^3,\right. \nonumber \\&\quad ~ \left| y_{11}^d- \frac{y_{12}^d y_{21}^d}{|y_{22}^d-y_{23}^d y_{32}^d/y_{33}^d|}\right. \nonumber \\&\quad \left. \left. - \frac{y_{13}^d |y_{31}^d y_{22}^d-y_{21}^d y_{32}^d|-y_{31}^d y_{12}^d y_{23}^d}{|y_{22}^d- y_{23}^d y_{32}^d/y_{33}^d| |y_{33}^d|} \right| \epsilon ^5\right\} v/\sqrt{2} , \end{aligned}$$
(6)
$$\begin{aligned}&\{m_\tau , m_\mu , m_e\} \nonumber \\&\quad \simeq \left\{ |y_{33}^l| \epsilon , ~ \left| y_{22}^l- \frac{y_{23}^l y_{32}^l}{|y_{33}^l|} \right| \epsilon ^3,\right. \nonumber \\&\quad ~ \left| y_{11}^l- \frac{y_{12}^l y_{21}^l}{|y_{22}^l-y_{23}^l y_{32}^l/y_{33}^l|}\right. \nonumber \\&\left. \left. \quad - \frac{y_{13}^l |y_{31}^l y_{22}^l-y_{21}^l y_{32}^l|-y_{31}^l y_{12}^l y_{23}^l}{|y_{22}^l- y_{23}^l y_{32}^l/y_{33}^l| |y_{33}^l|} \right| \epsilon ^5\right\} v/\sqrt{2}. \end{aligned}$$
(7)

The mixing angles of quarks are found to be [39],

$$\begin{aligned} \sin \theta _{12} \simeq |V_{us}|\simeq & {} \left| {y_{12}^d \over y_{22}^d} -{y_{12}^u \over y_{22}^u} \right| \epsilon ^2,\nonumber \\ \sin \theta _{23} \simeq |V_{cb}|\simeq & {} \left| {y_{23}^d \over y_{33}^d} -{y_{23}^u \over y_{33}^u} \right| \epsilon ^2,\nonumber \\ \sin \theta _{13} \simeq |V_{ub}|\simeq & {} \left| {y_{13}^d \over y_{33}^d} -{y_{12}^u y_{23}^d \over y_{22}^u y_{33}^d} - {y_{13}^u \over y_{33}^u} \right| \epsilon ^4. \end{aligned}$$
(8)

We notice that the \(\sin \theta _{12}\) and \(\sin \theta _{23}\) have the same order. The similar result is also reported in Ref. [8].

We present a fit of the experimental data to the masses of fermions in Appendix. It turns out that some of the couplings are not order one. We discuss a theoretical scenario for such couplings in the Appendix. This kind of not order one couplings are also reported in Refs. [8, 26].

2.4.1 Neutrino masses and mixing

The neutrino masses are obtained by adding three right-handed neutrinos as shown in Table 4. The Lagrangian for the tree-level Majorana mass is,

$$\begin{aligned} {\mathcal {L}}_{\mathrm{M_R}} = c_{ij} \left[ \frac{\chi ^\dagger }{\Lambda } \right] ^5 \chi ^\dagger \bar{\nu ^c}_{i,R} \nu _{j,R}, \end{aligned}$$
(9)

where ij are flavour indices.

The Majorana mass matrices \(\mathcal{M}_{R}\) is,

$$\begin{aligned} \mathcal{M}_{R} = M \begin{pmatrix} c_{11}&{} c_{12} &{} c_{13} \\ c_{12} &{} c_{22} &{} c_{23} \\ c_{13} &{} c_{23} &{} c_{33} \end{pmatrix}, \end{aligned}$$
(10)

where \( M= \langle \chi \rangle \left[ \frac{ \langle \chi \rangle }{\Lambda } \right] ^5 = \frac{f}{\sqrt{2}} \epsilon ^5 \).

The Dirac mass Lagrangian for neutrinos can be written as,

$$\begin{aligned} -{{\mathcal {L}}}_{\textrm{Yukawa}}^{\nu }= & {} y_{11}^\nu {\bar{\psi }}_{L_1}^\ell H \nu _{e_R} \left[ \dfrac{ \chi }{\Lambda } \right] ^{3} + y_{12}^\nu {\bar{\psi }}_{L_1}^\ell H \nu _{\mu _R} \left[ \dfrac{ \chi }{\Lambda } \right] ^{3} \nonumber \\{} & {} + y_{13}^\nu \bar{ \psi }_{L_1}^\ell H \nu _{\tau _R} \left[ \dfrac{ \chi }{\Lambda } \right] ^{3} + y_{21}^\nu {\bar{\psi }}_{L_2}^\ell H \nu _{e_R} \left[ \dfrac{ \chi }{\Lambda } \right] \nonumber \\{} & {} + y_{22}^\nu \bar{ \psi }_{L_2}^\ell H \nu _{\mu _R} \left[ \dfrac{ \chi }{\Lambda } \right] + y_{23}^\nu {\bar{\psi }}_{L_2}^\ell H \nu _{\tau _R} \left[ \dfrac{ \chi }{\Lambda } \right] \nonumber \\{} & {} + y_{31}^\nu \bar{ \psi }_{L_3}^\ell H \nu _{e_R} \left[ \dfrac{ \chi ^\dagger }{\Lambda } \right] + y_{32}^\nu {\bar{\psi }}_{L_3}^\ell H \nu _{\mu _R} \left[ \dfrac{ \chi ^\dagger }{\Lambda } \right] \nonumber \\{} & {} + y_{33}^\nu {\bar{\psi }}_{L_3}^\ell H \nu _{\tau _R} \left[ \dfrac{ \chi ^\dagger }{\Lambda } \right] + {\mathrm{H.c.}}. \end{aligned}$$
(11)

The Dirac mass matrix is given by,

$$\begin{aligned} \mathcal{M}_{\mathcal{D}} = \dfrac{v}{\sqrt{2}} \begin{pmatrix} y_{11}^\nu \epsilon ^3 &{} y_{12}^\nu \epsilon ^3 &{} y_{13}^\nu \epsilon ^{3} \\ y_{21}^\nu \epsilon &{} y_{22}^\nu \epsilon &{} y_{23}^\nu \epsilon \\ y_{31}^\nu \epsilon &{} y_{32}^\nu \epsilon &{} y_{33}^\nu \epsilon \end{pmatrix}. \end{aligned}$$
(12)

The mass matrix of neutrinos after including the Majorana mass terms can be written as,

$$\begin{aligned} \mathcal{M}= \begin{pmatrix} \mathcal{M}_{L} &{} \mathcal{M}_{\mathcal{D}} \\ \mathcal{M}_{\mathcal{D}}^T &{} \mathcal{M}_{R} \\ \end{pmatrix}. \end{aligned}$$
(13)

Since \(v\ll f\), we ignore the contribution of the mass matrix \(\mathcal{M}_{L}\) to the neutrino masses.Footnote 2 Now, we can use the type-I seesaw mechanism to determine the neutrino masses by assuming \( \mathcal{M}_{\mathcal{D}} \ll \mathcal{M}_{R}\) [40,41,42,43,44,45]. Thus the light neutrino mass matrix is,

$$\begin{aligned} { \mathcal{M}}~\approx & {} ~ - \mathcal{M}_{\mathcal{D}} \mathcal{M}_{R}^{-1} \mathcal{M}_{\mathcal{D}}^T, \nonumber \\\approx & {} \frac{v}{\sqrt{2} } \epsilon ^\prime \left( \begin{array}{ccc} -\frac{\epsilon ^4 \left( c_{22} c_{33} y_{11}^{\nu 2}-2 c_{22} y_{11}^\nu +c_{22} -2c_{33} y_{11}^\nu +c_{33}-y_{11}^{\nu 2}+4 y_{11}^\nu -2\right) }{(c_{22}-1) (c_{33}-1)} &{} -\epsilon ^2 y_{11}^\nu &{} -\frac{\epsilon ^2 ( c_{33} y_{11}^\nu - y_{11}^\nu y_{33}^\nu + y_{33}^\nu -1)}{c_{33}-1} \\ -\epsilon ^2 y_{11}^\nu &{} -1 &{} -1 \\ -\frac{\epsilon ^2 (c_{33} y_{11}^\nu - y_{11}^\nu y_{33}^\nu + y_{33}^\nu -1)}{c_{33}-1} &{} -1 &{} -\frac{ (c_{33}+(y_{33}^\nu -2) y_{33}^\nu )}{c_{33} -1} \\ \end{array} \right) , \end{aligned}$$
(14)

where \(\epsilon ^{\prime } = \frac{v}{ f \epsilon ^3} \), and we have assumed each and every coupling exactly one except those appearing in above equation.

We obtain two degenerate neutrino masses. The masses approximately are given by,

$$\begin{aligned} m_1\approx & {} \frac{ \left( -y_{11}^{\nu 2}+2 y_{11}^\nu -1\right) }{c_{22} -1} \epsilon ^{4} \epsilon ^{\prime } v/\sqrt{2}, \nonumber \\ m_2\approx & {} \frac{ \left( -\sqrt{4 c_{33}^2-8 c_{33}+y_{33}^{\nu 4}-4 y_{33}^{\nu 3}+6 y_{33}^{\nu 2}-4 y_{33}^\nu +5}-2 y_{33}^\nu -y_{33}^{\nu 2}+2 y_{33}^\nu +1\right) }{2 (c_{33}-1)} ~ \epsilon ^{\prime } v/\sqrt{2}, \nonumber \\ m_3\approx & {} \frac{ \left( \sqrt{4 c_{33}^2-8 c_{33}+y_{33}^{\nu 4}-4 y_{33}^{\nu 3}+6 y_{33}^{\nu 2}-4 y_{33}^\nu +5}-2 c_{33}- y_{33}^{\nu 2} +2 y_{33}^\nu +1\right) }{2 (c_{33}-1)} ~ \epsilon ^{\prime } v/\sqrt{2}. \end{aligned}$$
(15)

This kind of approximate degenerate neutrino masses are well studied in literature, for instance, see Refs. [46,47,48,49].

The leptonic mixing angles can be written as,

$$\begin{aligned} \sin \theta _{12}\simeq & {} \left| {y_{12}^\ell \over y_{22}^\ell } -y_{11}^\nu \right| \epsilon ^2, \nonumber \\ \sin \theta _{23}\simeq & {} \left| {1-c_{33} \over c_{33}+(y_{33}^\nu -2)y_{33}^\nu } \right| , \nonumber \\ \sin \theta _{13}\simeq & {} \left| \frac{ ( c_{33} y_{11}^\nu -y_{11}^\nu y_{33}^\nu + y_{33}^\nu -1)}{ c_{33}+(y_{33}^\nu -2 )y_{33}^\nu } \right| \epsilon ^2. \end{aligned}$$
(16)

The remarkable observation is the pattern of the neutrino mixing angles. The mixing angle \( \theta _{12}\) and \( \theta _{13}\) are of the same order of magnitude, where \( \theta _{13}\) is closer to the Cabibbo angle, and the mixing angle \( \theta _{23}\) is completely unsuppressed.

3 A non-minimal \({\mathcal {Z}}_2 \times {\mathcal {Z}}_9\) flavour symmetry

We note from the previous section that some of the Yukawa couplings for the minimal model based on the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_5\) flavour symmetry are not order one, which is a preferred choice in literature. However, so far purpose has been to introduce the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_N\) flavour paradigm. In this section, we show a non-minimal model based on the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_N\) flavour paradigm where the Yukawa couplings turn out to be order one, and are given in the appendix. We adopt a non-minimal \({\mathcal {Z}}_2 \times {\mathcal {Z}}_9\) flavour symmetry, and assign the charges to different fields as shown in Table 5.

The \({\mathcal {Z}}_2 \times {\mathcal {Z}}_9\) flavour symmetry allows us to write the following Lagrangian which provides masses to the charged fermions of the SM,

$$\begin{aligned} -{{\mathcal {L}}}_{\textrm{Yukawa}}= & {} \left( \dfrac{ \chi ^\dag }{\Lambda } \right) ^{8} y_{11}^u {\bar{\psi }}_{L_1}^q {\tilde{\varphi }} u_{R}+ \left( \dfrac{ \chi }{\Lambda } \right) ^{6} y_{12}^u {\bar{\psi }}_{L_1}^q {\tilde{\varphi }} c_{R} \\{} & {} + \left( \dfrac{ \chi ^\dag }{\Lambda } \right) ^{8} y_{13}^u {\bar{\psi }}_{L_1}^q {\tilde{\varphi }} t_{R} + \left( \dfrac{ \chi }{\Lambda } \right) ^{8} y_{21}^u {\bar{\psi }}_{L_2}^q {\tilde{\varphi }} u_{R} \\{} & {} + \left( \dfrac{ \chi }{\Lambda } \right) ^{4} y_{22}^u {\bar{\psi }}_{L_2}^q {\tilde{\varphi }} c_{R} + \left( \dfrac{ \chi }{\Lambda } \right) ^{8} y_{23}^u {\bar{\psi }}_{L_2}^q {\tilde{\varphi }} t_{R}\\{} & {} + y_{31}^u {\bar{\psi }}_{L_3}^q {\tilde{\varphi }} u_{R}+ \left( \dfrac{ \chi ^\dag }{\Lambda } \right) ^{4} y_{32}^u {\bar{\psi }}_{L_3}^q {\tilde{\varphi }} c_{R} \\{} & {} + y_{33}^u {\bar{\psi }}_{L_3}^q {\tilde{\varphi }} t_{R} + \left( \dfrac{ \chi }{\Lambda } \right) ^{7} y_{11}^d {\bar{\psi }}_{L_1}^q \varphi d_{R}\\{} & {} + \left( \dfrac{ \chi }{\Lambda } \right) ^{7} y_{12}^d {\bar{\psi }}_{L_1}^q \varphi s_{R} + \left( \dfrac{ \chi }{\Lambda } \right) ^{7} y_{13}^d {\bar{\psi }}_{L_1}^q \varphi b_{R} \\{} & {} + \left( \dfrac{ \chi }{\Lambda } \right) ^{5} y_{21}^d {\bar{\psi }}_{L_2}^q \varphi d_{R} + \left( \dfrac{ \chi }{\Lambda } \right) ^{5} y_{22}^d {\bar{\psi }}_{L_2}^q \varphi s_{R} \\{} & {} + \left( \dfrac{ \chi }{\Lambda } \right) ^{5} y_{23}^d {\bar{\psi }}_{L_2}^q \varphi b_{R} + \left( \dfrac{ \chi ^\dag }{\Lambda } \right) ^{3} y_{31}^d {\bar{\psi }}_{L_3}^q \varphi d_{R}\\{} & {} + \left( \dfrac{ \chi ^\dag }{\Lambda } \right) ^{3} y_{32}^d {\bar{\psi }}_{L_3}^q \varphi s_{R} + \left( \dfrac{ \chi ^\dag }{\Lambda } \right) ^{3} y_{33}^d \bar{ \psi }_{L_3}^q \varphi b_{R} \\{} & {} + \left( \dfrac{ \chi }{\Lambda } \right) ^{7} y_{11}^\ell {\bar{\psi }}_{L_1}^\ell \varphi e_{R} + \left( \dfrac{ \chi }{\Lambda } \right) ^{7} y_{12}^\ell {\bar{\psi }}_{L_1}^\ell \varphi \mu _{R} \\{} & {} + \left( \dfrac{ \chi }{\Lambda } \right) ^{7} y_{13}^\ell {\bar{\psi }}_{L_1}^\ell \varphi \tau _{R} + \left( \dfrac{ \chi }{\Lambda } \right) ^{5} y_{21}^\ell {\bar{\psi }}_{L_2}^\ell \varphi e_{R}\\{} & {} + \left( \dfrac{ \chi }{\Lambda } \right) ^{5} y_{22}^\ell {\bar{\psi }}_{L_2}^\ell \varphi \mu _{R} + \left( \dfrac{ \chi }{\Lambda } \right) ^{5} y_{23}^\ell {\bar{\psi }}_{L_2}^\ell \varphi \tau _{R} \\{} & {} + \left( \dfrac{ \chi }{\Lambda } \right) ^{3} y_{31}^\ell {\bar{\psi }}_{L_3}^\ell \varphi e_{R} + \left( \dfrac{ \chi }{\Lambda } \right) ^{3} y_{32}^\ell \bar{ \psi }_{L_3}^\ell \varphi \mu _{R} \\{} & {} + \left( \dfrac{ \chi }{\Lambda } \right) ^{3} y_{33}^\ell {\bar{\psi }}_{L_3}^\ell \varphi \tau _{R} + \text {H.c.} \end{aligned}$$
Table 5 The charges of left and right-handed fermions of three families of the SM, right-handed neutrinos, Higgs, and singlet scalar field under \({\mathcal {Z}}_2\) and \({\mathcal {Z}}_9\) symmetries, where \(\omega \) is the ninth root of unity

The mass matrices for up and down-type quarks and charged leptons turn out to be,

$$\begin{aligned} \mathcal{M}_u= & {} \dfrac{v}{\sqrt{2}} \begin{pmatrix} y_{11}^u \epsilon ^8 &{} y_{12}^u \epsilon ^{6} &{} y_{13}^u \epsilon ^{8} \\ y_{21}^u \epsilon ^8 &{} y_{22}^u \epsilon ^4 &{} y_{23}^u \epsilon ^8 \\ y_{31}^u &{} y_{32}^u \epsilon ^4 &{} y_{33}^u \end{pmatrix},\nonumber \\ \mathcal{M}_d= & {} \dfrac{v}{\sqrt{2}} \begin{pmatrix} y_{11}^d \epsilon ^7 &{} y_{12}^d \epsilon ^7 &{} y_{13}^d \epsilon ^7 \\ y_{21}^d \epsilon ^5 &{} y_{22}^d \epsilon ^5 &{} y_{23}^d \epsilon ^5 \\ y_{31}^d \epsilon ^3 &{} y_{32}^d \epsilon ^3 &{} y_{33}^d \epsilon ^3 \end{pmatrix},\nonumber \\ \mathcal{M}_\ell= & {} \dfrac{v}{\sqrt{2}} \begin{pmatrix} y_{11}^\ell \epsilon ^7 &{} y_{12}^\ell \epsilon ^7 &{} y_{13}^\ell \epsilon ^7 \\ y_{21}^\ell \epsilon ^5 &{} y_{22}^\ell \epsilon ^5 &{} y_{23}^\ell \epsilon ^5 \\ y_{31}^\ell \epsilon ^3 &{} y_{32}^\ell \epsilon ^3 &{} y_{33}^\ell \epsilon ^3 \end{pmatrix}. \end{aligned}$$
(17)

The masses of charged fermions are approximately given by [39],

$$\begin{aligned}&\{m_t, m_c, m_u\}\nonumber \\&\quad \simeq \left\{ |y_{33}^u| , ~ \left| y_{22}^u \epsilon ^4 - \frac{y_{23}^u y_{32}^u}{|y_{33}^u| } \epsilon ^{12} \right| ,\right. \nonumber \\&\quad ~ \left| y_{11}^u \epsilon ^8 - \frac{y_{12}^u y_{21}^u}{|y_{22}^u|} \epsilon ^{10}\right. \nonumber \\&\left. \left. \quad - \frac{y_{13}^u |y_{31}^u y_{22}^u-y_{21}^u y_{32}^u|}{|y_{22}^u| |y_{33}^u|} \epsilon ^8 \right| \right\} v/\sqrt{2} , \end{aligned}$$
(18)
$$\begin{aligned}&\{m_b, m_s, m_d\} \nonumber \\&\quad \simeq \left\{ |y_{33}^d| \epsilon ^3, ~ \left| y_{22}^d- \frac{y_{23}^d y_{32}^d}{|y_{33}^d|} \right| \epsilon ^5,\right. \nonumber \\&\quad \left| y_{11}^d- \frac{y_{12}^d y_{21}^d}{|y_{22}^d-y_{23}^d y_{32}^d/y_{33}^d|}\right. \nonumber \\&\left. \left. \quad - \frac{y_{13}^d |y_{31}^d y_{22}^d-y_{21}^d y_{32}^d|-y_{31}^d y_{12}^d y_{23}^d}{|y_{22}^d- y_{23}^d y_{32}^d/y_{33}^d| |y_{33}^d|} \right| \epsilon ^7\right\} v/\sqrt{2} , \end{aligned}$$
(19)
$$\begin{aligned}&\{m_\tau , m_\mu , m_e\} \nonumber \\&\quad \simeq \left\{ |y_{33}^l| \epsilon ^3, ~ \left| y_{22}^l- \frac{y_{23}^l y_{32}^l}{|y_{33}^l|} \right| \epsilon ^5,\right. \nonumber \\&\quad \left| y_{11}^l- \frac{y_{12}^l y_{21}^l}{|y_{22}^l-y_{23}^l y_{32}^l/y_{33}^l|}\right. \nonumber \\&\quad \left. \left. - \frac{y_{13}^l |y_{31}^l y_{22}^l-y_{21}^l y_{32}^l|-y_{31}^l y_{12}^l y_{23}^l}{|y_{22}^l- y_{23}^l y_{32}^l/y_{33}^l| |y_{33}^l|} \right| \epsilon ^7\right\} v/\sqrt{2} . \end{aligned}$$
(20)

Similarly, the mixing angles of quarks read [39],

$$\begin{aligned} \sin \theta _{12} \simeq |V_{us}|\simeq & {} \left| {y_{12}^d \over y_{22}^d} -{y_{12}^u \over y_{22}^u} \right| \epsilon ^2,\nonumber \\ \sin \theta _{23} \simeq |V_{cb}|\simeq & {} \left| {y_{23}^d \over y_{33}^d} \epsilon ^2 -{y_{23}^u \over y_{33}^u} \epsilon ^8 \right| ,\nonumber \\ \sin \theta _{13} \simeq |V_{ub}|\simeq & {} \left| {y_{13}^d \over y_{33}^d} \epsilon ^4 -{y_{12}^u y_{23}^d \over y_{22}^u y_{33}^d} \epsilon ^4 - {y_{13}^u \over y_{33}^u} \epsilon ^8 \right| . \end{aligned}$$
(21)

3.1 Neutrino masses and mixing

The masses and mixing of neutrinos in the non-minimal model is identical to that of the minimal model. Thus, we write the Majorana Lagrangian for right-handed neutrinos as,

$$\begin{aligned} {\mathcal {L}}_{\mathrm{M_R}} = c_{ij} \left[ \frac{\chi }{\Lambda } \right] ^3 \chi \bar{\nu ^c}_{i,R} \nu _{j,R}, \end{aligned}$$
(22)

where ij are flavour indices.

The Majorana mass matrices \(\mathcal{M}_{R}\) can be written as,

$$\begin{aligned} \mathcal{M}_{R} = M \begin{pmatrix} c_{11}&{} c_{12} &{} c_{13} \\ c_{12} &{} c_{22} &{} c_{23} \\ c_{13} &{} c_{23} &{} c_{33} \end{pmatrix}, \end{aligned}$$
(23)

where \( M= \langle \chi \rangle \left[ \frac{ \langle \chi \rangle }{\Lambda } \right] ^3 = \frac{f}{\sqrt{2}} \epsilon ^3 \).

The Dirac mass Lagrangian for neutrinos is,

$$\begin{aligned} -{{\mathcal {L}}}_{\textrm{Yukawa}}^{\nu }= & {} y_{11}^\nu {\bar{\psi }}_{L_1}^\ell H \nu _{e_R} \left[ \dfrac{ \chi }{\Lambda } \right] ^{3} + y_{12}^\nu {\bar{\psi }}_{L_1}^\ell H \nu _{\mu _R} \left[ \dfrac{ \chi }{\Lambda } \right] ^{3} \nonumber \\{} & {} + y_{13}^\nu \bar{ \psi }_{L_1}^\ell H \nu _{\tau _R} \left[ \dfrac{ \chi }{\Lambda } \right] ^{3} + y_{21}^\nu {\bar{\psi }}_{L_2}^\ell H \nu _{e_R} \left[ \dfrac{ \chi }{\Lambda } \right] \nonumber \\{} & {} + y_{22}^\nu \bar{ \psi }_{L_2}^\ell H \nu _{\mu _R} \left[ \dfrac{ \chi }{\Lambda } \right] + y_{23}^\nu {\bar{\psi }}_{L_2}^\ell H \nu _{\tau _R} \left[ \dfrac{ \chi }{\Lambda } \right] \nonumber \\{} & {} + y_{31}^\nu \bar{ \psi }_{L_3}^\ell H \nu _{e_R} \left[ \dfrac{ \chi ^\dagger }{\Lambda } \right] + y_{32}^\nu {\bar{\psi }}_{L_3}^\ell H \nu _{\mu _R} \left[ \dfrac{ \chi ^\dagger }{\Lambda } \right] \nonumber \\{} & {} + y_{33}^\nu {\bar{\psi }}_{L_3}^\ell H \nu _{\tau _R} \left[ \dfrac{ \chi ^\dagger }{\Lambda } \right] + {\mathrm{H.c.}} \end{aligned}$$
(24)

The Dirac mass matrix for neutrinos now reads,

$$\begin{aligned} \mathcal{M}_{\mathcal{D}} = \dfrac{v}{\sqrt{2}} \begin{pmatrix} y_{11}^\nu \epsilon ^3 &{} y_{12}^\nu \epsilon ^3 &{} y_{13}^\nu \epsilon ^{3} \\ y_{21}^\nu \epsilon &{} y_{22}^\nu \epsilon &{} y_{23}^\nu \epsilon \\ y_{31}^\nu \epsilon &{} y_{32}^\nu \epsilon &{} y_{33}^\nu \epsilon \end{pmatrix}. \end{aligned}$$
(25)

The mass matrix of neutrinos after including the Majorana mass term is,

$$\begin{aligned} \mathcal{M}= \begin{pmatrix} \mathcal{M}_{L} &{} \mathcal{M}_{\mathcal{D}} \\ \mathcal{M}_{\mathcal{D}}^T &{} \mathcal{M}_{R} \\ \end{pmatrix}. \end{aligned}$$
(26)

The light neutrino mass matrix is,

$$\begin{aligned} { \mathcal{M}}~\approx & {} ~ - \mathcal{M}_{\mathcal{D}} \mathcal{M}_{R}^{-1} \mathcal{M}_{\mathcal{D}}^T, \nonumber \\\approx & {} \frac{v}{\sqrt{2} } \epsilon ^\prime \left( \begin{array}{ccc} -\frac{\epsilon ^4 \left( c_{22} c_{33} y_{11}^{\nu 2}-2 c_{22} y_{11}^\nu +c_{22} -2c_{33} y_{11}^\nu +c_{33}-y_{11}^{\nu 2}+4 y_{11}^\nu -2\right) }{(c_{22}-1) (c_{33}-1)} &{} -\epsilon ^2 y_{11}^\nu &{} -\frac{\epsilon ^2 ( c_{33} y_{11}^\nu - y_{11}^\nu y_{33}^\nu + y_{33}^\nu -1)}{c_{33}-1} \\ -\epsilon ^2 y_{11}^\nu &{} -1 &{} -1 \\ -\frac{\epsilon ^2 (c_{33} y_{11}^\nu - y_{11}^\nu y_{33}^\nu + y_{33}^\nu -1)}{c_{33}-1} &{} -1 &{} -\frac{ (c_{33}+(y_{33}^\nu -2) y_{33}^\nu )}{c_{33} -1} \\ \end{array} \right) , \end{aligned}$$
(27)

where \(\epsilon ^{\prime } = \frac{v}{ f \epsilon } \), and we have again assumed each and every coupling exactly one except those appearing in above equation.

The neutrino masses approximately are,

$$\begin{aligned} m_1\approx & {} \frac{ \left( -y_{11}^{\nu 2}+2 y_{11}^\nu -1\right) }{c_{22} -1} \epsilon ^{4} \epsilon ^{\prime } v/\sqrt{2}, \nonumber \\ m_2\approx & {} \frac{ \left( -\sqrt{4 c_{33}^2-8 c_{33}+y_{33}^{\nu 4}-4 y_{33}^{\nu 3}+6 y_{33}^{\nu 2}-4 y_{33}^\nu +5}-2 y_{33}^\nu -y_{33}^{\nu 2}+2 y_{33}^\nu +1\right) }{2 (c_{33}-1)} ~ \epsilon ^{\prime } v/\sqrt{2}, \nonumber \\ m_3\approx & {} \frac{ \left( \sqrt{4 c_{33}^2-8 c_{33}+y_{33}^{\nu 4}-4 y_{33}^{\nu 3}+6 y_{33}^{\nu 2}-4 y_{33}^\nu +5}-2 c_{33}- y_{33}^{\nu 2} +2 y_{33}^\nu +1\right) }{2 (c_{33}-1)} ~ \epsilon ^{\prime } v/\sqrt{2}. \end{aligned}$$
(28)

The neutrino mixing angles are,

$$\begin{aligned} \sin \theta _{12}\simeq & {} \left| {y_{12}^\ell \over y_{22}^\ell } -y_{11}^\nu \right| \epsilon ^2, \nonumber \\ \sin \theta _{23}\simeq & {} \left| {1-c_{33} \over c_{33}+(y_{33}^\nu -2)y_{33}^\nu } \right| , \nonumber \\ \sin \theta _{13}\simeq & {} \left| \frac{ ( c_{33} y_{11}^\nu -y_{11}^\nu y_{33}^\nu + y_{33}^\nu -1)}{ c_{33}+(y_{33}^\nu -2 )y_{33}^\nu } \right| \epsilon ^2. \end{aligned}$$
(29)

4 The scalar potential

The scalar potential of the model can be written in the following form,

$$\begin{aligned} - {\mathcal {L}}_\text {potential}&=- \mu ^2 \varphi ^\dagger \varphi +\lambda (\varphi ^\dagger \varphi )^2 - \mu _\chi ^2\, \chi ^* \chi + \lambda _\chi \, (\chi ^* \chi )^2 \nonumber \\&\quad + \, (\rho \ \chi ^2 + {\mathrm{H.c.}} ) + \lambda _{\varphi \chi } (\chi ^* \chi ) (\varphi ^\dagger \varphi ), \end{aligned}$$
(30)

where we have introduced a soft breaking of the \({\mathcal {Z}}_5\) symmetry in the fifth term. We are assuming \( \lambda _{\varphi \chi } =0\), i.e., no Higgs-flavon mixing [36]. If this term is non-zero, the phenomenology of the flavon field will be different, for instance, see Ref. [28]. The only parameter which can have a phase in the scalar potential is \(\rho \). However, this phase can be removed by a phase rotation of the flavon field \(\chi \) leading to a real value of the VEV of the field \(\chi \).

We can parametrize the flavon field by excitations around its VEV,

$$\begin{aligned} \chi (x)=\frac{f + s(x) +i\, a(x)}{\sqrt{2}}. \end{aligned}$$
(31)

In a similar manner, the Higgs field can be written as,

$$\begin{aligned} \varphi (x)=\frac{v+ h(x) }{\sqrt{2}}. \end{aligned}$$
(32)

The minimization conditions can be written in terms of the scalar and pseudo-scalar components having the following masses:

$$\begin{aligned} m_s =\sqrt{\mu _\chi - 2 \rho } = \sqrt{\lambda _\chi } f \quad \text {and} \quad m_a= \sqrt{-2 \rho }. \end{aligned}$$
(33)

We observe that the mass of the pseudoscalar component of the flavon field depends on the soft-breaking parameter \(\rho \). Therefore, it is a free parameter of the model. Now using Eq. 31, we can write

$$\begin{aligned} \frac{\chi }{\Lambda } = \epsilon [ 1 + \frac{s + i a}{f} ]. \end{aligned}$$
(34)

The couplings of the scalar and pseudoscalar components of the flavon field are obtained from Eq. 3 by writing the effective Yukawa couplings in the following form:

$$\begin{aligned} Y_{ij}^f \varphi= & {} y_{ij}^f \left( \frac{\chi }{\Lambda } \right) ^{n^{f}_{ij}} \left( \frac{v+ h }{\sqrt{2}}.\right) \nonumber \\\cong & {} y_{ij}^f \epsilon ^{n^{f}_{ij}} \frac{v}{\sqrt{2}} \left[ 1 + \frac{n^f_{ij} (s + i a )}{f} + \frac{h}{v}\right] \nonumber \\= & {} \mathcal{M}_f \left[ 1 + \frac{n^f_{ij} (s + i a )}{f} + \frac{h}{v}\right] , \end{aligned}$$
(35)

where \(f= u,d,\ell \), and \(n^{f}_{ij}\) is the power of the parameter \(\epsilon \) appearing in the mass matrices \(\mathcal{M}_f\).

We note that for our phenomenological investigation, we have only retained the terms linear in the flavon field components s and a in Eq. 35. The terms which are higher than the linear terms are not interesting in the present work. The couplings of the Higgs boson field h to the charged fermions are real and diagonal since the mass matrices \(\mathcal{M}_f\) can be diagonalized resulting in real and positive masses of the charged fermions. However, the couplings of the scalar and pseudoscalar components s and a of the flavon field are given by \(n^f_{ij} \mathcal{M}_f\). This product cannot be diagonalized exactly and as a consequence, the couplings of s and a cannot be made real and diagonal. This in turn gives rise to the flavour-changing and CP-violating interactions of the flavon field.

The couplings of a field with fermions for minimal \({\mathcal {Z}}_2 \times {\mathcal {Z}}_5\) symmetry are now given by,

$$\begin{aligned}{} & {} y_{af_{iL} f_{jR}}^{u} \equiv y_{aij}^u = \frac{v}{\sqrt{2}f} \begin{pmatrix} 4 y_{11}^u \epsilon ^4 &{} 4 y_{12}^u \epsilon ^4 &{} 4 y_{13}^u \epsilon ^4 \\ 2 y_{21}^u \epsilon ^2 &{} 2 y_{22}^u \epsilon ^2 &{} 2 y_{23}^u \epsilon ^2 \\ 0 &{} 0 &{} 0 \end{pmatrix},\nonumber \\{} & {} y_{aij}^{d} = \frac{v}{\sqrt{2}f} \begin{pmatrix} 5 y_{11}^d \epsilon ^5 &{} 5 y_{12}^d \epsilon ^5 &{} 5 y_{13}^d \epsilon ^5 \\ 3 y_{21}^d \epsilon ^3 &{} 3 y_{22}^d \epsilon ^3 &{} 3 y_{23}^d \epsilon ^3 \\ y_{31}^d \epsilon &{} y_{32}^d \epsilon &{} y_{33}^d \epsilon \end{pmatrix}, \nonumber \\{} & {} y_{aij}^{\ell } = \frac{v}{\sqrt{2}f} \begin{pmatrix} 5 y_{11}^\ell \epsilon ^5 &{} 5 y_{12}^\ell \epsilon ^5 &{} 5 y_{13}^\ell \epsilon ^5 \\ 3 y_{21}^\ell \epsilon ^3 &{} 3 y_{22}^\ell \epsilon ^3 &{} 3 y_{23}^\ell \epsilon ^3 \\ y_{31}^\ell \epsilon &{} y_{32}^\ell \epsilon &{} y_{33}^\ell \epsilon \end{pmatrix}. \end{aligned}$$
(36)

In the similar way, the couplings of a field with fermions for non-minimal \({\mathcal {Z}}_2 \times {\mathcal {Z}}_9\) symmetry are given by

$$\begin{aligned}{} & {} y_{af_{iL} f_{jR}}^{u} \equiv y_{aij}^u = \frac{v}{\sqrt{2}f} \begin{pmatrix} 8 y_{11}^u \epsilon ^8 &{} 6 y_{12}^u \epsilon ^6 &{} 8 y_{13}^u \epsilon ^8 \\ 8 y_{21}^u \epsilon ^8 &{} 4 y_{22}^u \epsilon ^4 &{} 8 y_{23}^u \epsilon ^8 \\ 0 &{} 4 y_{32}^u \epsilon ^4 &{} 0 \end{pmatrix},\nonumber \\{} & {} y_{aij}^{d} = \frac{v}{\sqrt{2}f} \begin{pmatrix} 7 y_{11}^d \epsilon ^7 &{} 7 y_{12}^d \epsilon ^7 &{} 7 y_{13}^d \epsilon ^7 \\ 5 y_{21}^d \epsilon ^5 &{} 5 y_{22}^d \epsilon ^5 &{} 5 y_{23}^d \epsilon ^5 \\ 3 y_{31}^d \epsilon ^3 &{} 3 y_{32}^d \epsilon ^3 &{} 3 y_{33}^d \epsilon ^3 \end{pmatrix}, \nonumber \\{} & {} y_{aij}^{\ell } = \frac{v}{\sqrt{2}f} \begin{pmatrix} 7 y_{11}^\ell \epsilon ^7 &{} 7 y_{12}^\ell \epsilon ^7 &{} 7 y_{13}^\ell \epsilon ^7 \\ 5 y_{21}^\ell \epsilon ^5 &{} 5 y_{22}^\ell \epsilon ^5 &{} 5 y_{23}^\ell \epsilon ^5 \\ 3 y_{31}^\ell \epsilon ^3 &{} 3 y_{32}^\ell \epsilon ^3 &{} 3 y_{33}^\ell \epsilon ^3 \end{pmatrix}. \end{aligned}$$
(37)

For the pseudoscalar component of flavon field, the following notation is used:

$$\begin{aligned} y_{ij}= y_{sf_{iL} f_{iR}} = - i y_{af_{iL} f_{iR}}. \end{aligned}$$
(38)

The couplings of a to fermions are identical to that of s except with a relative phase factor i. This factor becomes trivial in the squared amplitude of a Feynman diagram mediated by a. Thus, it does not give rise to CP-violating interactions to the order investigated in this work.

5 Quark flavour physics in the minimal and non-minimal \({\mathcal {Z}}_2 \times {\mathcal {Z}}_N\) flavour symmetry

The quark flavour physics places stronger bounds on the parameter space of our model, which is parametrized by the VEV of the flavon field f, the mass of the pseudoscalar flavon \(m_a\), and the quartic coupling \(\lambda _\chi \). In particular, measurement of the loop-induced processes in the SM such as neutral meson mixing and rare mesonic decays constrain the parameter space of the model. The numerical inputs used in this work are given in Table 6.

5.1 Neutral meson mixing

The non-diagonal couplings of the flavon to fermions introduce the FCNC interactions at tree-level. Therefore, they are expected to be highly suppressed from neutral meson–antimeson mixing. These interactions of the flavon can be parametrized by writing the \(\Delta F =2\) effective Hamiltonian as follows,

$$\begin{aligned} \mathcal{H}_\text {NP}^{\Delta F=2}&=C_1^{ij} \,( {{\bar{q}}}^i_L\,\gamma _\mu \, q^j_L)^2+{{\widetilde{C}}}_1^{ij} \,( {{\bar{q}}}^i_R\,\gamma _\mu \, q^j_R)^2 \nonumber \\&\quad +C_2^{ij} \,( {{\bar{q}}}^i_R \, q^j_L)^2+{{\widetilde{C}}}_2^{ij} \,( {{\bar{q}}}^i_L \, q^j_R)^2+ C_4^{ij}\, ( {{\bar{q}}}^i_R \, q^j_L)\, \nonumber \\&\quad \times ( {{\bar{q}}}^i_L \, q^j_R)\, +C_5^{ij}\, ( {{\bar{q}}}^i_L \,\gamma _\mu \, q^j_L)\, ( \bar{q}^i_R \, \gamma ^\mu q^j_R)\,+ \text {H.c.}, \end{aligned}$$
(39)

where \(q_{R,L} = \frac{1\pm \gamma _5}{2} q\) and the colour indices are omitted for simplicity.

Table 6 Values of the experimental and theoretical quantities used as input parameters

The tree-level contribution to neutral meson mixing due to the flavon exchange gives rise to the following Wilson coefficients [59, 60],

$$\begin{aligned} C_2^{ij}&= -(y_{ji}^*)^2\left( \frac{1}{m_s^2}-\frac{1}{m_a^2}\right) \nonumber \\ {\tilde{C}}_2^{ij}&= -y_{ij}^2\left( \frac{1}{m_s^2}-\frac{1}{m_a^2}\right) \nonumber \\ C_4^{ij}&= -\frac{y_{ij}y_{ji}}{2}\left( \frac{1}{m_s^2}+\frac{1}{m_a^2}\right) , \end{aligned}$$
(40)

where \(m_{s}\) and \(m_{a}\) are the masses of scalar and pseudoscalar component of flavon field, respectively.

The Wilson coefficients \(C_i\) are computed at a scale \(\Lambda \), where heavier new degrees of freedom are integrated out. They need to be evolved down to the hadronic scales 4.6 GeV for bottom mesons, 2.8 GeV for charmed mesons, and 2 GeV for kaons. These particular scales are used in the lattice computations of the corresponding matrix elements [52, 56, 57]. In this work, renormalization group running of the matrix elements is implemented as discussed in Ref. [57] and matrix elements are taken from Refs. [52, 56]. Thus, the new physics contribution to the \(B_q -{{\bar{B}}}_q\) mixing amplitudes due to the Wilson coefficients \(C_i\) at a scale \(\Lambda \) can be written as [57],

$$\begin{aligned} \langle {{\bar{B}}}_q \vert \mathcal{H}_{\textrm{eff}}^{\Delta B=2} \vert B_q \rangle _i= & {} \sum _{j=1}^5 \sum _{r=1}^5 \left( b^{(r,i)}_j + \eta \,c^{(r,i)}_j\right) \nonumber \\{} & {} \times \eta ^{a_j} \,C_i(\Lambda )\, \langle {{\bar{B}}}_q \vert Q_r^{bq} \vert B_q \rangle , \end{aligned}$$
(41)

where \(q=d,s\), \(\alpha _s\) is the strong coupling constant, \(\eta =\alpha _s(\Lambda )/\alpha _s(m_t)\), and \(a_j\), \(b^{(r,i)}_j\), \(c^{(r,i)}_j\) are the so-called the magic numbers which are taken from Ref. [61]. We can write a similar formula for \(D^0 - {{\bar{D}}}^0\) mixing with magic numbers given in Ref. [57]. For \(K^0-{{\bar{K}}}^0\) mixing, the formula becomes [57],

$$\begin{aligned} \langle {{\bar{K}}}^0 \vert \mathcal{H}_{\textrm{eff}}^{\Delta S=2} \vert K^0 \rangle _i= & {} \sum _{j=1}^5 \sum _{r=1}^5 \left( b^{(r,i)}_j + \eta \,c^{(r,i)}_j\right) \nonumber \\{} & {} \times \eta ^{a_j} \,C_i(\Lambda )\, R_r \, \langle {{\bar{K}}}^0 \vert Q_1^{sd} \vert K^0 \rangle , \end{aligned}$$
(42)

where \(R_r\) are the ratio of the matrix elements of NP operators over that of SM [62] and their numerical values are directly taken from Ref. [57] for our analysis. The magic numbers for \(K^0-{{\bar{K}}}^0\) mixing are taken from Ref. [52].

The mixing observables of the \(K^0-{{\bar{K}}}^0\) mixing can be used now to constrain the flavon mass and VEV by employing their experimental measurements. These are [57],

$$\begin{aligned} C_{\epsilon _K}&=\frac{ \text {Im} \langle K^0|{\mathcal {H}}_{\text {eff}}^{\Delta F=2}|{{\bar{K}}}^0\rangle }{\text {Im} \langle K^0| {\mathcal {H}}_{\text {SM}}^{\Delta F=2} |{{\bar{K}}}^0 \rangle } = 1.12_{-0.25}^{+0.27}, \nonumber \\ C_{\Delta m_K}&=\frac{\text {Re}\langle K^0|{\mathcal {H}}_{\text {eff}}^{\Delta F=2}|{{\bar{K}}}^0\rangle }{\text {Re} \langle K^0| {\mathcal {H}}_{\text {SM}}^{\Delta F=2} |{{\bar{K}}}^0 \rangle } = 0.93_{-0.42}^{+1.14} , \end{aligned}$$
(43)

where numbers are given at \(95\%\) C.L., \({\mathcal {H}}_{\text {eff}}^{\Delta F=2}\) contains the SM and flavon contributions, and \({\mathcal {H}}_{\text {SM}}^{\Delta F=2}\) represents only the SM contribution.

The mixing observables for the \(B_q -{{\bar{B}}}_q\) mixing are,

$$\begin{aligned} C_{B_{q}}e^{2i\phi _{B_{q}}}&=\frac{ \text {Im} \langle B_{q}^0|{\mathcal {H}}^{\Delta F=2}|{{\bar{B}}}_{q}^0\rangle }{\text {Im} \langle B_{q}^0| {\mathcal {H}}_{\text {SM}}^{\Delta F=2} |{{\bar{B}}}_{q}^0 \rangle } \end{aligned}$$

where \(q=s,d\) for \(B_{s}\) and \(B_{d}\) mixing respectively. The following measurements at 95% CL limits are used in this work [57],

$$\begin{aligned} C_{B_{s}}&= 1.110 \pm 0.090 [0.942, 1.288],\\ \phi _{B_{s}}^o&= 0.42 \pm 0.89 [-1.35, 2.21]\\ C_{B_{d}}&= 1.05 \pm 0.11 [0.83, 1.29], \\ \phi _{B_{d}}^o&= -2.0 \pm 1.8 [ -6.0, 1.5]. \end{aligned}$$

The new physics contributions to neutral meson mixing can be written as,

$$\begin{aligned} M_{12}^{d,s,K} = (M_{12}^{d,s,K} )_{\textrm{SM}} \left( 1 + h_{d,s,K} e^{2 i \sigma _{d,s,K}} \right) . \end{aligned}$$
(44)

We assume the minimum flavour violation scenario that corresponds to \(\sigma _{d,s,K} =0\). We adopt the following future sensitivity phases in this work [63]:

  1. 1.

    Phase I which is \(50~\text {fb}^{-1}\) LHCb and \(50~\text {ab}^{-1}\) Belle II (late 2020s);

  2. 2.

    Phase II which is \(300~\text {fb}^{-1}\) LHCb and \(250~\text {ab}^{-1}\) Belle II (late 2030s).

The expected sensitivities to \(C_{\Delta m_K} \) and \(C_{B_{q}}\) in future phase I and II of LHCb and Belle II can be obtained from Table 7.

Table 7 Future projected sensitivity of the neutral meson mixing
Fig. 1
figure 1

The allowed parameter space by flavour observables \(C_{\epsilon _K}\) and \(C_{\Delta m_K}\) in the \(m_a - f\) plane for the minimal \(({\mathcal {Z}}_2 \times {\mathcal {Z}}_5)\) flavour symmetry. On the left panel in a, the allowed bounds for \(\lambda _\chi = 2\) with current limits for \(C_{\Delta m_K}\) and \(C_{\epsilon _K}\) are shown by red and violet boundaries, respectively. Also, the allowed bound with projected limits of \(C_{\Delta m_K}\) is shown with yellow boundary. The effect of the variation of the quartic coupling \(\lambda _\chi \) on the observable \(C_{\epsilon _K}\) is shown on the right panel in b

In Fig. 1, we show the bounds on the VEV of the flavon and the mass of the pseudo-scalar flavon arising due to the neutral kaon mixing observables \(C_{\epsilon _K}\) and \(C_{\Delta m_K}\) for the minimal model based on the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_5\) flavour symmetry. On the left, the allowed region by the observables \(C_{\epsilon _K}\) and \(C_{\Delta m_K}\) is shown for the quartic coupling \(\lambda _\chi = 2\). There is a sudden dip in the allowed parameter space given by \(C_{\epsilon _K}\) which appears due to a cancellation in the Wilson coefficients \(C_2^{ij}\) and \({\tilde{C}}_2^{ij} \) when masses of scalar and pseudoscalar flavon become identical. For \(C_{\Delta m_K}\), this dip is not visible in this plot, and is excluded for the quartic coupling for \(\lambda _\chi = 2\). The region bounded by the yellow curve is the allowed parameter space by the future projected sensitivity as shown in Table 7. On the right panel, we show the allowed regions of the parameter space by the observable \(C_{\epsilon _K}\) for \(\lambda _\chi = 0.5,~ 2,~ 4\pi \). It is observed that the allowed region shrinks as \(\lambda _\chi \) approaches smaller values.

Similar results for non-minimal model based on the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_9\) flavour symmetry are shown in Fig. 2. The constraints on the allowed parameter space by the neutral kaon mixing observables \(C_{\epsilon _K}\) and \(C_{\Delta m_K}\), in this case, are more stringent in comparison to that of the minimal \(({\mathcal {Z}}_2 \times {\mathcal {Z}}_5)\) flavour symmetry. In particular, we observe that the region with the sudden dip is excluded for the non-minimal model. Moreover, there is no allowed parameter space for the future projected sensitivity of the observable \(C_{\Delta m_K}\) for the bench-mark values of the couplings given in the Appendix. Therefore, we show these bounds for different values \((|y_{12}^d| = 1\) and \(|y_{21}^d| = \pi )\) of the couplings which are allowed by a more relaxed fit.

Fig. 2
figure 2

The allowed parameter space by flavour observables \(C_{\epsilon _K}\) and \(C_{\Delta m_K}\) in the \(m_a - f\) plane for the non-minimal \({\mathcal {Z}}_2 \times {\mathcal {Z}}_9\) flavour symmetry. On the left panel in a, the allowed bounds for \(\lambda _\chi = 2\) with current limits for \(C_{\Delta m_K}\) and \(C_{\epsilon _K}\) are shown by green and magenta boundaries, respectively. Also, the allowed bound with projected limits of \(C_{\Delta m_K}\) is shown with olive coloured boundary. The effect of the variation of the quartic coupling \(\lambda _\chi \) on the observable \(C_{\epsilon _K}\) is shown in the right panel in b

We show the allowed regions of parameter space by the \(B_s -\bar{B}_s \) mixing observables \(C_{B_{s}}\) and \(\phi _{B_{s}}\) for \(\lambda _\chi = 2\) in the \(m_a - f\) plane for the minimal model based on the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_5\) flavour symmetry and the non-minimal model based on the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_9\) flavour symmetry in Fig. 3. In the left panel in Fig. 3a, the red and yellow coloured boundaries are representing allowed flavon contribution for current values of \(C_{B_{s}}\) and \(\phi _{B_{s}}\), respectively in the minimal model, while that for the non-minimal model are shown in the right panel in Fig. 3b, by magenta and green coloured boundaries, respectively. We note that the region in the left panel, bounded by the blue curve, represents the allowed bounds for the observable \(C_{B_{s}}\) with projected limits of LHCb Phase-II for the minimal model, while the same for the non-minimal model is shown by the region surrounded by the olive coloured curve in the right panel. For the LHCb phase-I, the bounds are shown by the pink coloured curves for the minimal as well as for the non-minimal models, which are not appreciably different than that of the LHCb phase-II. We also notice an isolated allowed strip of the parameter space for the non-minimal model below the green boundary in the right panel. The effects of the observables \(C_{B_{s}}\) and \(\phi _{B_{s}}\) are relatively strong in the non-minimal model based on the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_9\) flavour symmetry, which is obvious from the figure itself.

Fig. 3
figure 3

The parameter space allowed by flavour observables \(C_{B_{s}}\) and \(\phi _{B_{s}}\) for \(\lambda _\chi = 2\) in the \(m_a - f\) plane for the minimal \(({\mathcal {Z}}_2 \times {\mathcal {Z}}_5)\) model in the left panel and for the non-minimal \(({\mathcal {Z}}_2 \times {\mathcal {Z}}_9)\) model in the right panel

Fig. 4
figure 4

The allowed parameter space by flavour observables \(C_{B_{d}}\) and \(\phi _{B_{d}}\) for \(\lambda _\chi = 2\) in the \(m_a - f\) plane for the minimal \(({\mathcal {Z}}_2 \times {\mathcal {Z}}_5)\) model in the left panel and for the non-minimal \(({\mathcal {Z}}_2 \times {\mathcal {Z}}_9)\) model in the right panel

Figure 4 shows the allowed parameter space by flavour observables \(C_{B_{d}}\) and \(\phi _{B_{d}}\) of the \(B_d -{{\bar{B}}}_d \) mixing for \(\lambda _\chi = 2\) in the \(m_a - f\) plane. This is shown for the minimal model on the left in Fig. 4a and for the non-minimal model on the right in Fig. 4b. In the left panel in Fig. 4a, the red and yellow coloured boundaries are representing allowed flavon contribution for current values of \(C_{B_{d}}\) and \(\phi _{B_{d}}\), respectively for the minimal model based on the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_5\) symmetry while that for non-minimal model is shown in the right panel in Fig. 4b by green and purple coloured boundaries, respectively. Moreover, the region surrounded by the blue curve in the left panel shows the allowed parameter space for the observable \(C_{B_{d}}\) with projected limits of LHCb Phase-II for the minimal model while the same for the non-minimal model is shown by olive coloured boundary in the right panel. The bounds for the LHCb phase-I are shown by the pink coloured boundaries for the minimal as well as for the non-minimal model, and are very similar to that of the LHCb phase-II.

The SM contribution to \(D^0 - {{\bar{D}}}^0\) mixing is marred by large hadronic uncertainties. Therefore, for constraining the parameter space of our model, we keep only the flavon contribution to \(D^0 - {{\bar{D}}}^0\) mixing such that it always lies within the \(2\sigma \) experimental bound [65]:

$$\begin{aligned} |M_{12}^D|= |\langle D^0|{\mathcal {H}}^{\Delta F=2}|{{\bar{D}}}^0\rangle | < 7.5 \times 10^{-3}~\text {ps}^{-1}. \end{aligned}$$
(45)

The bound in the \(m_a - f\) panel arising from the \(D^0 - \bar{D}^0\) mixing is shown in Fig. 5 for the minimal model based on the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_5\) flavour symmetry and the non-minimal model based on the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_9\) flavour symmetry. The first remarkable observation is the allowed parameter space for the minimal model is much smaller than that of the non-minimal model. This is because the \(D^0 - {{\bar{D}}}^0\) mixing has an enhancement of the order \(\epsilon ^2\) (see the coupling \(2 y_{21}^u \epsilon ^2 \) in Eq. 36) in the minimal model based on the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_5\) flavour symmetry, which is not present in the case of the non-minimal model. Therefore, the bound derived by the \(D^0 - {{\bar{D}}}^0\) mixing is the most stringent bound among the bounds given by the mixing observables in the minimal model. However, this is not the case for the non-minimal model.

The one-loop contribution to this mixing from the box diagram depends on the relatively large \(y_{ct}\) and \(y_{tc}\) couplings of the flavon to fermions. In the minimal model, \(y_{tc}\) is zero. Thus, this contribution is proportional to \(\epsilon ^4 /(4 \pi ^2 f^2)\) in the minimal model and proportional to \(\epsilon ^{16} /(4 \pi ^2 f^2)\) in the non-minimal model. Therefore, this contribution is highly suppressed with respect to the tree-level contribution used in deriving the bounds in Fig. 5.

Fig. 5
figure 5

The allowed parameter space by \(|M_{12}^D|\) for \(\lambda _\chi = 2\) in the \(m_a - f\) plane for the minimal \(({\mathcal {Z}}_2 \times {\mathcal {Z}}_5)\) model is shown by the magenta coloured boundary while that for the non-minimal \(({\mathcal {Z}}_2 \times {\mathcal {Z}}_9)\) model is shown by the red coloured boundary

5.2 Leptonic decays of mesons

The effective Hamiltonian for flavon mediated decays of neutral mesons into two charged leptons can be written as,

$$\begin{aligned} {\mathcal {H}}_{\text {eff}}&=-\frac{G_F^2m_W^2}{\pi ^2}\,\left( C_S^{ij}\, ({{\bar{q}}}_iP_L q_j){{\bar{\ell }}} \ell +{\tilde{C}}_S^{ij}\, ({{\bar{q}}}_iP_R q_j){{\bar{\ell }}} \ell \right. \nonumber \\&\quad \left. + C_P^{ij}({{\bar{q}}}_iP_L q_j){{\bar{\ell }}} \gamma _5 \ell +{\tilde{C}}_P^{ij}({{\bar{q}}}_iP_R q_j){{\bar{\ell }}} \gamma _5 \ell \right) + \text {H.c.} \end{aligned}$$
(46)

The branching ratio of a meson decaying to two charged leptons reads,

$$\begin{aligned}&\text {BR}(M\rightarrow \ell ^+ \ell ^- )\nonumber \\&\quad = \frac{G_F^4m_W^4}{8\pi ^5} \beta \,m_M f_M^2 m_\ell ^2 \tau _M \nonumber \\&\times \left( \left| \frac{m_M^2\big (C_P^{ij}-\tilde{C}^{ij}_P\big )}{2m_\ell (m_i+m_j)}-C_A^{\text {SM}}\right| ^2+ \left| \frac{m_M^2\big (C_S^{ij}-{\tilde{C}}^{ij}_S\big )}{2m_\ell (m_i+m_j)}\right| ^2\beta ^2 \right) , \end{aligned}$$
(47)

where \(\beta (x)=\sqrt{1-4x^2}\) with \(x = m_\ell /m_M\).

The Wilson coefficients having tree-level contribution of the flavon are given as [59, 60],

$$\begin{aligned} C_S^{ij}&=\frac{\pi ^2}{2G_F^2 m_W^2}\frac{2y_{\ell \ell }y_{ji}}{m_s^2} \nonumber \\ {\tilde{C}}_S^{ij}&=\frac{\pi ^2}{2G_F^2 m_W^2}\frac{2y_{\ell \ell }y_{ij}}{m_s^2} \nonumber \\ C_P^{ij}&=\frac{\pi ^2}{2G_F^2 m_W^2}\frac{2y_{\ell \ell }y_{ji}}{m_a^2} \nonumber \\ {\tilde{C}}_P^{ij}&=\frac{\pi ^2}{2G_F^2 m_W^2}\frac{2y_{\ell \ell }y_{ij}}{m_a^2}. \end{aligned}$$
(48)

In the SM, processes of mesons decaying to two charged leptons are induced by one-loop contribution, and for the \(B_s\) meson it is given by [60],

$$\begin{aligned} C_A^{\text {SM}}= -V_{tb}^*V_{ts}\,Y\left( \frac{m_t^2}{m_W^2}\right) -V_{cb}^*V_{cs}\,Y\left( \frac{m_c^2}{m_W^2}\right) , \end{aligned}$$
(49)

where Inami–Lim function Y(x) is given by [66],

$$\begin{aligned} Y(x)=\eta _{\text {QCD}} \frac{x}{8}\left[ \frac{4-x}{1-x}+\frac{3x}{(1-x)^2} \log x \right] , \end{aligned}$$
(50)

where \(\eta _{\text {QCD}}=1.0113\) includes NLO QCD effects [67]. For \(B_d\) meson, the SM predictions are obtained by a simple replacement of indices in Eq. 49.

The average of the branching fraction of \(B_{s} \rightarrow \mu ^+\mu ^-\) from HFLAV group is [68],

$$\begin{aligned} \text {BR}(B_s \rightarrow \mu ^+\mu ^-) =(3.45 \pm 0.29 ) \times 10^{-9}. \end{aligned}$$
(51)

The latest measurement of the branching fraction of \(B_{d} \rightarrow \mu ^+\mu ^-\) is [69, 70],

$$\begin{aligned} \text {BR}(B_d\rightarrow \mu ^+\mu ^-) < 2.6 \times 10^{-10}. \end{aligned}$$
(52)

As observed in Ref. [71], due to sizeable width difference, of the \(B_s\) meson, theoretical branching ratio can be converted to experimental branching ratio by multiplying \((1-y_s)^{-1}\), where \(y_s=0.088\pm 0.014\) [72]. This correction is negligible in the case of the \(B_d\) meson.

Table 8 The current and expected experimental precision for rare B decays observables where LHCb-I corresponds to \(23~\text {fb}^{-1}\), LHCb-II corresponds to \(300~\text {fb}^{-1}\), CMS and ATLAS correspond to \(3~\text {ab}^{-1}\) [73, 74]

In addition to \(\text {BR}(B_s\rightarrow \mu ^+\mu ^-) \) branching ratio, the LHCb collaboration has also measured the ratio of the \(\text {BR}(B_d\rightarrow \mu ^+\mu ^-) \) and \(\text {BR}(B_s\rightarrow \mu ^+\mu ^-) \) branching fractions, \( {\mathcal {R}}_{\mu \mu } \) [69, 70]. The CMS has measured the effective lifetime, \(\tau _{\mu \mu }\), of the \(B_s\rightarrow \mu ^+\mu ^- \) decay [73]. We note that the ratio \( {\mathcal {R}}_{\mu \mu } \) is an excellent observable to probe the minimal flavour violation [74]. On the other side, the effective lifetime, \(\tau _{\mu \mu }\), can be used to discriminate between the contributions due to any possible new scalar and pseudoscalar mediators [74]. The measured value of the ratio of branching fractions, \( {\mathcal {R}}_{\mu \mu } \), is [69, 70],

$$\begin{aligned} {\mathcal {R}}_{\mu \mu }= & {} \frac{\text {BR}(B_d\rightarrow \mu ^+\mu ^-) }{\text {BR}(B_s\rightarrow \mu ^+\mu ^-)} \nonumber \\= & {} 0.039^{+0.030+0.006}_{-0.024-0.004}. \end{aligned}$$
(53)

The effective lifetime \(\tau _{\mu \mu }\) and the branching fraction of \(B_s\rightarrow \mu ^+\mu ^-\) are also measured by CMS, and are [73],

$$\begin{aligned} \tau _{\mu \mu }= & {} 1.83^{+0.23+ 0.04}_{-0.20-0.04}~\text {ps}, \end{aligned}$$
(54)
$$\begin{aligned} \text {BR}(B_s\rightarrow \mu ^+\mu ^-)= & {} 3.83^{+0.38+ 0.19+0.14}_{-0.36-0.16-0.13} \times 10^{-9}. \end{aligned}$$
(55)

We have also taken HFLAV measurement average into account for effective lifetime \(\tau _{\mu \mu }\), which is [68],

$$\begin{aligned} \tau _{\mu \mu } = 2.00^{+0.27}_{-0.26} ~\text {ps}, \end{aligned}$$
(56)

with \(\text {BR}(B_s\rightarrow \mu ^+\mu ^-)\) given in Eq. 51.

The current and future sensitivities of these observables are summarized in Table 8. The effective lifetime can be written in the following form [75],

$$\begin{aligned} \tau _{\mu \mu } = \tau _{B_s} \frac{(B_s\rightarrow \mu ^+\mu ^-)^\textrm{experiment}}{(B_s\rightarrow \mu ^+\mu ^-)^{\textrm{theory}}}, \end{aligned}$$
(57)

where we have assumed the SM value of the final state dependent observable \({\mathcal {A}}^f_{\Delta \Gamma } =1\) [72].

Fig. 6
figure 6

The allowed parameter space by \(\text {BR}(B_d\rightarrow \mu ^+\mu ^-)\) in the left panel and that for the \(\text {BR}(B_s\rightarrow \mu ^+\mu ^-)\) in the right panel for \(\lambda _\chi = 2\) with current experimental bounds for the minimal and the non-minimal models

In Fig. 6, the bounds coming from the branching ratios \(\text {BR}(B_d\rightarrow \mu ^+\mu ^-)\) and the \(\text {BR}(B_s\rightarrow \mu ^+\mu ^-)\) for \(\lambda _\chi = 2\) in the \(m_a - f\) plane are shown. The branching ratios \(\text {BR}(B_d\rightarrow \mu ^+\mu ^-)\) and \(\text {BR}(B_s\rightarrow \mu ^+\mu ^-)\) place weaker constraints on the parameter space of the minimal model. However, for the non-minimal model, the bounds, particularly from the branching ratios \(\text {BR}(B_s\rightarrow \mu ^+\mu ^-)\), are quite stronger. For the projected sensitivities of the LHCb Phase-I and II and of the ATLAS for the \(\text {BR}(B_s\rightarrow \mu ^+\mu ^-)\), we do not obtain any appreciable improvements in our bounds. Therefore, we do not show them in Fig. 6.

Fig. 7
figure 7

The allowed parameter space by \( \tau _{\mu \mu } \) for \(\lambda _\chi = 2\) in the \(m_a - f\) plane for the recent measurement and the future projected sensitivity of the LHCb for the minimal \(({\mathcal {Z}}_2 \times {\mathcal {Z}}_5)\) model on the left, and for the non-minimal \(({\mathcal {Z}}_2 \times {\mathcal {Z}}_9)\) model on the right

Figure 7 shows the allowed bounds derived from the HFLAV average of the effective lifetime \( \tau _{\mu \mu } \) and the branching-ratio \(\text {BR}(B_s\rightarrow \mu ^+\mu ^-)\) by the green coloured curve for the minimal model in Fig. 7a and through the olive coloured boundary for the non-minimal model in Fig. 7b. The bounds from the current measurement by CMS, and from the future projected sensitivity of the LHCb phase-II for the minimal model based on \({\mathcal {Z}}_2 \times {\mathcal {Z}}_5\), are shown by red and yellow coloured boundaries respectively, in Fig. 7a. The same bounds for the non-minimal model based on the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_9\) flavour symmetry are shown in Fig. 7b by green and magenta coloured curves respectively. We observe that the bounds are stronger for the non-minimal model in comparison to that of the minimal model. For the projected sensitivity of the LHCb Phase-I, we do not find any appreciable improvement in the allowed bounds over that from the current measurement. Therefore, we do not show it in Fig. 7.

Fig. 8
figure 8

The left panel represents the allowed parameter space by \( {\mathcal {R}}_{\mu \mu } \) for the minimal \(({\mathcal {Z}}_2 \times {\mathcal {Z}}_5)\) model with \(\lambda _\chi = 2\) with the current measurement and for the future projected sensitivity of the LHCb Phase-I and LHCb Phase-II. The same allowed parameter space for the non-minimal \(({\mathcal {Z}}_2 \times {\mathcal {Z}}_9)\) model is shown in the right panel

One of the most important observables for our models is the ratio \( {\mathcal {R}}_{\mu \mu } \) whose future projected sensitivity will be crucial to constrain our models. On the left panel in Fig. 8a, we show the bounds arising from the ratio \( {\mathcal {R}}_{\mu \mu } \) for the minimal model. These bounds are weaker for the current measurement as well as for the future projected sensitivity of the LHCb Phase-I shown by the red and yellow boundaries, respectively. However, the future projected sensitivity of the LHCb Phase-II dramatically changes this scenario and provide extremely stringent constraints on the parameter space of the minimal model shown by the purple coloured strip. Therefore, the LHCb Phase-II will be decisive for the minimal model based on the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_5\) flavour symmetry. We also do not find any improvement over the bounds given by the LHCb Phase-I using the future projected sensitivity of the CMS experiment. Therefore, we do not show it in Fig. 8a.

On the other hand, for the non-minimal model based on the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_9\) symmetry, the bounds from the ratio \( {\mathcal {R}}_{\mu \mu } \) are shown in Fig. 8b in the right panel. The bounds from the current measurement are shown by the green boundary while the bounds from the LHCb Phase-I are surrounded by the purple coloured curve. Moreover, we also have bounds from the future projected sensitivity of the CMS experiment surrounded by olive coloured boundary. These are more stringent than that of the projected sensitivity of the LHCb Phase-I. We note that similar to the minimal model, the bounds for the projected sensitivity of the LHCb Phase-II are highly stringent depicted by the orange coloured strip. This result does not change even if we deviate from the bench-mark values of the Yukawa couplings used for these bounds. Therefore, these bounds are robust, and crucial to test the parameter space of the non-minimal model based on the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_9\) symmetry in the future projected sensitivity of the LHCb Phase-II.

For the \(K_L \rightarrow \mu ^+ \mu ^-\) decay, we have only reliable estimate of the so-called short distance (SD) part of \(K_L \rightarrow \mu ^+ \mu ^-\) decay [59]. We use the SM prediction obtained in Ref. [59] and given by

$$\begin{aligned} C_A^{\text {SM}}= -V_{ts}^*V_{td}\, Y\left( \frac{m_t^2}{m_W^2}\right) - V_{cs}^* V_{cd} Y_{\textrm{NNL}}, \end{aligned}$$
(58)

where at NNLO \(Y_{\textrm{NNL}} = \lambda ^4 P_c(Y)\), \(\lambda = |V_{us}|\) and \(P_c(Y) = 0.113 \pm 0.017\) [76]. The short distance contribution can be extracted from the experimental measurement and has an upper limit [60],

$$\begin{aligned} \text {BR}(K_L\rightarrow \mu ^+\mu ^-)_{\textrm{SD}} < 2.5 \times 10^{-9}. \end{aligned}$$
(59)

For the case of \(D \rightarrow \mu ^+ \mu ^-\) decay, the SM contribution is plagued by large non-perturbative effects. Therefore, we only require that the flavon contribution does not generate more than the experimental upper bound on the branching ratio that is given at \(90\%\) C.L. [77],

$$\begin{aligned} \text {BR}(D \rightarrow \mu ^+\mu ^-) < 6.2 \times 10^{-9}. \end{aligned}$$
(60)
Fig. 9
figure 9

The parameter space allowed by \(\text {BR}(K_L\rightarrow \mu ^+\mu ^-)_{\textrm{SD}}\) with \(\lambda _\chi = 2\) in the \(m_a - f\) plane for the minimal \(({\mathcal {Z}}_2 \times {\mathcal {Z}}_5)\) and non-minimal \(({\mathcal {Z}}_2 \times {\mathcal {Z}}_9)\) model are shown with red and magenta coloured boundaries, respectively

We show bounds arising from the \(\text {BR}(K_L\rightarrow \mu ^+\mu ^-)_\textrm{SD}\) for \(\lambda _\chi = 2\) for the minimal and the non-minimal models in the \(m_a - f\) plane in Fig. 9. It turns out that the constraints on the parameter space of the minimal model are weaker. However, the bounds on the parameter space of the non-minimal model are more stringent. The constraints from the \(D \rightarrow \mu ^+ \mu ^-\) decay are much weaker than that from the \(\text {BR}(K_L\rightarrow \mu ^+\mu ^-)_{\textrm{SD}}\). Therefore, we do not show them in this work.

6 Leptonic flavour physics in the minimal and non-minimal \({\mathcal {Z}}_2 \times {\mathcal {Z}}_N\) flavour symmetry

Charged lepton flavour violation (CLFV) has a substantial potential to provide bounds on flavon physics in future upcoming experiments. The quark flavour constraints currently dominate our flavon model. However, it is expected that the future projected sensitivities of CLFV, which are shown in Table 9, may significantly improve the bounds from quark flavour physics.

Table 9 Experimental upper limits on various Leptonic flavour violation (LFV) processes

6.1 Radiative leptonic decays

The effective Lagrangian for the radiative leptonic decays can be written as,

$$\begin{aligned} {\mathcal {L}}_{\text {eff}}&=m_{\ell '}\, C_T^L\,{{\bar{\ell }}} \sigma ^{\rho \lambda }P_L\,\ell ' \,F_{\rho \lambda }\nonumber \\&\quad +m_{\ell '}\, C_T^R\,{{\bar{\ell }}} \sigma ^{\rho \lambda }P_R\,\ell '\,F_{\rho \lambda }. \end{aligned}$$
(61)

The radiative leptonic decays are mediated by dipole operators and their branching ratio reads,

$$\begin{aligned} \text {BR}(\ell '\rightarrow \ell \gamma )=\frac{m_{\ell '}^5}{4\pi \Gamma _{\ell '}}\left( |C_T^L|^2+|C_T^R|^2\right) . \end{aligned}$$
(62)

The one-loop contribution to the radiative leptonic decays is shown in Fig. 10. The corresponding Wilson coefficients are [35],

$$\begin{aligned} C_T^L&= (C_T^R)^* =\frac{e}{32\pi ^2}\sum _{k=e,\mu ,\tau } \left\{ \frac{1}{6}\left( \,y^*_{\ell k}y_{\ell ' k}+\frac{m_\ell }{m_k}y^*_{k \ell }y_{k\ell ' }\right) \right. \nonumber \\&\quad \times \left( \frac{1}{m_s^2}- \frac{1}{m_a^2}\right) -y_{\ell k}y_{k\ell '}\frac{m_k}{m_{\ell '}} \left[ \frac{1}{m_s^2} \left( \frac{3}{2}+\log \frac{m_{\ell '}^2}{m_s^2}\right) \right. \nonumber \\&\quad \left. \left. -\frac{1}{m_a^2} \left( \frac{3}{2}+ \log \frac{m_{\ell '}^2}{m_a^2} \right) \right] \right\} . \end{aligned}$$
(63)
Fig. 10
figure 10

Feynman diagram representing \(\mu \rightarrow e \gamma \) decay

Fig. 11
figure 11

In the left panel, the red and yellow coloured boundaries represent the allowed parameter space by the current and projected sensitivities of the MEG experiment on the \(\text {BR}(\mu \rightarrow e \gamma )\) for the minimal \(({\mathcal {Z}}_2 \times {\mathcal {Z}}_5)\) model with \(\lambda _\chi = 2\) in the \(m_a - f\) plane. The same bounds for the non-minimal \(({\mathcal {Z}}_2 \times {\mathcal {Z}}_9)\) model are shown in the right panel with green and magenta coloured curves, respectively

The radiative leptonic decays \(\mu \rightarrow e \gamma \) places weak constraints on the parameter space of the minimal model as shown in Fig. 11a. This does not change much even for the future projected sensitivities of the MEG-II experiment. For the non-minimal model, the bounds from the current measurement of the MEG experiment, shown in Fig. 11b by the green boundary, are quite stringent relative to that of the minimal model. Moreover, for the future projected sensitivities of the MEGII experiment, the constraints are further stronger depicted by the magenta coloured boundary. The constraints coming from the decays \( \tau \rightarrow e \gamma \) and \( \tau \rightarrow \mu \gamma \) are weaker than that of the decay \(\mu \rightarrow e \gamma \). Therefore, they are not shown in this work.

6.2 \(A~\mu \rightarrow A~e\) conversion

The effective Lagrangian describing \(A~ \mu \rightarrow A ~e\) conversion can be written as,

$$\begin{aligned} {\mathcal {L}}_{\text {eff}}&= C_{qq}^{VL}\,{{\bar{e}}} \gamma ^\nu P_L \mu \, {{\bar{q}}} \gamma _\nu q +m_\mu m_q\,C_{qq}^{SL}{{\bar{e}}} P_R \mu \, {{\bar{q}}} q\nonumber \\&\quad +m_\mu \alpha _s C_{gg}^L\,{{\bar{e}}} P_R \mu \,G_{\rho \nu }G^{\rho \nu }\,+ (R\leftrightarrow L), \end{aligned}$$
(64)

Moreover, there is additional contribution to \(A~ \mu \rightarrow A~ e\) conversion from the dipole operators given in Eq. (61).

The Feynman diagram for \(A~ \mu \rightarrow A~ e\) conversion is shown in Fig. 12. The corresponding Wilson coefficients arise due to the diagram on the left in Fig. 12 [35],

$$\begin{aligned} C^{SL}_{qq}&=\left( \frac{1}{m_s^2}+\frac{1}{m_a^2}\right) y_{\mu e}^*\text {Re}(y_{qq}),\nonumber \\ C^{SR}_{qq}&=\left( \frac{1}{m_s^2}-\frac{1}{m_a^2}\right) y_{ e\mu } \text {Re}(y_{qq}). \end{aligned}$$
(65)

The nuclear effects which include effects of quarks inside the nucleons as well as the contribution of the Feynman diagram on the right side of Fig. 12 are absorbed in the nucleon-level Wilson coefficients defined by,

$$\begin{aligned} {\tilde{C}}_p^{VL}&=\sum _{q=u,d} C^{VL}_{qq}\, f_{V_q}^p, \nonumber \\ {\tilde{C}}_p^{SL}&=\sum _{q=u,d,s} C^{SL}_{qq}\, f_{q}^p-\sum _{Q=c,b,t} C^{SL}_{QQ} \,f_{\text {heavy}}^p, \end{aligned}$$
(66)

where the quark content of the proton is accounted by vector and scalar couplings \(f_{V_q}^{p}, f_q^p\), and \(f_{\text {heavy}}^p=2/27\big (1-f_u^p-f_d^p-f_s^p\big )\) [92]. For right-handed operators, analogous expressions are obtained by replacing L with R, and for the neutron p is replaced by n. The vector operators contribute extremely less than the scalar operators and can be neglected [35]. The numerical values of vector and scalar couplings are taken from Refs. [93, 94], which are based on the lattice average given in Ref. [95],

$$\begin{aligned} f_u^p&=0.0191 \qquad f_u^n=0.0171,\nonumber \\ f_d^p&=0.0363 \qquad f_d^n=0.0404,\nonumber \\ f_s^p&=f_s^n=0.043. \end{aligned}$$
(67)
Fig. 12
figure 12

Feynman diagram showing \(A~ \mu \rightarrow A~ e\) conversion

The \(A~ \mu \rightarrow A~ e\) conversion rate including nuclear effects can be written as [35],

$$\begin{aligned} \Gamma _{A ~\mu \rightarrow A~ e}&=\frac{m_\mu ^5}{4}\left| C_T^L D +4\left[ m_\mu m_p{\tilde{C}}_p^{SL}+{\tilde{C}}_p^{VL}V^p\right. \right. \nonumber \\&\quad \left. \left. + (p\rightarrow n) \right] \right| ^2\, + L \rightarrow R, \end{aligned}$$
(68)

where the dimensionless coefficients \(D, S^{p,n}\) and \(V^{p,n}\) depend on the overlap integrals of the initial state muon and the final-state electron wave-functions with the target nucleus, and their numerical values are given in Table 10 [96],

Table 10 Numerical values of the dimensionless coefficients \(D, S^{p,n}\), \(V^{p,n}\) and the muon capture rate for different nuclei
Fig. 13
figure 13

The left panel represents the allowed parameter space by \(\text {BR}(\mu \rightarrow e)\) conversion in the \(m_a - f\) plane with different experimental limits of various target nuclei in Table 9 with \(\lambda _\chi = 2\), for the minimal \(({\mathcal {Z}}_2 \times {\mathcal {Z}}_5)\) model. Similar allowed parameter space for the non-minimal \(({\mathcal {Z}}_2 \times {\mathcal {Z}}_9)\) model is shown in the right panel

The bounds from \(\text {BR}(\mu \rightarrow e)\) conversion for different target nucleus with \(\lambda _\chi = 2\) in the \(m_a - f\) plane are shown in Fig. 13. The bounds for the minimal model are shown in Fig. 13a for the current measurement and for the different future projected sensitivities, and the same for the non-minimal model are shown in Fig. 13b. The strongest bounds among them arise from the projected future sensitivities of the PRISM/PRIME experiment for the minimal as well as non-minimal models.

6.3 \(\mu \rightarrow 3e\) and \(\tau \rightarrow 3 \mu \) decays

The three body flavour violating leptonic decays \(\mu \rightarrow 3 e \) and \(\tau \rightarrow 3 \ell \) where \(\ell = e, \mu \) provide additional tests of the dipole operators given in Eq. (61). Their decay width can be written as [35],

$$\begin{aligned} \Gamma (\ell '\rightarrow 3 \ell )=\frac{ \alpha m_\ell '^5}{12 \pi ^2}\left| log \frac{{m_\ell '}^2}{{m_\ell }^2} - \frac{11}{4}\right| \left( |C_T^L|^2 + |C_T^R|^2\right) , \end{aligned}$$
(69)

where the tree-level contribution is ignored due to the strong chiral-suppression which is dominated by the logarithmic enhancement of the dipole operators [35]. Other contributions, such as Z-mediated penguin are strongly suppressed and ignored [97].

Fig. 14
figure 14

The allowed parameter space by \(\text {BR}( \mu \rightarrow 3e)\) for the minimal \(({\mathcal {Z}}_2 \times {\mathcal {Z}}_5)\) model on the left and for the non-minimal \(({\mathcal {Z}}_2 \times {\mathcal {Z}}_9)\) model on the right with \(\lambda _\chi = 2\)

In Fig. 14, we show the allowed parameter space by \(\text {BR}( \mu \rightarrow 3e)\) for \(\lambda _\chi = 2\) in the \(m_a - f\) plane. For the minimal model, the bounds are shown in Fig. 14a. They are weaker for the current measurement and for the future projected sensitivity as well. In Fig. 14b, the same bounds on the allowed parameter space are shown for the non-minimal model. For the current measurement, the bound is given by the green boundary, and for the future projected sensitivity, it is depicted by the magenta coloured curve. As obvious from Fig. 14b, the bounds for the non-minimal model are quite stringent relative to the minimal model. The bounds from the decay \(\tau \rightarrow 3 \ell \) are much weaker and are not shown in this work.

Fig. 15
figure 15

Summarized significant bounds on the minimal \({\mathcal {Z}}_2 \times {\mathcal {Z}}_5 \) and the non-minimal \({\mathcal {Z}}_2 \times {\mathcal {Z}}_9 \) model in the left and the right panel respectively

Finally, we show a summary of the most relevant and stringent bounds on the parameter space of the minimal and the non-minimal models in Fig. 15.

7 Summary

We have discussed a simple FN mechanism in the framework of a new \({\mathcal {Z}}_2 \times {\mathcal {Z}}_N \) flavour symmetry. This symmetry is inspired by the extensively explored 2HDM and the MSSM, where the \({\mathcal {Z}}_2\) symmetry is an essential ingredient of the theoretical framework. We show that a minimal form of this symmetry, the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_5 \), is capable of providing an explanation to the charged fermion mass pattern and quark mixing along with a mechanism to predict neutrino masses and mixing angles. However, for the minimal model based on the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_5 \) flavour symmetry, all the Yukawa couplings are not order one, which is a preferred choice in literature. This observation leads to a non-minimal model based on the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_9 \) flavour symmetry, where all the couplings are order one. The FN mechanism created through the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_N \) flavour symmetry is different from the conventional FN mechanism, where a continuous U(1) symmetry is employed to achieve a solution of the flavour problem of the SM.

The leading question is the scale where the flavour symmetry \({\mathcal {Z}}_2 \times {\mathcal {Z}}_N \) is broken. This is addressed by deriving the bounds on the parameter space of the model using flavour physics data. Moreover, future sensitivities of the CMS and the phase-I and II of the LHCb for the flavour physics observables turn out to be an interesting and fertile ground to investigate the breaking scale of the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_N \) flavour symmetry. Particularly the reach of the experiments such as MEG II and PRISM/PRIME, which are going to test lepton flavour violating effects, could play a crucial role in improving the present limits by orders of magnitude.

For the minimal model based on the flavour symmetry \({\mathcal {Z}}_2 \times {\mathcal {Z}}_5 \), the quark flavour physics plays a crucial role. For instance, the most stringent constraints on the parameter space of the minimal model comes from the \(K^0-{{\bar{K}}}^0\) and \(D^0 - {{\bar{D}}}^0\) mixing using the present data. In particular, the \(D^0 - {{\bar{D}}}^0\) provides the very tight bounds since it is highly unsuppressed due to the PMS in the minimal model. The \(B_d -{{\bar{B}}}_d\) mixing provides tighter bounds than the \(B_s -{{\bar{B}}}_s\) mixing. The future projected sensitivity of the \(B_d -{{\bar{B}}}_d\) mixing will further constrain the parameter space of the minimal model in future. The rare decays \(\text {BR}(B_{d,s}\rightarrow \mu ^+\mu ^-) \) branching ratios place relatively weaker bounds on the parameter space of the minimal model for the present measurements as well as for the future projected sensitivities of phase-I and II of the LHCb. However, the future projected sensitivity of the rare B decays observable \( {\mathcal {R}}_{\mu \mu } \) provides robust and the most stringent bound on the parameter space of the minimal model. Among the leptonic observables, the future projected sensitivity of the PRISM/PRIME for the BR \((\mu \rightarrow e )^{\textrm{Ti}} \) will substantially be able to constrain the parameter space of the minimal model.

In the case of the non-minimal model based on the flavour symmetry \({\mathcal {Z}}_2 \times {\mathcal {Z}}_9 \), both the quark as well as leptonic flavour physics play decisive role in constraining the parameter space of the non-minimal model. On the quark flavour side, the most stringent bounds are coming from the \(K^0-{{\bar{K}}}^0\) mixing while from the leptonic flavour side, it comes from the radiative leptonic decay \(\mu \rightarrow e \gamma \). We observe that every flavour observable including the branching ratio of the \(K_L \rightarrow \mu ^+ \mu ^-\) decay, is able to provide important bounds on the parameter space of the non-minimal model. In particular, the future projected sensitivity of the rare B decays observable \( {\mathcal {R}}_{\mu \mu } \) for the LHCb and CMS experiments, will test the non-minimal model rigorously. Moreover, the future projected sensitivity of the PRISM/PRIME for the BR \((\mu \rightarrow e )^{\textrm{Ti}} \) will eliminate a large part of the allowed parameter space.

In short, quark flavour physics will determine the allowed parameter space of the minimal model, and leptonic flavour violating observables independently can further probe the parameter space up to a very high scale. For the non-minimal model based on the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_9\) flavour symmetry, quark and lepton flavour physics play a crucial role in constraining the allowed parameter space of the model. The future sensitivities of the experiments such as MEG II, Mu3e, DeeMe, COMET, Mu2e and PRISM/PRIME will be able to constrain the parameter space of the non-minimal model based on the \({\mathcal {Z}}_2 \times {\mathcal {Z}}_9\) flavour symmetry. On the quark side, the \(B_d -{{\bar{B}}}_d\) mixing in the future phase-I and II of the LHCb will be able to eliminate a sufficient region of the parameter space. In future phase-II of the LHCb, the ratio \( {\mathcal {R}}_{\mu \mu } \) will be crucial in ruling out the major part of the flavon parameter space. Thus, future projected sensitivities of the LHCb phase-I and II will play a defining role in determining the fate of the flavon models discussed in this work.