1 Introduction

Extensive work has been dedicated to the study of the electroweak phase transition in the standard model (SM) as well as in many of its extensions. This interest is based for a large part on the possibility that it might explain the baryon asymmetry in the universe and that this asymmetry could be produced at around the scale of the electroweak symmetry breaking in the primordial hot Big Bang universe (for reviews see, e.g., Refs. [13]). One of the necessary conditions for a model to explain the baryon asymmetry of the universe is the presence of nonequilibrium effects. In a phase transition, this can be achieved if the transition is first-order and its strength is strong enough, in what is usually called a strong first-order phase transition. This condition is parameterized by the ratio \(R=\langle \phi \rangle (T_\mathrm{c})/T_\mathrm{c}\), where \(\langle \phi \rangle (T_\mathrm{c})\) is the value for the degenerate vacuum for the Higgs field at the critical temperature \(T_\mathrm{c}\). A strong first-order phase transition is usually characterized by the condition \(R>1\). In the SM this condition cannot be achieved. Lattice Monte Carlo numerical simulations of the electroweak standard model [46] have shown that there is an endpoint in the phase diagram of the model for a Higgs mass \(m_\mathrm{H} \sim 80\) GeV, where the phase transition is weak first-order as the endpoint is approached from the left and the transition becomes a smooth crossover for larger Higgs masses. According to recent results from the Large Hadron Collider (LHC), from the current combined results from ATLAS and CMS experiments [7] have indicated the existence of a Higgs boson with a mass \(125.1 \pm 0.3\) GeV. Thus, this gives no hope of achieving the necessary conditions for producing a baryon asymmetry in the context of the SM, since no significant departure from thermal equilibrium can be obtained during the phase transition dynamics. This is one of the motivations for looking for extensions of the SM and/or alternative models and the searches for new scalar particles at the LHC, aiming to reveal the ingredients needed for the strong first-order electroweak phase transition (EWPT), as required to produce the resulting observed baryon asymmetry.

On the theoretical side, some extensions of the SM have been analyzed and the kind of scalar was selected so as to remedy the SM shortcomings. These extensions used to enhance the SM are usually constructed with a scalar gauge singlet [8], a complex scalar or a scalar from supersymmetric degrees of freedom (in the context of supersymmetry extensions of the SM) [9]. On the other hand, there are alternative models, with a larger particle spectrum than the SM, that predict the existence of new gauge bosons and exotic quarks that acquire mass from their couplings to new scalar fields. In particular, in this paper, we are exploring the phenomenological aspects of an alternative to the SM based on the \(SU(3)_c \times SU(3)_L \times U(1)_X\) gauge symmetry, commonly known as the 3-3-1 model [10, 11]. In this model, the scalars are accommodated in a convenient fundamental representation of the \(SU(3)_L\) symmetry group. From the electric charge operator one can select its model version. One particular version predicts the existence of new very massive gauge bosons and exotic quarks. In this work, we want to study and better understand the possible phase transition sequences associated with the symmetry breaking pattern \(SU(3)_L \otimes U(1)_X \rightarrow SU(2)_L \otimes U(1)_Y \rightarrow U(1)_{EM}\) in the 3-3-1 model and whether it can produce the necessary conditions required for generating a baryon asymmetry.

Let us outline some features of the model. Although at low energies the model has the same spectrum as the SM, it offers an explanation for basic open questions to the SM. In this model, the family replication problem is solved when considering that all three families are required for the anomaly cancellation procedure, resulting in the number of fermion families to be a multiple of the quark color number. Considering that the QCD asymptotic freedom condition is valid only if the number of families of quarks is less than five, one concludes that there are three generations. Another interesting feature of the minimal version of the model is the prediction of an upper bound for the Weinberg angle, which follows from a peculiar relation between new gauge boson masses.

The remainder of this paper is organized as follows. In Sect. 2, we present the main ingredients of the 3-3-1 model related to the spontaneous symmetry breaking (SSB). We give a brief exposition of its gauge, scalar, and fermionic sectors, with a description of its spectrum of particles and the main motivations for us seeing it as an interesting and natural extension of the standard model. We also introduce the more general potential compatible with the given symmetry. In Sect. 3, we obtain the scalar spectrum after the SSB, explicitly showing the combination of the scalar fields transferring mass to the massive gauge bosons. In Sect. 4, we give the expression for the quantum and thermal corrections at the one-loop order to the tree-level potential for the model. In Sect. 5, we analyze and characterize the structure of symmetry breaking patterns in the model and we discuss our strategy for fixing the many parameters of the model so as to maximize the possibility of finding a strong first-order phase transition. We study the temperature-dependent one-loop corrected potential as a function of each value expectation value of the background fields and we graphically identify the temperature corresponding to symmetry restoration. From this analysis of the temperature dependence of the one-loop corrected model spectrum, we conclude that, in the 3-3-1 model, it shows two scales for first-order phase transition, with the final one corresponding to the usual electroweak phase transition, as being very weak first-order, or probably second-order in practice. Finally, in Sect. 6, we give our concluding remarks.

2 The minimal version of the 3-3-1 model

In this section we recall the main characteristics of the minimal version of the 3-3-1 model [10, 11] related to the spontaneous symmetry breaking mechanism. We start by the definition of the electric charge operator,

$$\begin{aligned} Q = T_3 + \beta \ T_8 + X I, \end{aligned}$$
(1)

where \(T_3\) and \(T_8\) are two of the eight generators \(T_i\) (\(i=1, \ldots , 8\)) satisfying the SU(3) algebra, I is the unit matrix, and X denotes the U(1) charge. The minimal version of the model, used in this work, corresponds to the choice of the parameter \(\beta = - \sqrt{3}\).

To generate masses for all gauge and exotic quark fields through spontaneous symmetry breaking, three triplets of scalars, denoted by \(\eta , \rho \), and \(\chi \), respectively, are needed,

$$\begin{aligned}&\eta =\left( \eta ^{0} \ \eta _{1}^{-} \ \eta _{2}^{+} \right) ^\mathrm{T} , \nonumber \\&\rho =\left( \rho ^{+} \ \rho ^{0} \ \rho ^{++} \right) ^\mathrm{T}, \nonumber \\&\chi =\left( \chi ^{-} \ \chi ^{--} \ \chi ^{0}\right) ^\mathrm{T}. \end{aligned}$$
(2)

The deviations of these fields from their ground state configuration \(v_\eta , v_\rho \), and \(v_\chi \), are denoted by

$$\begin{aligned}&\eta ^{0}= v_\eta + \xi _{\eta }+ i \zeta _{\eta }, \nonumber \\&\rho ^{0}= v_\rho + \xi _{\rho } +i \zeta _{\rho }, \nonumber \\&\chi ^{0}= v_{\chi } + \xi _{\chi }+ i \zeta _{\chi }, \end{aligned}$$
(3)

where \(\xi _{\eta , \rho , \chi }\) and \(\zeta _{\eta , \rho , \chi }\) are the deviations for the real and imaginary components of the fields, respectively, and we assume that the neutral part of each scalar develops a nonzero real vacuum expectation value (VEV): \(\langle v_{\eta } \rangle = v_{\eta _0}\), \(\langle v_{\rho } \rangle = v_{\rho _0}\), and \( \langle v_{\chi } \rangle = v_{\chi _0}\). We impose the consistency of the model with the SM phenomenology by adopting \(v_{\chi _0} \gg v_{\rho _0}, v_{\eta _0}\) and \(v_{\rho _0}^2 + v_{\eta _0}^2 = v_W^2= \left( 246 \, \mathrm{GeV}\right) ^2\), where \(v_{\chi _0}\) gives the energy scale for the symmetry breaking \(SU(3)_L \otimes U(1)_X \rightarrow SU(2)_L \otimes U(1)_Y\), which is usually assumed to be at the TeV scale, for consistency with the current observations [12].

The gauge bosons, associated with the gauge symmetry \(SU(3)_L\) of the model, consist of an octet \(W^i_\mu \) (\(i = 1,\ldots , 8\)) and a singlet \(B_\mu \), associated with \(U(1)_X\). The model also predicts five vector bileptons: a single charged (\(V_\mu ^\pm \)), a doubly charged (\(Y_\mu ^{\pm \pm }\)) and a new neutral gauge boson (\(Z_\mu ^\prime \)), in addition to the charged standard model gauge bosons (\(W_\mu ^\pm \)), the neutral (\(Z_\mu \)), and the photon (\(A_\mu \)). These gauge bosons are defined as

$$\begin{aligned} W_\mu ^\pm= & {} \frac{1}{\sqrt{2}}\left( W_\mu ^1 \mp i W_\mu ^2\right) , \end{aligned}$$
(4)
$$\begin{aligned} V_\mu ^\pm= & {} \frac{1}{\sqrt{2}} \left( W_\mu ^4 \mp i W_\mu ^5 \right) , \end{aligned}$$
(5)
$$\begin{aligned} Y_\mu ^{\pm \pm }= & {} \frac{1}{\sqrt{2}} \left( W_\mu ^6 \mp i W_\mu ^7\right) , \end{aligned}$$
(6)
$$\begin{aligned} A_\mu= & {} \frac{1}{\sqrt{g^2+4 g^{{\prime }^2}}} \left[ g B_\mu + g^\prime \left( W_\mu ^3 + \sqrt{3} W_\mu ^8 \right) \right] , \end{aligned}$$
(7)
$$\begin{aligned} Z_\mu= & {} \frac{1}{\sqrt{g^2+4 g^{{\prime }^2}}}\frac{1}{\sqrt{g^2+3 g^{{\prime }^2}}} \nonumber \\&\times \left[ g g^\prime \ B_\mu + \sqrt{3} g^{{\prime }^2} W^8_\mu -\left( g^2 + 3 g^{{\prime }^2}\right) W^3_\mu \right] , \end{aligned}$$
(8)
$$\begin{aligned} Z^\prime _\mu= & {} \frac{1}{\sqrt{g^2+3 g^{{\prime }^2}}} \left[ g W^8_\mu + \sqrt{3} g^\prime B_\mu \right] , \end{aligned}$$
(9)

where g and \(g^\prime \) are the couplings defined in the covariant derivative of the scalar fields \(\Phi = \eta , \rho , \chi ,\)

$$\begin{aligned} D_\mu \Phi = \partial _\mu \Phi +i g W_\mu ^i T_i \Phi -i g^\prime B_\mu \Phi . \end{aligned}$$
(10)

The new gauge fields acquire mass at a high scale when the \(SU(3)_L \times U(1)_X\) group breaks down to \(SU(2)_L \times U(1)_Y\), by the \(\chi \) \(SU(3)_L\) scalar triplet, while the ordinary quarks and SM gauge bosons get acceptable masses at the next stage of SSB provided by the \(\eta \) and \(\rho \) triplets [13]. The resulting gauge boson tree-level field-dependent masses are given by

$$\begin{aligned}&M_W = \frac{g}{2} \sqrt{v_{{\eta _0}}^2 + v_{{\rho _0}}^2}, \end{aligned}$$
(11)
$$\begin{aligned}&M_V = \frac{g}{2} \sqrt{v_{{\eta _0}}^2 + v_{{\chi _0}}^2}, \end{aligned}$$
(12)
$$\begin{aligned}&M_Y =\frac{g}{2} \sqrt{v_{{\rho _0}}^2+ v_{{\chi _0}}^2}, \end{aligned}$$
(13)
$$\begin{aligned}&M_Z = \frac{g}{2 c_W} \sqrt{v_{{\eta _0}}^2 + v_{{\rho _0}}^2}, \end{aligned}$$
(14)
$$\begin{aligned}&M_{Z^\prime } = \frac{v_{\chi _0}}{\sqrt{3}}\ \sqrt{g^ 2 + 3 g^{\prime 2}}, \end{aligned}$$
(15)

where \(M_W/M_Z = c_W\), with \(s_W^ 2 = 1- c_W^2 = 0.223\) [14] and \(g^\prime \), corresponding to the \(U(1)_X\) gauge coupling, given by \(g^\prime = g \, s_W/\sqrt{1 - 4 s_W^2}\).

Let us mention that, if the leptons are to get their masses at tree level within the usual Higgs mechanism, their Yukawa couplings would require a scalar (S) belonging to a dimension six symmetric representation of the \(SU(3)_L\) group [15]. We do not evaluate the tiny lepton masses generated by SSB because they give a negligible contribution to the effective potential. Moreover, introducing a sextet scalar S with a background neutral field developing a VEV, say \(v_{\sigma _1} = \langle \sigma _1 \rangle \), would modify the previous relation between the field vacuum expectation values with the Weinberg scale to \(v_{\sigma _1}^2 + v_{\eta _0}^2 + v_{\rho _0}^2 = v_W^2= \left( 246 \,\mathrm{GeV}\right) ^2\), but keeping the adopted estimate \(v_\chi \gg v_\eta , v_\rho , v_{\sigma _1}\).

The quark content is embedded in the extended group according to the multiplets \(\displaystyle Q_{mL} = \left( d_m, u_m, j_m \right) ^\mathrm{T}_L\) and \(Q_{3 L} = \left( u_3, d_3, J \right) ^\mathrm{T}_L \), where the SM quarks are \(u_{1,2,3}\) and \(d_{1,2,3}\), whereas J, \(j_1\), and \(j_2\) are the exotic heavy quarks needed to complete the fundamental representation. We define the background-field dependence of the top and exotic quark masses as

$$\begin{aligned}&m_\mathrm{top}^2\left( v_\eta , v_\rho \right) = \frac{m_\mathrm{top}^2\left( v_W \right) }{v_W^2} \left( v_\eta ^2 + v_\rho ^2 \right) , \end{aligned}$$
(16)
$$\begin{aligned}&m_{Q}^2\left( v_\chi \right) = \frac{m_{Q}^2\left( v_{\chi _0} \right) }{v_{\chi _0}^2} v_\chi ^2. \end{aligned}$$
(17)

Finally, the scalar masses are obtained from the most general, gauge invariant, and renormalizable potential [13] for the scalar fields \(\eta , \rho \), and \(\chi \),

$$\begin{aligned} V\left( \eta , \rho , \chi \right)= & {} \mu _1^2\eta ^\dagger \eta + \mu _2^2\rho ^\dagger \rho + \mu _3^2\chi ^\dagger \chi + \lambda _1\left( \eta ^\dagger \eta \right) ^2 \nonumber \\&+ \lambda _2\left( \rho ^\dagger \rho \right) ^2 + \lambda _3\left( \chi ^\dagger \chi \right) ^2 \nonumber \\&+ \left[ \lambda _4\left( \rho ^\dagger \rho \right) + \lambda _5\left( \chi ^\dagger \chi \right) \right] \left( \eta ^\dagger \eta \right) \nonumber \\&+ \lambda _6\left( \rho ^\dagger \rho \right) \left( \chi ^\dagger \chi \right) + \lambda _7\left( \rho ^\dagger \eta \right) \left( \eta ^\dagger \rho \right) \nonumber \\&+ \lambda _8\left( \chi ^\dagger \eta \right) \left( \eta ^\dagger \chi \right) + \lambda _9\left( \rho ^\dagger \chi \right) \left( \chi ^\dagger \rho \right) \nonumber \\&+ \frac{1}{2} \left( f_1\epsilon ^{ijk}\eta _i\rho _j\chi _k + {\hbox {H. c.}}\right) . \end{aligned}$$
(18)

The tree-level potential, expressed in terms of the background fields \(v_\eta \), \(v_\rho \), and \(v_\chi \), is

$$\begin{aligned} V_\mathrm{tree}(v_\eta ,v_\rho ,v_\chi )= & {} \mu _1^2 v_\eta ^{2}+ \mu _2^2 v_\rho ^{2} + \mu _3^2 v_\chi ^{2}+ \lambda _{1} v_\eta ^{4} \nonumber \\&+ \lambda _2 v_\rho ^{4}+ \lambda _3 v_\chi ^{4} + \left( \lambda _4 v_\rho ^{2} + \lambda _5 v_\chi ^{2} \right) v_\eta ^{2} \nonumber \\&+ \lambda _6 v_\rho ^{2} v_\chi ^{2} + f_1 v_\eta v_\rho v_\chi . \end{aligned}$$
(19)

By following a similar choice of parameters as used, e.g., in Refs. [13, 16], we fix the trilinear coupling \( f_1\) as \( f_1 = - {\bar{f}}_1 v_{\chi _0}\), where \({\bar{f}}_1\) is a dimensionless constant. In particular, a common choice in the literature [13] is \({\bar{f}}_1=1\). The mass parameters \(\mu _{1,2,3}\) are determined by minimizing the tree-level potential in the vacuum, which gives

$$\begin{aligned} \mu _1^2= & {} \frac{{\bar{f}}_1}{2} \frac{v_{{\chi _0}}^2 v_{{\rho _0}}}{ v_{{\eta _0}}} - 2 \lambda _{{1}} v_{{\eta _0}}^ {2}-\lambda _{{4}} v_{{\rho _0}}^{2}-\lambda _{{5}} v_{{\chi _0}}^{2}, \end{aligned}$$
(20)
$$\begin{aligned} \mu _2^2= & {} \frac{{\bar{f}}_1}{2} \frac{v_{{\chi _0}}^{2} v_{{\eta _0}}}{v_{{\rho _0}}} - 2 \lambda _{{2}} v_{{\rho _0}}^{2} -\lambda _{{4}} v_{{\eta _0}}^{2}-\lambda _{{6}} v_{{\chi _0}}^{2}, \end{aligned}$$
(21)
$$\begin{aligned} \mu _3^2= & {} \frac{{\bar{f}}_1}{2} v_{\eta _0} v_{\rho _0} -2\lambda _{{3}} v_{{\chi _0}}^{2}-\lambda _{{5}} v_{{\eta _0} }^{2}-\lambda _{{6}} v_{{\rho _0}}^{2}. \end{aligned}$$
(22)

The potential (18) has a too large number of, in principle, free parameters, represented by the different possible magnitudes for the ten couplings, \(\lambda _i,\,f_1\), \(i=1,\ldots ,9\), the vacuum expectation values for the triplet scalars, \(v_{\chi _0}\), \(v_{\eta _0}\), and \(v_{\rho _0}\). Note that the constraint \(v_{\eta _0}^2 + v_{\rho _0}^2 = v_W^2\) only tells us that these two VEVs are related to the same scale (the Weinberg scale), but it does not fix the proportionality factor between them, i.e., we can parameterize \(v_{\eta _0}\) and \(v_{\rho _0}\) as \(v_{\eta _0} = \phi _0 \sin (\beta )\) and \(v_{\rho _0} = \phi _0 \cos (\beta )\), where \(\phi _0 = v_W\), but, in principle, with an arbitrary projection angle \(\beta \). Note that a natural choice is having \(v_{\eta _0} = v_{\rho _0}\), i.e., \(\beta =45^{\circ }\), however, in the literature there are some motivations for having \(v_{\eta _0} \ne v_{\rho _0}\); see, e.g., Refs. [17, 18]. Thus, we have a total of 12 free parameters for the scalar sector, composed of the ten couplings, the high energy scale \(v_{\chi _0}\) associated with the first symmetry breaking \(SU(3)_L \times U(1)_X \rightarrow SU(2)_L\times U(1)_Y\) and the projection angle \(\beta \). In the scalar sector we can still fix one of these parameters in terms of the others by making use of the Higgs mass \(m_\mathrm{H}\). The stability of the potential only constrains the possible values for the couplings. In particular, \(\lambda _{1,2,3}\) should be positive for overall stability of the potential in the \(\eta ,\,\rho \), and \(\chi \) directions, while the mixed couplings \(\lambda _{4,5,6,7,8}\) can in principle be negative.

We observe that the symmetry breaking scale for the electroweak theory down to \(U(1)_\mathrm{EM}\) is governed by \(v_W\). In addition, the lack of information as regards the individual roles of \(v_\eta \) and \(v_\rho \) fields in the SSB leads us to adopt, as already mentioned above, the polar parameterization \(v_{\eta } = \phi \sin (\beta )\) and \(v_{\rho } = \phi \cos (\beta )\), such that in the vacuum, \(\phi _0 = v_W\). Thus, after the second spontaneously symmetry breaking, where \(SU(2)_L\times U(1)_Y \rightarrow U(1)_{EM}\), it produces VEVs simultaneously for both \(\eta \) and \(\rho \), but with an in principle arbitrary projection angle \(\beta \).

3 Mass spectrum for the scalars

The scalar sector for the 3-3-1 model can be divided in CP-even and CP-odd scalar sectors. The CP-even and CP-odd scalar sectors are further composed of a neutral scalar mass matrix, two single charged scalar, and one double charged scalar matrices. For the CP-even scalar sector, the neutral scalar mass matrix is

$$\begin{aligned} M_\mathrm{neutral} = \left[ \begin{array}{lll} \mu _1^2+ 6 \lambda _1 v_\eta ^2 + \lambda _4 v_\rho ^2 +\lambda _5 v_\chi ^{2} &{}\quad 2 \lambda _4 v_\eta v_\rho - \frac{{\bar{f}}_1}{2} v_{\chi _0} v_\chi &{}\quad 2 \lambda _5 v_\eta \, v_\chi - \frac{{\bar{f}}_1}{2} v_{\chi _0} v_\rho \\ 2 \lambda _4 v_\eta v_\rho - \frac{{\bar{f}}_1}{2} v_{\chi _0} v_\chi &{}\quad \mu _2^2 + 6 \lambda _2\,{v_\rho }^2 + \lambda _4 v_\eta ^2 +\lambda _6 v_\chi ^{2} &{}\quad 2 \lambda _6 v_\rho v_\chi - \frac{{\bar{f}}_1}{2} v_{\chi _0} v_\eta \\ 2 \lambda _5 v_\eta \, v_\chi - \frac{{\bar{f}}_1}{2} v_{\chi _0} v_\rho &{}\quad 2 \lambda _6 v_\rho v_\chi - \frac{{\bar{f}}_1}{2} v_{\chi _0} v_\eta &{}\quad \mu _3^2 + 6 \lambda _3 v_\chi ^{2} + \lambda _5 v_\eta ^2 + \lambda _6 v_\rho ^2 \end{array} \right] . \end{aligned}$$
(23)

In this sector one identifies two single charged scalars mass matrices, \(M_{\mathrm{char}_1}\) and \(M_{\mathrm{char}_2}\), given, respectively, by

$$\begin{aligned} M_{\mathrm{char}_1}=\left[ \begin{array}{ll} \mu _1^{2} + 2 \lambda _1 v_\eta ^2+ (\lambda _4+\lambda _7) v_\rho ^2 + \lambda _5 v_\chi ^2 &{}\quad \lambda _7 v_\rho v_\eta + \frac{{\bar{f}}_1}{2} v_{\chi _0} v_\chi \\ \lambda _7 v_\rho v_\eta + \frac{{\bar{f}}_1}{2} v_{\chi _0} v_\chi &{}\quad \mu _2^{2} + 2 \lambda _2 v_\rho ^2+ (\lambda _4 +\lambda _7) v_\eta ^2 + \lambda _6 v_\chi ^2 \end{array} \right] \end{aligned}$$
(24)

and

$$\begin{aligned} M_{\mathrm{char}_2}=\left[ \begin{array}{ll} \mu _1^{2} + 2 \lambda _1 v_\eta ^2+ \lambda _4 v_\rho ^2 + (\lambda _5+\lambda _8) v_\chi ^2 &{} \quad \lambda _8 v_\eta v_\chi + \frac{{\bar{f}}_1}{2} v_{\chi _0} v_\rho \\ \lambda _8 v_\eta v_\chi + \frac{{\bar{f}}_1}{2} v_{\chi _0} v_\rho &{}\quad \mu _3^{2} + 2 \lambda _3 v_\chi ^2 + \lambda _6 v_\rho ^2 + (\lambda _5 +\lambda _8) v_\eta ^2 \end{array} \right] . \end{aligned}$$
(25)

There is a double charge \(M_{\mathrm{double}}\) scalar mass matrix, given by

$$\begin{aligned} M_\mathrm{double}= \left[ \begin{array}{ll} \mu _2^{2} + 2 \lambda _2 v_\rho ^2 + \lambda _4 v_\eta ^2 + \left( \lambda _6 + \lambda _9 \right) v_\chi ^2 &{}\quad \lambda _9 v_\rho v_\chi + \frac{{\bar{f}}_1}{2} v_{\chi _0} v_\eta \\ \lambda _9 v_\rho v_\chi + \frac{{\bar{f}}_1}{2} v_{\chi _0} v_\eta &{}\quad \mu _3^{2} +2\lambda _3 v_\chi ^{2} + \lambda _5 v_\eta ^2 + \left( \lambda _6 + \lambda _9\right) v_\rho ^2 \end{array} \right] . \end{aligned}$$
(26)

Next, we give the CP-odd scalar sector. The mass matrix of the neutral scalars is given by

$$\begin{aligned} M^{CP}_{\mathrm{neutral}}=\left[ \begin{array}{lll} \mu _1^{2} + 2 \lambda _1 \,v_\eta ^{2} +\lambda _4 v_\rho ^2 + \lambda _5 v_\chi ^{2} &{}\quad \frac{{\bar{f}}_1}{2} v_{\chi _0} v_\chi &{}\quad \frac{{\bar{f}}_1}{2} v_{\chi _0} v_\rho \\ \frac{{\bar{f}}_1}{2} v_{\chi _0} v_\chi &{}\quad \mu _2^{2}+ 2 \lambda _2 v_\rho ^2+ \lambda _4 v_\eta ^2 + \lambda _6 v_\chi ^2 &{}\quad \frac{{\bar{f}}_1}{2} v_{\chi _0} v_\eta \\ \frac{{\bar{f}}_1}{2} v_{\chi _0} v_\rho &{}\quad \frac{{\bar{f}}_1}{2} v_{\chi _0} v_\eta &{}\quad \mu _3^{2} + 2 \lambda _3 v_\chi ^2 + \lambda _5 v_\eta ^2 + \lambda _6 v_\rho ^2 \end{array}\right] . \end{aligned}$$
(27)

In this sector there are two mass matrices of single charged scalars, given, respectively, by

$$\begin{aligned} M^{CP}_{\mathrm{char}_1}= \left[ \begin{array}{ll} \mu _1^{2}+ 2 \lambda _1 v_\eta ^2 + \left( \lambda _4+\lambda _7 \right) v_\rho ^2+ \lambda _5 v_\chi ^2 &{}\quad -\lambda _7 v_\eta v_\rho - \frac{{\bar{f}}_1}{2} v_{\chi _0} v_\chi \\ -\lambda _7 v_\eta v_\rho - \frac{{\bar{f}}_1}{2} v_{\chi _0} v_\chi &{}\quad \mu _2^{2} + 2 \lambda _2 v_\rho ^2 + \left( \lambda _4+ \lambda _7 \right) v_\eta ^2 + \lambda _6 v_\chi ^2 \end{array} \right] \end{aligned}$$
(28)

and

$$\begin{aligned} M^{CP}_{\mathrm{char}_2}= \left[ \begin{array}{ll} \mu _1^{2}+ 2 \lambda _1 v_\eta ^2 + \lambda _4 v_\rho ^2 + \left( \lambda _5+\lambda _8 \right) v_\chi ^2 &{}\quad -\lambda _8 v_\eta v_\chi - \frac{{\bar{f}}_1}{2} v_{\chi _0} v_\rho \\ -\lambda _8 v_\eta v_\chi - \frac{{\bar{f}}_1}{2} v_{\chi _0} v_\rho &{}\quad \mu _3^{2} + 2 \lambda _3 v_\chi ^2 + \lambda _6 v_\rho ^2 + \left( \lambda _5+ \lambda _8 \right) v_\eta ^2 \end{array} \right] , \end{aligned}$$
(29)

and a matrix for doubly charged CP-odd scalars,

$$\begin{aligned} M^{CP}_{\mathrm{double}}= \left[ \begin{array}{ll} \mu _2^{2}+ 2 \lambda _2 v_\rho ^2 + \lambda _4 v_\eta ^2 + \left( \lambda _6+\lambda _9 \right) v_\chi ^2 &{}\quad -\lambda _9 v_\rho v_\chi - \frac{{\bar{f}}_1}{2} v_{\chi _0} v_\eta \\ -\lambda _9 v_\rho v_\chi - \frac{{\bar{f}}_1}{2} v_{\chi _0} v_\eta &{}\quad \mu _3^{2} + 2 \lambda _3 v_\chi ^2 + \lambda _5 v_\eta ^2 + \left( \lambda _6+ \lambda _9 \right) v_\rho ^2 \end{array} \right] . \end{aligned}$$
(30)

The scalar and gauge boson masses depend on two VEVs (\(v_W\) and \(v_{\chi _0}\)), on the projection angle \(\beta \), and on the ten couplings. We obtain the scalar masses by diagonalizing the corresponding \(3\times 3\) and \(2 \times 2\) mass matrices given above.

We have in mind previous analyses [13, 16, 19] predicting that the neutral scalar CP-even sector must contain a low mass component corresponding to the SM Higgs particle H. In addition, in this model, one expects two neutral scalars, \(H_1^0\) and \(H_2^0\), in the CP-even sector. In the CP-odd sector, there is another neutral scalar, \(H_{cp}^0\), along with two Goldstone bosons. For the charged states, one expects six massive scalars, \(H_1^\pm , H_2^\pm \), and \(H^{\pm \pm }\), and another six Goldstone bosons. All the eight Goldstone bosons give mass to the massive gauge bosons, i.e., the SM gauge bosons Z and \(W^{\pm }\) and the additional heavy bosons predicted by the present version of the model, \(Z^\prime \), \(V^\pm \), and \(Y^{\pm \pm }\). The four singly charged massive gauge fields (\(W^\pm \) and \(V^\pm \)), two doubly charged massive gauge fields (\(Y^{\pm \pm }\)), and two neutral massive gauge fields (Z and \(Z^\prime \)). These gauge fields have to obtain mass from the Higgs mechanism occurring at the electroweak \(v_W\) scale and at the \(v_{\chi _0}\) higher energy scale.

Our aim is to obtain the scalars, exotic quarks, and gauge boson masses using minimal arbitrariness. In order to fit the Higgs mass and the Goldstone fields with some set of parameters, one must respect the recent gauge boson \(Z^\prime \) mass lower limit determined from the upper limit on the ATLAS/LHC electron and muon production cross section [12] (note also that there are also similar constraints for the \(Z'\) from calculations of the muon magnetic moment [20]). By diagonalizing the \(M_\mathrm{neutral}\) matrix it is possible to respect the LHC constraint [12], \(Z_{331 \mathrm{minimal}} > 2.93\) TeV, and to reproduce the SM Higgs mass. This in turn, from the expression for \(Z'\), Eq. (15), leads to a lower bound on the scale, \(v_{\chi _0} \gtrsim 3\) TeV.

Once a given set of couplings are given, we obtain the whole scalar spectrum from the eigenvalues of the corresponding matrices calculated in the vacuum, \(v_\eta = v_{\eta _0} \equiv v_W \sin (\beta )\), \(v_\rho = v_{\rho _0} \equiv v_W \cos (\beta )\), and \(v_\chi =v_{\chi _0}\). It results in nine scalars constructed from the real components of the fields. The SM Higgs is constructed from the combination \(\eta ^0 - \rho ^0\). The other eight heavy scalars, namely \(H_1^0\) and \(H_2^0\), are related to the \(\eta ^0 \)\(\chi ^0\) and \(\rho ^0 \)\( \chi ^0\) combinations, respectively, \(H_1^\pm \) and \(H_2^\pm \) are related to the \(\eta _1^\pm \)\(\chi ^\pm \) and \(\eta _2^\pm \)\(\chi ^\pm \) combinations, respectively, and \(H^{\pm \pm }\), which is related to the \(\rho ^{\pm \pm }\)\(\chi ^{\pm \pm }\) combination. In the CP-odd sector, there is one heavy neutral \(H_{cp}^0\), which is related to the imaginary part of the neutral field components. The gauge bosons \(W^\pm , V^\pm \), and \(Y^{\pm \pm }\) acquire their masses from the imaginary part of the fields in the \(\eta _1^\pm \)\(\rho ^\pm \), \(\eta _2^\pm \)\(\chi ^\pm \), and \(\rho ^{\pm \pm } \)\( \chi ^{\pm \pm }\) combinations, respectively. The neutral gauge bosons Z and \(Z'\) get their masses from the imaginary part of the fields in the \(\eta ^0\)\(\rho ^0\) and \(\rho ^0\)\(\chi ^0\) combinations, respectively.

As already observed, the model has a too large number of free parameters, which makes it an almost impossible job to study the complete parameter region allowed. Since our objective in this work is to determine whether a strong first-order phase transition in the model is possible, our strategy for fixing the many couplings is then chosen so as to maximize this goal. For this purpose, we can borrow some of the lessons already learned when studying the phase transition in the SM and other extensions of it (see, e.g., Refs. [2123] and references therein). To satisfy the usual criterion for a strong first-order phase transition, namely that the ratio of the field expectation value at the critical temperature and the critical temperature be larger than one, \(v(T_\mathrm{c})/T_\mathrm{c} > 1\), we need, optimally, either a larger VEV at \(T_\mathrm{c}\) and/or a small \(T_\mathrm{c}\). Typically \(T_\mathrm{c}\) is constrained by the scale, \(T_\mathrm{c} \propto v_0\), which for us is rather large (recalling that \(v_{\chi _0} \gtrsim 3\) TeV and \(v_{\eta _0}\) and \(v_{\rho _0}\) are constrained by the Weinberg scale. Since in general the VEV is given in terms of a combination of couplings and masses, \(v \propto m_i/\sqrt{\lambda _i}\), an ideally situation is to try to work with the smallest couplings possible. There is, however, a trade off. Too small couplings lead in general to a light particle spectrum, which for us is still limited by the scales and observational bounds (in particular, other scalars than the SM Higgs are expected to be sufficiently heavy for not being detected yet). The Higgs mass \(m_\mathrm{H}\) itself is the only limiting observational quantity we have in the scalar sector. Since the Higgs is a mass eigenvalue for the CP-even scalar neutral matrix, Eq. (23), it only (weakly) constrains the couplings \(\lambda _{i}\), \(i=1,\ldots ,6\), and \({\bar{f}}_1\).

In the analysis below, we fix \(m_\mathrm{H} = 125\) GeV and work with four different sets of choices for couplings. Other possibilities can be shown to fall in one of these sets. In each of the sets used, we look for the ideal conditions for having a strong first-order phase transition, which, as mentioned above, favors the smallest choice of couplings in general. In all the sets we consider below, we found it more convenient to vary the inter-couplings between the different fields, \(\lambda _4, \, \lambda _5\), and \(\lambda _6\), due to their relation to the SM Higgs mass (the other inter-couplings \(\lambda _7, \, \lambda _8\), and \(\lambda _9\), only appear on the heavy charged scalars and are unconstrained by the Higgs). We will consider the following four large sets of parameters:

  1. (a)

    Set I: The couplings \(\lambda _i\), with \(i=1,2,3,5,6,7,8,9\), such that \(\lambda _i = \lambda \) varied together with \(\lambda _4\);

  2. (b)

    Set II: The couplings \(\lambda _i\), with \(i=1,2,3,4,6,7,8,9\), such that \(\lambda _i = \lambda \) varied together with \(\lambda _5\);

  3. (c)

    Set III: The couplings \(\lambda _i\), with \(i=1,2,3,4,5,7,8,9\), such that \(\lambda _i = \lambda \) varied together with \(\lambda _6\);

  4. (d)

    Set IV: The couplings \(\lambda _i\), with \(i=1,2,3,7,8,9\),such that \(\lambda _i = \lambda \) and \(\lambda _4 = \lambda _5=\lambda _6={\bar{\lambda }}\), which are then varied.

For all sets we have fixed \({\bar{f}}_1=1\). This is motivated by the fact that \({\bar{f}}_1\) determines the asymmetry of the potential in the \(\chi \) direction and the more asymmetrical is the potential, the more we expect to have a stronger first-order phase transition. Note also that the values of couplings larger than one can make us enter in a nonperturbative regime of parameters. We avoid this situation here, since we work only at the one-loop level for the effective potential for the model (see next section). In particular, we have explicitly checked that smaller values of \({\bar{f}}_1\) always lead to weaker transitions. In addition, for each of the sets explained above, we have chosen to work with the higher energy scale \(v_{\chi _0}\) with values \(v_{\chi _0}= 3\), 4, and 5 TeV, satisfying the current constraints on the \(Z'\) mass, as already mentioned. Likewise, for the projection angle \(\beta \), we have considered for each of the sets and values of the scale, the three values \(\beta = 30^{\circ }, \, 45^{\circ },\, 60^{\circ }\). Again, we have explicitly verified that larger asymmetries on the \(v_{\eta }\) and \(v_\rho \) directions are either disfavored, or there is a trade off, since as we decrease the projection in one direction, there is a compensation by the increase of the projection in the other direction (recalling again that \(v_{\eta }\) and \(v_\rho \) are constrained by the Weinberg scale \(v_W\)). Nonetheless, our analysis shows that the symmetrical case \(v_\eta = v_\rho \), i.e., \(\beta = 45^{\circ }\), tends to be favored as far as the strength of the transition is concerned.

Each one of the parameters in the above four sets is then chosen so as to satisfy the Higgs mass \(m_\mathrm{H} = 125\) GeV. The resulting relations between these couplings subject to this constraint are shown in Fig. 1a–d.

Fig. 1
figure 1

Set of couplings fitting the SM Higgs mass. Dotted lines are for projection angle \(\beta = 30^{\circ }\), dashed lines is for \(\beta = 45^{\circ }\) and solid lines for \(\beta = 60^{\circ }\). The insets show a region around the minimal values for the couplings

We note from the results shown in Fig. 1 that for sets II, III, and IV, there are always minimal values for the couplings. In the case of the set I, \(\lambda _i\) is a decreasing function of \(\lambda _4\).

For illustration, the resulting scalar mass spectrum in each set is shown in Table 1. For convenience, we show only the values at the minimal values of couplings in the case of the sets II, III, and IV shown in Fig. 1. In the case of set I, the values of the masses are for \(\lambda _4=1\), which we take as the limit for a “perturbative” coupling. Note that exchanging \(\beta =30^{\circ }\) by \(\beta = 60^{\circ }\) corresponds to change \(v_\eta \) by \(v_\rho \). The dependence of \(M_2^\pm \) on the set {\(v_{\rho ^0}, v_{\eta ^0}\)} is the same as the dependence of \(M^{\pm \pm }\) on {\(v_{\eta ^0}, v_{\rho ^0}\)}. As a consequence, the mass of the singly charged scalar \(H_2^\pm \) for \(\beta = 30^{\circ }\) is the same as that for the double charged \(H^{\pm \pm }\) for \(\beta = 60^{\circ }\). For completeness, the heavy gauge boson masses \(M_{Z^\prime }\), \(M_{V^\pm }\) and \(M_{Y^{\pm \pm }}\) are shown in Table 2, for the cases of \(v_{\chi _0}=3,\, 4\), and 5 TeV. Here again the dependence of \(M_{V^\pm }\) on the set {\(v_{\eta ^0}, v_{\rho ^0}\)} is the same as that of \(M_{Y^{\pm \pm }}\) on the set {\(v_{\eta ^0}, v_{\rho ^0}\)} and so we observe that \(M_{V^\pm }\) for \(\beta =30^{\circ }\) is equal to \(M_{Y^{\pm \pm }}\) for \(\beta =60^{\circ }\), for any \(v_{\chi ^0}\) value.

Table 1 The masses (in units of TeV) for the additional scalars in the model. The scale \(v_{\chi _0}\) has been fixed at the value of 3 TeV
Table 2 The masses (in units of TeV) for the heavy gauge bosons \(Z^\prime \), \(V^\pm \) and \(Y^{\pm \pm }\)

In the next section we introduce the quantum and thermal corrections at the one-loop level for the effective potential in the 3-3-1 model and we analyze its temperature dependence, for the different set of parameters explained above, obtaining the symmetry restoration temperatures. As input, we use the SM values for the masses of the Higgs, quark top, gauge boson Z, and \(W^\pm \): \(m_\mathrm{H}=125\) GeV, \(m_{top} = 173.21\) GeV, \(M_Z = 91.19\) GeV, and \(M_{W^\pm } = 80.39\) GeV. The exotic heavy quarks masses have been fixed as \(m_\mathrm{Q} \left( v_{\chi _0}\right) = v_{\chi _0}/2\).

4 The one-loop effective potential for the 3-3-1 model

The effective potential is expressed as a function of the background values for the scalars \(\langle \eta \rangle =v_\eta \), \(\langle \rho \rangle = v_\rho \), and \(\langle \chi \rangle = v_\chi \). It depends on the loop contributions from the gauge bosons, through their tree-level background-field-dependent masses, as well as those from fermions that can give a significant contribution to the effective potential, namely the top quark (t) and the three exotic heavy quarks (Q). Finally, we also have to add the contributions from the SM Higgs and from the nine scalars that become heavy in the vacuum after the first SSB. Besides, when choosing a gauge other than the unitary gauge, we have also to include the contributions from the eight Goldstone bosons. In this work, as is usual in the literature (see, e.g., Ref. [21] and references therein), we give the expression for the effective potential in the ‘t Hooft–Landau gauge.

Fig. 2
figure 2

The vacuum subtracted tree-level potential (solid line) and the one-loop quantum corrected effective potential (dashed line) in the direction of the background scalar field \(v_\chi \) (at \(v_\eta = v_\rho =v_W/\sqrt{2}\)) (a) and in the direction of the background scalar field \(v_\eta = v_\rho \) (b), expressed in terms of \(\phi = \pm \sqrt{v_\eta ^2 + v_\rho ^2}\). The potential is in units of TeV\(^4\) and the background fields are in units of TeV. The parameters used are those from set II, for \(\tan (\beta )=1\) and \(v_{\chi _0}=3\) TeV

The effective potential in terms of the background fields is expressed as

$$\begin{aligned} V_\mathrm{eff}(v_\eta ,v_\rho ,v_\chi , T)= & {} V_\mathrm{tree}(v_\eta ,v_\rho ,v_\chi ) +\Delta V_0 (v_\eta ,v_\rho ,v_\chi ) \nonumber \\&+ \Delta V_T (v_\eta ,v_\rho ,v_\chi , T), \end{aligned}$$
(31)

where \(V_\mathrm{tree}\) is the tree-level potential, Eq. (19), \(\Delta V_0 (v_\eta ,v_\rho ,v_\chi )\) is the zero temperature (quantum) contribution for the one-loop effective potential, while \(\Delta V_T (v_\eta ,v_\rho ,v_\chi , T)\) is the finite temperature contribution at the one-loop level.

The one-loop quantum contribution \(\Delta V_0\) is ultraviolet divergent and needs to be renormalized. In the cutoff regularization scheme with the subtraction point chosen at the scalar vacuum expectation values (thus preserving the values of \(v_{\eta _0}, v_{\rho _0}\), and \(v_{\chi _0}\)), \(\Delta V_0 (v_\eta ,v_\rho ,v_\chi )\) is given by [21]

$$\begin{aligned} \Delta V_0 (v_\eta ,v_\rho ,v_\chi )= & {} \frac{1}{64\pi ^2}\sum _{i} n_i \left\{ m_i^4(v)\left[ \ln \frac{m_i^2(v)}{m_i^2(v_0)}-\frac{3}{2}\right] \right. \nonumber \\&+ \left. 2 m_i^2(v_0) m_i^2(v) \frac{}{}\phantom {\left[ \ln \frac{m_i^2(v)}{m_i^2(v_0)}-\frac{3}{2}\right] }\right\} \nonumber \\&+\frac{1}{64\pi ^2} \sum _{G} n_G \, m_G^4(v)\left[ \ln \frac{m_G^2(v)}{m_\mathrm{H}^2(v_0)}-\frac{3}{2}\right] ,\nonumber \\ \end{aligned}$$
(32)

where \(n_i\) in Eq. (32) denotes the field degrees of freedom: The massive charged gauge bosons have \(n_i=6\) (e.g., \(W^{\pm }, V^\pm , Y^{\pm \pm }\)), the neutral massive gauge bosons have \(n_i=3\) (e.g., \(Z,Z'\)), the heavy quarks have \( n_i=-12\) (e.g., the top t and the three new exotic quarks Q) and each of the neutral and charged scalars has \(n_i=1\). The last sum in Eq. (32) is over the Goldstone modes, each one contributing with \(n_G=1\). Finally, the masses \(m_{i(G)}(v)\) and \(m_{i(G)}(v_0)\), with \(v \equiv v_\eta , v_\rho , v_\chi \), and \(v_0 \equiv v_{\eta _0}, v_{\rho _0}, v_{\chi _0}\), stand for the particle masses computed at the background and vacuum expectation values, respectively.

As an example, in Fig. 2 we show both the tree-level and the one-loop quantum corrected potentials in the case of \(\tan (\beta )=1\), i.e., for \( v_\rho =v_\eta \), in the set II case of parameters explained in the previous section, taken at the minimal values of couplings. We have also considered the scale \(v_{\chi _0}= 3\) TeV. For convenience of presentation, we have subtracted from the potential the vacuum contribution at the origin (corresponding to an overall shift of the whole potential).

The finite temperature contribution in Eq. (31), \(\Delta V_T(v_\eta ,v_\rho ,v_\chi , T)\), is given by [24]

$$\begin{aligned} \Delta V_T (v_\eta ,v_\rho ,v_\chi , T)= & {} \frac{T^4}{2\pi ^2}\left[ \sum _{i=\mathrm{bosons}} n_i J_B[m_i^2(v)/T^2] \right. \nonumber \\&+ \left. \sum _{i=\mathrm{fermions}} n_i J_F[m_i^2(v)/T^2] \right] ,\nonumber \\ \end{aligned}$$
(33)

where the functions \(J_B\) and \(J_F\) are defined as

$$\begin{aligned} J_B(y)=\int _0^{\infty } \mathrm{d}x\ x^2\ln \left[ 1-\mathrm{e}^{-\sqrt{x^2+y^2}}\right] \; \end{aligned}$$
(34)

and

$$\begin{aligned} J_F(y)=\int _0^{\infty } \mathrm{d}x\ x^2\ln \left[ 1+\mathrm{e}^{-\sqrt{x^2+y^2}}\right] \;. \end{aligned}$$
(35)

The thermal bosonic one-loop integral (34) admits a high-temperature expansion, for \(y \ll 1\) (where \(y=m(v)/T\)). It is given by [24]

$$\begin{aligned} J_B(y)= & {} -\frac{\pi ^4}{45}+ \frac{\pi ^2}{12}y^2-\frac{\pi }{6} \left( y^2\right) ^{3/2}-\frac{1}{32} y^4\ln \frac{y^2}{a_b} \nonumber \\&-2\pi ^{7/2}\sum _{\ell =1}^{\infty }(-1)^{\ell }\frac{\zeta (2\ell +1)}{(\ell +1)!}\Gamma \left( \ell +\frac{1}{2}\right) \left( \frac{y^2}{4\pi ^2} \right) ^{\ell +2},\nonumber \\ \end{aligned}$$
(36)

while the thermal fermionic one-loop integral (35), for \(y \ll 1\), can likewise be expressed as

$$\begin{aligned} J_F(y)= & {} \frac{7\pi ^4}{360}- \frac{\pi ^2}{24}y^2-\frac{y^4}{32} \ln \frac{y^2}{a_f} \nonumber \\&-\frac{\pi ^{7/2}}{4}\sum _{\ell =1}^{\infty }(-1)^{\ell } \frac{\zeta (2\ell +1)}{(\ell +1)!} \left( 1-2^{-2\ell -1}\right) \nonumber \\&\times \Gamma \left( \ell +\frac{1}{2}\right) \left( \frac{y^2}{\pi ^2} \right) ^{\ell +2}. \end{aligned}$$
(37)

In the above expressions, \(a_b=16\pi ^2\exp (3/2-2\gamma _E)\) (and \(\ln a_b=5.4076\)), \(a_f=\pi ^2\exp (3/2-2\gamma _E)\) (or \(\ln a_f=2.6351\)), and \(\zeta \) is the Riemann \(\zeta \)-function.

In the opposite regime of a low temperature, \(y> 1\), the integrals (34) and (35) are well approximated by

$$\begin{aligned} J_{B(F)}(y) \simeq \mp \sqrt{\frac{\pi }{2}} y^{3/2} \mathrm{e}^{-y} \left( 1 + \frac{15}{8} \frac{1}{y} \right) . \end{aligned}$$
(38)

It is interesting to find where the behavior of \(J_{B(F)}\) at low temperature matches its high-temperature expression. We find that the transition between the low- and high-temperature approximations occurs at \(y \simeq 2.25\) for the bosonic thermal integral and at \(y\simeq 1.85\) for the fermionic thermal integral. This is sometimes more useful for the numerical analysis than using the exact expressions (34) and (35). More explicitly, we find that a simple interpolation of the two regimes and a truncation in the high-temperature series in Eqs. (36) and (37) result in the following expressions:

$$\begin{aligned} J_B(y)\simeq & {} \left[ -\frac{\pi ^4}{45}+ \frac{\pi ^2}{12}y^2-\frac{\pi }{6} \left( y^2\right) ^{3/2}-\frac{1}{32} y^4\ln \frac{y^2}{a_b} \right. \nonumber \\&- \left. \frac{\zeta (3)}{128 \pi ^2} y^6\right] \theta (2.25-y) \nonumber \\&- \sqrt{\frac{\pi }{2}} y^{3/2} \mathrm{e}^{-y} \left( 1 + \frac{15}{8} \frac{1}{y} \right) \theta (y-2.25) \end{aligned}$$
(39)

and

$$\begin{aligned} J_F(y)\simeq & {} \left[ \frac{7\pi ^4}{360}- \frac{\pi ^2}{24}y^2-\frac{y^4}{32} \ln \frac{y^2}{a_f} \right] \theta (1.85-y) \nonumber \\&+ \sqrt{\frac{\pi }{2}} y^{3/2} \mathrm{e}^{-y} \left( 1 + \frac{15}{8} \frac{1}{y} \right) \theta (y-1.85), \end{aligned}$$
(40)

which provide an excellent coverage of the exact integrals (34) and (35), respectively, in the whole region of high and low temperatures. It is crucial in the present work to pay particular attention to the specific approximation to be used in the effective potential, due to the large disparateness of the mass scales that the model has. At a given temperature, some particles may acquire a mass that is below the temperature and others might have a mass above the temperature, so we have different contributions to the effective potential at different temperatures. This is particularly important when we investigate the behavior of the effective potential in between the first transition and the final electroweak phase transition in the model.

In the next section we will present the results for the phase transition pattern in the 3-3-1 model for the different choices of the sets of parameters explained in Sect. 3.

5 Phase transition pattern in the 3-3-1 model

We can clearly identify the two transitions in the model as the temperature is lowered from values \(T \gg v_{\chi _0}\) to values below the electroweak scale, \(T \ll v_W\). First, the higher symmetry \(SU(3)_L \times U(1)_X\) group is broken down to the electroweak one, \(SU(2)_L\times U(1)_Y\), at a temperature below the 3-3-1 scale \(v_{\chi _0}\). Then follows the usual electroweak phase transition, \(SU(2)_L\times U(1)_Y \rightarrow U(1)_{EM}\), at a temperature slightly below the Weinberg scale \(v_W\).

As an illustrative example of the phase transition related to the two SSB in the model, the temperature-dependent potential is shown in Fig. 3, considering the parameters of set II, with the couplings at its minimum values, for \(\tan (\beta )=1\) and \(v_{\chi _0}=3\) TeV. In Fig. 3a we show the effective potential in the direction of \(v_\chi \). It shows the behavior of the effective potential with the temperature. For temperatures above the scale for the electroweak symmetry breaking, the temperature-dependent values \(v_{\eta }(T)\) and \(v_{\rho }(T)\) vanish (or, equivalently, \(\phi (T) = \sqrt{v_{\eta }^2(T) + v_{\rho }^2(T)}=0\)), since the electroweak symmetry is still in the symmetry restored phase. Figure 3a then shows that there is a temperature \(T=T_{\mathrm{c}_1}\) for which the potential displays degenerate minima at the origin. Below this critical temperature the minimum with non-vanishing background field value becomes the global minimum and for temperatures slight above the critical value it is a local minimum, with the origin being the state of minimum energy. This corresponds to a background value for the \(\chi \) field, \(\langle \chi \rangle \equiv v_\chi \), that changes discontinuously with the temperature, jumping from a value \(v_\chi =0\) to a non-vanishing value at the temperature \(T=T_{\mathrm{c}_1}\). This is the characteristic of a first-order phase transition (as opposite to a second-order phase transition, where the background field changes continuously with the temperature). The same behavior as seen in Fig. 3a is also shown in Fig. 3b. In Fig. 3b we show the effective potential in the direction of \(v_\eta =v_\rho \) (expressed in terms of \(\phi \)) for temperatures below the Weinberg scale \(v_W\). For these low values of temperature, we have \(v_\chi \approx v_{\chi _0}\), i.e., the thermal expectation value for the \(\chi \) field already approaches its vacuum value \(v_{\chi _0}\). All the heavy particles that make the extra particle spectrum of the 3-3-1 model acquire masses close to their vacuum valuesFootnote 1 and they contribute little for the effective potential at this scale. Hence, the particle content dominating the effective potential at \(T \sim T_{\mathrm{c}_2} \ll v_{\chi _0}\) is essentially that of the standard model. Figure 3b shows that the final transition, corresponding to the standard model one, \(SU(2)_L\times U(1)_Y \rightarrow U(1)_{EM}\), happens at a temperature \(T=T_{\mathrm{c}_2} < v_W\) and it is of the type of a first-order phase transition.

Fig. 3
figure 3

The vacuum subtracted one-loop temperature-dependent effective potential in the direction of the background scalar field \(v_\chi \) (at \(v_\eta = v_\rho =0\)) (a) and in the direction of the background scalar field \(v_\eta = v_\rho \) (b), expressed in terms of \(\phi = \pm \sqrt{v_\eta ^2 + v_\rho ^2}\). The temperatures considered are above, at, and below the critical values. \(T_{\mathrm{c}_1}\) and \(T_{\mathrm{c}_2}\) correspond to the values computed at the scale \(v_{\chi _0}=3.0\) TeV, for the case of parameter set II and for \(\tan (\beta )=1\) and whose values are quoted in Table 3. The potential is in units of TeV\(^4\) and the background fields are in units of TeV

In Table 3 we summarize the results for the two phase transitions in the 3-3-1 model, where we give the value for the critical temperatures for the two phase transitions in the model. The first one happens at a temperature \(T_{\mathrm{c}_1}\) and corresponding to the symmetry breaking \(SU(3)_L \times U(1)_X \rightarrow SU(2)_L\times U(1)_Y\), and the second transition, corresponding to the electroweak symmetry breaking \(SU(2)_L\times U(1)_Y \rightarrow U(1)_{EM}\), happens at the temperature \(T_{\mathrm{c}_2}\). Results are shown for the four sets of parameters considered in this work and for the three different projection angles \(\beta \) (\(v_\eta /v_\rho =\tan (\beta )\)). We also show the ratio of the background field working as an order parameter for each transition to the temperature at the critical point. As already mentioned, this is a useful measure of the “strength” of the phase transition, as usually considered in the literature [13, 21] (for other alternative forms of characterizing the strength of the transition, particularly useful for weak first-order phase transitions; see, e.g., Refs. [25, 26]).

Table 3 The critical temperature (in units of TeV) for each of the transitions and the ratio of the vacuum expectation value of the relevant field by the critical temperature. \(T_{\mathrm{c}1}\) corresponds to the critical temperature for the first transition, \(SU(3)_L \times U(1)_X \rightarrow SU(2)_L\times U(1)_Y\), while \(T_{\mathrm{c}2}\) corresponds to the one for the second transition, \(SU(2)_L\times U(1)_Y \rightarrow U(1)_{EM}\). The results shown are for the scale \(v_{\chi _0}=3\) TeV

In all cases shown in Table 3 the parameter sets are taken at their minimal values, shown in Fig. 1. We have explicitly verified that the values of \(T_{\mathrm{c}_1}\) specified correspond to the minimum possible critical temperature found within the range of couplings shown in Fig. 1. Changing the values of the couplings away from the minimum values satisfying the Higgs mass constraint always tend to increase the value of \(T_{\mathrm{c}_1}\) and, consequently, decrease the ratio \(\langle v_\chi (T_{\mathrm{c}_1})\rangle /T_{\mathrm{c}_1}\). The same is true in general for \(T_{\mathrm{c}_2}\), except for the case of set I, where we find that decreasing the value of the coupling \(\lambda _4\), \(T_{\mathrm{c}_2}\) tends to decrease, thus increasing the ratio \(\langle \phi (T_{\mathrm{c}_2})\rangle /T_{\mathrm{c}_2}\), but the minimum values of \(T_{\mathrm{c}_2}\) we have found are still limited by the minimum values shown in Table 3, and \(\langle \phi (T_{\mathrm{c}_2})\rangle /T_{\mathrm{c}_2} \lesssim 0.08\). This then implies that the results for the second transition, corresponding to \(SU(2)_L\times U(1)_Y \rightarrow U(1)_{EM}\), has values for the ratio \(\langle \phi (T_{\mathrm{c}_2})\rangle /T_{\mathrm{c}_2}\) that are always much smaller than one, which characterizes a very weak first-order, possibly second-order, phase transition. We have explicitly verified that by increasing the scale \(v_{\chi _0}\) it causes very little changes to the second transition. This is consistent with the fact that the higher is \(v_{\chi _0}\), the sooner the heavier particles decouple from the SM electroweak spectrum. We also note that closer to the transition point it is also known that self-energy corrections to the effective potential can make the transition even weaker [21, 27].

Fig. 4
figure 4

The functional dependence for the masses for the scalars \(H_2^0\) and \(H_2^\pm \) for the parameter set IV case and for \(\tan (\beta )=1\). Units are in TeV

As far as the first transition is concerned, corresponding to \(SU(3)_L \times U(1)_X \rightarrow SU(2)_L\times U(1)_Y\), we find that it has a value for the critical temperature that increases proportionally to the scale \( v_{\chi _0} \), as we would expect on general grounds. The ratio \(\langle v_\chi (T_{\mathrm{c}_1})\rangle /T_{\mathrm{c}_1} \) tends to be closer to one, becoming larger when the scale increases. Among the different sets of parameters we have considered, the one most favorable for producing a strong first-order phase transition, \(\langle v_\chi (T_{\mathrm{c}_1})\rangle /T_{\mathrm{c}_1}> 1\), is set IV, as explicitly noted from the values shown in Table 3. In particular, the case with \(\tan (\beta ) =1\), i.e., \(v_\eta = v_\rho \), is the one that is able to produce the strongest transition. Note, however, looking at the values for the scalar mass spectrum shown in Table 1, that this is also the case that leads to the smallest mass for the Higgs like scalar particle \(H_2^0\), with a mass \(M_{H_2^0} \simeq 383\) GeV. Increasing the scale \(v_{\chi _0} \) this value of the mass also increases. For example, for \(v_{\chi _0}= 5\) TeV, we have \(M_{H_2^0} \simeq 616\) GeV and \(T_{\mathrm{c}_1} = 605\) GeV, with a ratio \(\langle v_\chi (T_{\mathrm{c}_1})\rangle /T_{\mathrm{c}_1}\simeq 8.12\). In Fig. 4 we show the functional dependence for the masses of the lightest scalars after the standard model Higgs, i.e., for the scalars \(H_2^0\) and \(H_2^\pm \) (note that the double charged scalar \(H^{\pm \pm }\) is degenerate in mass with \(H_2^\pm \) when \(\tan (\beta )=1\)). The value of \(T_{\mathrm{c}_1}\) tends to increase (and consequently the ratio \(\langle v_\chi (T_{\mathrm{c}_1})\rangle /T_{\mathrm{c}_1}\) decreases) as we move from the smallest values of masses toward the largest values. For example, in the case \(v_{\chi _0}=3 \) TeV, for \(M_{H_2^0} = 1\) TeV, we find that \(T_{\mathrm{c}_1}\simeq 1.1\) TeV and \(\langle v_\chi (T_{\mathrm{c}_1})\rangle /T_{\mathrm{c}_1} \simeq 1.8\). Typically, we find that for all sets of parameters considered, \(T_{\mathrm{c}_1} \approx M_{H_2^0}\) within around \(10\,\%\). Note that this automatically implies a lower bound, \(M_{H_2^0} \gtrsim v_W\), since the first transition must obviously occur at a temperature above the electroweak one.

6 Conclusions

In this work we have studied the symmetry breaking patterns of the 3-3-1 model at finite temperature. Making use of the minimal version of the model, we have first analyzed its scalar sector, which is constructed from three scalars in the triplet representation of \(SU(3)_L\) and the most general renormalizable interactions that can be constructed with these fields. Despite the very large parameter space of the model, we have made an extensive analysis of the model making use of four large sets of parameters that give relations between the scalar couplings. This was done in such a way as to maximize the possibility of finding a strong first-order phase transition on this model, motivated by its possible role in baryogenesis scenarios in extensions of the SM. This allowed us to make a systematic (though far from complete, it should be sufficiently representative for our purposes in this work) investigation of the two symmetry transitions in the model, \(SU(3)_L \times U(1)_X \rightarrow SU(2)_L\times U(1)_Y\) and the standard electroweak one, \(SU(2)_L\times U(1)_Y \rightarrow U(1)_{EM}\).

On studying the temperature effects on the effective potential at the one-loop level, and within the approximations used, we have shown that the model encodes two first-order phase transitions. The last one, corresponding to the standard electroweak phase transition, \(SU(2)_L\times U(1)_Y \rightarrow U(1)_{EM}\), turns out to be always very weak, most likely turning into a second-order or a crossover in practice. For the first transition, corresponding to \(SU(3)_L \times U(1)_X \rightarrow SU(2)_L\times U(1)_Y\), we find that there are regions of parameters that can favor a strong first-order phase transition and, in particular, we have found that the critical temperature in this case is always close to the mass of the Higgs-like scalar \(H_2^0\). This indicates that we can use the estimated value for the mass \(M_{H_2^0}\) as a reasonable estimate for the temperature of transition \(T_{\mathrm{c}_1}\).

Our results should be contrasted with some previous analyses of the phase transition performed in some variants of the 3-3-1 model done in Refs. [28, 29]. In Ref. [28] the authors have used the so-called reduced minimal 3-3-1 model, while in Ref. [29] the economical 3-3-1 model was used. These models differ from the one we have used in the present work in the sense that they have a reduced number of couplings in the potential for only two scalar triplets interactions. The economical 3-3-1 model has a much richer leptonic content than the reduced minimal version and both versions exclude quarks with exotic electric charges. The SSB mechanism applied for determining how each ordinary or new gauge boson acquires the mass follows the same road as usual. The Goldstone bosons are identified, but no numerical value for the masses of the scalars have been shown. In Refs. [28, 29] the authors find parameter regimes where strong first-order phase transitions are found for both the \(SU(3)_L \times U(1)_X \rightarrow SU(2)_L\times U(1)_Y\) and the \(SU(2)_L\times U(1)_Y \rightarrow U(1)_{EM}\) transitions. This discrepancy as regards the strength of the predicted two step phase transition in these alternative models with our present results deserves an interpretation.

We believe that the most important source for the difference between our results and the previous ones come from the fact that in Refs. [28, 29] it was assumed that only one field direction would contribute at each transition, e.g., with \(v_\eta =0\) in the first transition along the \(v_\chi \) direction and with \(v_\chi =0\) in the second transition along the \(v_\eta \) direction. While this is basically true in the first case, where the temperature is sufficiently high to have \(v_{\eta ,\rho }=0\), this is not the case for the second transition. As we have explained in Sect. 5, in the first transition the electroweak phase would still be in its symmetry restored phase for temperatures \(T \gg v_W\), thus \(v_\eta =v_\rho =0\). However, for the second transition, the temperature is already low enough, \(T < v_W\), so that \(v_\chi \approx v_{\chi _0}\) and all the heavy particles that make the extra particle spectrum of the 3-3-1 model acquire masses close to their vacuum values decoupling from the particle spectrum (e.g., their temperature-dependent contributions to the effective potential become all Boltzmann suppressed). The particle content at these low values of the temperature is then dominated essentially by that of the standard model. As such, we expect the results not to differ strongly from the ones known for the phase transition in the standard model. That the heavy mass particles of the 3-3-1 model contributes little at the electroweak phase transition is confirmed by the results. We have found in all cases of sets of parameters considered here that the critical temperature \(T_{\mathrm{c}_2}\) as well the ratio \(\langle v_\eta (T_{\mathrm{c}_2})\rangle /T_{\mathrm{c}_2} \) is very weakly dependent on the scale \(v_{\chi _0}\), which controls the masses of the heavy particles.

Our results show that using the parameter set IV, in particular for \(\tan (\beta )=1\), can lead to a very low critical temperature for the first transition. In particular, we have obtained the result that, for all parameters studied, \(T_{\mathrm{c}_1} \approx M_{H_2^0}\) within around \(10\,\%\). This result and the possibility of having a critical temperature \(T_{\mathrm{c}_1}\) not too high above that for the EWPT, \(T_{\mathrm{c}_2}\), are deserving of further analysis in the future and may have for this model important implications as regards astroparticle physics and cosmology.