Spontaneous symmetry breaking in the $S_3$-symmetric scalar sector

We present a detailed study of the vacua of the $S_3$-symmetric three-Higgs-doublet potential, specifying the region of parameters where these minimisation solutions occur. We work with a CP conserving scalar potential and analyse the possible real and complex vacua with emphasis on the cases in which the CP symmetry can be spontaneously broken. Results are presented both in the reducible-representation framework of Derman, and in the irreducible-representation framework. Mappings between these are given. Some of these implementations can in principle accommodate dark matter and for that purpose it is important to identify the residual symmetries of the potential after spontaneous symmetry breakdown. We are also concerned with constraints from vacuum stability.

Scalar sectors with three doublets have been considered, sometimes inspired by the existence of three generations of fermions. The general case involves a large number of free parameters [17], and these parameters are only constrained by general principles, like positivity, and a viable spectrum. Several authors, starting in 1977 with Pakvasa and Sugawara [18], have imposed an S 3 permutation symmetry, in part motivated by a desire to model the fermion generations. As compared to the most general three-Higgs-doublet model, it is also attractive since it reduces the number of parameters.
Pakvasa and Sugawara exploited the reduction of S 3 to its irreducible doublet and singlet. However, their potential, which has seven quartic terms, was later [19] shown not to be the most general one, the term here referred to as the λ 4 -term, was missing. In the sequel we show that this term plays a very important rôle. An alternative formulation in terms of the three Higgs doublets in the reducible representation transforming under the six three-by-three matrices of permutation (to be referred to in the following as the reducible triplet) was presented by Derman [20] and further studied in a paper with Tsao [21].
The descriptions of the potential in terms of the reducible-triplet and the irreducible frameworks are equivalent. But other sectors of the theory, in particular the Yukawa sector, would differentiate these frameworks and lead to different physics.
The vacua of the S 3 -symmetric potential have been classified in terms of their residual symmetries by Ivanov and Nishi [22]. Here, we shall present another classification, namely in terms of constraints on the potential. For real vacua, it was known that the condition λ 4 = 0 (see below) is relevant for the classification of the different vacua. We find that this parameter is also important for the complex vacua. Furthermore, there are several other constraints that are useful in this classification. Such constraints are very important for model-building purposes. Therefore, we give a complete list of all possible vacua, starting from a scalar potential with real coefficients, and analyse their properties, giving the constraints on the parameter space which allow for each solution.
The paper is organised as follows. In section 2 we set up some notation and discuss the potential. Section 3 is devoted to a general discussion of how the different vacua constrain the potential, whereas sections 4 and 5 present our results for the real and complex vacua. In section 6 we discuss connections between complex and real vacua, and in section 7 we briefly comment on the special case of λ 4 = 0, when the potential has an additional, continuous symmetry. Section 8 is devoted to a detailed discussion of spontaneous CP violation, and in section 9 we comment on dark-matter scenarios. Section 10 contains some concluding remarks. Technical issues are delegated to appendices A (mappings), B (positivity) and C (stationarity conditions). 2 The S 3 -symmetric potential 2

.1 Field notations
We consider S 3 -symmetric models with three SU(2) × U(1) reducible-triplet fields: (2.1) Allowing for complex vacuum expectation values, each field is decomposed as where ρ i is in general complex, whereas the fields η i and χ i are real. The reducible-triplet fields can alternatively be replaced by an S 3 doublet: and an S 3 singlet decomposed as where again w i and w S can be complex. Finally, the reducible-triplet fields may be replaced by a doublet and a pseudosinglet, denoted h A , in which case the potential will take a slightly different form. The potential, which has a quadratic and a quartic part, can be expressed either in terms of the reducible-triplet fields φ i , in terms of h 1 , h 2 , and h S , or in terms of h 1 , h 2 , and h A . The first two formulations are equivalent. It is useful to note that the (complex) vevs are related: with the inversion 8a) 8b) (2.8c) Whereas the formulation in terms of reducible-triplet fields is symmetric in φ 1 , φ 2 , φ 3 , the singlet-doublet representation is not. The decomposition into the doublet and singlet representations singles out a direction in terms of the φ fields. Any permutation of φ i fields in Eq. (2.3) would lead to an equally good definition for the components of the doublet. This is a trivial fact. However, this is the reason why in the tables of possible vacuum states that follow, some cases that are equivalent in terms of vacuum states of the reducible-triplet representation, given by (2.8), have to be split into different cases in terms of those of the irreducible framework (2.7) corresponding to different consistency conditions in terms of the minimisation of the potential.

The potential in terms of reducible-triplet fields
In terms of the reducible-triplet fields, the potential was written by Derman [20] as: There are ten different coefficients in these equations.

The potential in terms of S 3 singlet and doublet fields
In terms of the S 3 singlet and doublet fields, the potential can be written as [19,23,24]: In Appendix A we give the translation between the parametrisations of the potential in terms of reducibletriplet fields and the one in terms of singlet and doublet fields. Once again there are ten independent parameters. There are only four terms in this potential that are sensitive to the relative phase of different doublets, those in λ 2 , λ 3 , λ 4 and those in λ 7 . In terms of the reducible-triplet fields, in Eqs. (2.9) the number of such terms is higher since here we have γ, D, E 1 , E 2 , E 3 , and E 4 .
In this formulation it is clear that the potential has an extra Z 2 symmetry of the form h 1 → −h 1 . In terms of the equivalent doublet representation: which has also been used in the literature [25,26], the above symmetry translates into a symmetry for the interchange of the fieldsχ 1 andχ 2 . Another interesting feature is the fact that the choice λ 4 = 0 leads to a continuous SO(2) symmetry defined by: This shows that the term with λ 4 plays a special rôle in the potential.

The potential in terms of S 3 pseudosinglet and doublet fields
Instead of choosing the three Higgs doublets as being the singlet and the doublet irreducible representations of S 3 we may choose them to be a pseudosinglet, and the doubet. These are also irreducible representations. Under S 3 the pseudosinglet, h A , transforms into (−h A ). In this case there is no direct translation of these fields into the defining reducible representation.
In terms of the S 3 pseudosinglet and doublet fields, the potential can be written as Apart from the "trivial" substitution h S ↔ h A , the two formulations (2.10b) and (2.13b) differ in the λ 4 -term, the two doublet fields are interchanged: h 1 ↔ h 2 . Within the constraint of renormalizability (only quadratic and quartic terms) this scalar potential is equivalent to the previous one. However, this choice of representations will obviously have implications for the Yukawa sector. We do not examine these implications in the present work.
In the discussion of vacua, all results obtained for the irreducible framework in terms of the S 3 singlet and doublet can be trivially translated into this case. Therefore, our discussion will only refer to two different frameworks.

Positivity
Das and Dey have given necessary conditions for positivity [24]. For the general potential, the sufficient conditions are rather involved. However, in the case of λ 4 = 0, they can be expressed quite explicitly, and are given in Appendix B.
The vacua can be determined from the conditions that derivatives of the potential with respect to the three independent fields must vanish. These derivatives are linear in the coefficients of the potential, but cubic in terms of the (complex) vacuum expectation values. One approach would be to take the potential parameters as input, and solve these cubic equations for the vevs. In this section we shall follow another approach, which is to take the vevs as input, and use the derivatives to constrain the potential. The quartic potential will also be constrained by positivity and an imposed particle spectrum.
We shall start this discussion by first quoting the minimisation conditions in the two frameworks. Clearly, one and the same vacuum will be phrased differently in the two frameworks. But one framework may give a simpler description than the other.
After writing out these derivatives in the next subsections, we shall first discuss how these conditions constrain the potential. Then (in section 4), we review the real case (no CP violation), followed (in section 5) by a discussion of the complex case, which may accommodate spontaneous CP violation.

The reducible-triplet framework
Within the reducible-triplet framework, three complex derivatives must vanish: and ∂V /∂ρ * 2 and ∂V /∂ρ * 3 can be obtained by cyclic permutations. We note that these derivatives do not depend on C and C separately, only on the sum, C + C. Likewise, they only depend on E 2 and E 3 via their sum. This means that the vacuum conditions are independent of the space spanned by the two parameters orthogonal to these, namely C − C and E 2 − E 3 . However, the spectrum will depend also on these parameters.

The irreducible framework: singlet and doublet fields
The three relevant derivatives that must vanish are now We note that these derivatives do not depend on λ 5 and λ 6 separately, only on the sum, λ 5 + λ 6 . Likewise, they do not depend on λ 1 , λ 2 and λ 3 separately, only on two combinations orthogonal to λ 1 + λ 2 − 2λ 3 = 0.

Constraining the potential by the vevs
We are interested in the possibility of having spontaneous CP violation, therefore we impose that all the parameters of the potential should be real. Let us now consider the vevs as given a priori and solve the above minimisation conditions in terms of parameters of the potential. Our basic discussion will be in the reducible triplet framework. In this case the three vevs can be denoted as: and we can write six minimisation conditions by computing the derivatives of V with respect to each of the v i 's and of the τ i 's. It is clear from Eqs. (2.9) that, concerning phases, the potential is only sensitive to phase differences. In particular, we could choose without loss of generality a phase convention where one of these phases is rotated away, however, in this case we would loose symmetry among these equations. The explicit forms of these equations are given in Appendix C. As mentioned above, the pair of coefficients C and C as well as the pair E 2 and E 3 occur in each equation with a common factor, and therefore we are left with eight independent combinations of coefficients and five independent real equations which should be chosen as the three equations obtained from ∂V /∂v i = 0 and any pair of those from ∂V /∂τ i = 0. We could in principle solve these equations for any set of five of the eight independent parameters of the potential. These equations take the form: a 11 P 1 + a 12 P 2 + a 13 P 3 + a 14 P 4 + a 15 P 5 = b 1 , a 21 P 1 + a 22 P 2 + a 23 P 3 + a 24 P 4 + a 25 P 5 = b 2 , a 31 P 1 + a 32 P 2 + a 33 P 3 + a 34 P 4 + a 35 P 5 = b 3 , (3.7) a 41 P 1 + a 42 P 2 + a 43 P 3 + a 44 P 4 + a 45 P 5 = b 4 , a 51 P 1 + a 52 P 2 + a 53 P 3 + a 54 P 4 + a 55 P 5 = b 5 , where the P i denote different parameters of the potential. However not all of the possible 8 5 = 56 combinations will lead to five independent equations.
These five equations define five hyperplanes in the parameter space. In the case of the reducible-representation framework, since C and C appear together, as do E 2 and E 3 , we have effectively an 8-dimensional parameter space. Where the 5 hyperplanes intersect, we then have an 8 − 5 = 3-dimensional parameter space, over which the vacuum is the same.
The requirement for the five equations to be independent is that the determinant of the matrix A defined by: should be different from zero. It can readily be verified that the coefficients of the three parameters λ, A and (C + C) are not independent and therefore these equations cannot be solved simultaneously for these three parameters. The terms with these coefficients are not sensitive to the relative phases and therefore they do not appear in the equations obtained from differentiating with respect to the phases. As a result, in order to check this point it suffices to compute the 3 × 3 determinant involving the coefficients obtained from the first three minimisation conditions. This determinant is zero.
In the case of no spontaneous CP violation, the relative phases of the ρ i are zero and the corresponding minimisation condition, obtained from Eq. (C.3) and cyclic permutations, reduce to 0 = 0 since each term in these equations is proportional to the sine of relative phases. We are then left with only three independent equations and we can solve at most for three parameters of the potential.
Returning to the complex case, we are now ready to classify the vacua, according to how many independent equations we have. In order for the five equations to be independent, it is sufficient that one of these 56 determinants be non-zero. Conversely, in order for at most four of the equations to be independent, all 56 possible such 5×5 determinants must vanish.
For arbitrary vevs, we find that 16 out of the 56 possible 5 × 5 determinants vanish identically, whereas the remaining 40 are non-zero. The five equations (3.7) can for any of these choices be solved in terms of the five parameters P 1 , . . . , P 5 , with the exception of 5-parameter sets containing (C, C), (E 2 , E 3 ), (λ, A, C) or (λ, γ, E 1 , E 2 ). The complements of these account for 14 out of the 16 vanishing ones. The remaining two are (γ, A, C, E 1 , E 2 ) and (γ, C, D, E 2 , E 4 ). In these sets, C could be replaced by C, and E 2 by E 3 . The remaining 40 determinants factorise, and vanish when either In these cases we can have at most 4 independent equations among the set (3.7), and must investigate the corresponding 4 × 4 sub-determinants.
We shall distinguish the real and complex cases.

Real vacua
For a real vacuum, the five equations (3.7) discussed above reduce to a set of three. Again, they are not necessarily all independent. If we, for example, try to solve for λ, γ and A, the 3 × 3 determinant corresponding to (3.8) is particularly simple: Thus, when this quantity is non-zero, we can solve for λ, γ and A. Conversely, when det A 3×3 = 0 (meaning the sum of the vevs is zero, or two are equal), then we have at most two independent equations, and can for example only solve for λ and γ.
In the irreducible-representation framework, the three vacuum conditions (3.3)-(3.5) can be solved to give µ 2 0 and µ 2 1 in terms of the quartic coefficients: 1 The two equations (4.2b) and (4.2c) are not valid when w 1 = 0 and w 2 = 0, since they were derived from (3.4) and (3.5) dividing by w 1 = 0 and w 2 = 0, respectively. Furthermore, they are not automatically consistent. Consistency requires For w 1 = 0 the derivative of the potential with respect to w 1 is identically zero and therefore there is no clash in the determination of µ 2 1 from the derivative with respect to w 2 . From equation (4.3b) we see that these two derivatives are consistent if either λ 4 = 0 or w 1 = ± √ 3w 2 or else w S = 0. The case w S = 0 is special since if we now take into account the derivative of the potential with respect to w S , which is given by Eq. (3.3), we are left in the real case with which is the only term in Eq. (3.3) where w S does not appear as a factor. As a result, solutions with w S = 0 require in addition that λ 4 = 0 or w 2 = ± √ 3w 1 , or else w 2 = 0. See cases R-I-2 in Table 1. These do not require λ 4 = 0, while case R-II-3 has w S = 0 and requires λ 4 = 0.
1 There are misprints in the corresponding expressions given in Ref. [24], their Eq. (9): (i) a factor of 1/2 is missing on the right-hand side of all three expressions, and (ii) in µ 2 0 (µ 2 3 in their notation) the coefficient of λ 4 should be (v 2 /2v 3 )(v 2 2 − 3v 2 1 ). These misprints were corrected in the Erratum provided by the authors and included in Ref. [24].
The different solutions can be summarised as given in Table 1, where the descriptions in terms of both the reducible-and irreducible-representation frameworks are given. For the purpose of making this table as well as the corresponding one for complex vacua more compact, we introduce the abbreviations (4.5b) Table 1: Possible real vacua (partly after Derman and Tsao [21]). The classification of vacua uses the notation R-X-y, where R means that the vacuum is real. The roman numeral X is the number of constraints on the parameters of the potential that arise from solving the stationary-point equations. The letter y is used for distinguishing different vev's that have the same X, and λ a is defined in Eq. (4.5). Vacuum One should note that • Vacuum R-I-1 is a special case of Vacuum R-II-1. In this case, the vacuum value x is determined by • For Vacuum R-I-1, in the irreducible framework, we have which corresponds to Eq. (4.6), with w 2 S = 3x 2 .
• In the Vacua R-I-2a, R-I-2b, R-I-2c, the vacuum value x is determined by: In the irreducible framework, µ 2 0 is not constrained by Eq. (4.2a), whereas µ 2 1 = −λ − 1 2 γ is determined by the minimisation condition given above. Vacua R-I-2a, R-I-2b and R-I-2c, which correspond to w S = 0, require special discussion. It is clear from Eq. (4.2a) that µ 2 0 remains undetermined. According to Eqs. (4.2b) and (4.2c), in these vacua the following relation must hold: (Special cases are given in Table 1.) Using the translation given in Appendix A, and taking 2x 2 = w 2 1 + w 2 2 , this constraint is seen to be equivalent to (4.8).
These are reminiscent of vacua R-I-2a, R-I-2b and R-I-2c, with the interchange of w 1 and w 2 .
• In the Vacua R-II-1 and R-III, the two coefficients of the bilinear potential, (µ 2 0 and µ 2 1 ) or (λ and γ), can be determined from chosen vacuum values, together with the quartic potential.
• Vacua R-II are characterised by two independent vevs, referred to as x and y in the reducible-triplet framework, and as w and w S in the irreducible framework. In the framework of the reducible-triplet representation the three permutations presented as subcases a), b) and c) are trivial, however, it should be pointed out that in the irreducible framework different consistency conditions apply in each case, either • Vacuum R-III requires λ 4 = 0 or (in the reducible-triplet framework): In this sense, only two of the three minimisation equations are independent. As a special case of this solution, we can also have w S = 0. This is R-II-3, with µ 2 0 unconstrained. Table 1 clearly illustrates the point we have made before about the translation from the reducible-representation framework (RRF) to the irreducible one (IRF). The splitting of the R-I case into three cases (a, b, c) would be meaningless due to the S 3 symmetry, if we were only considering ρ 1 , ρ 2 and ρ 3 . However, in terms of w 1 , w 2 and w S they appear as different cases. The consistency of the derivatives with respect toŵ 1 andŵ 2 in this framework is verified since they all have w S = 0. However now, taking into account the derivative with respect to w S these solutions must obey Eq. (4.4) and each case fulfils this requirement in a different way. We have w 2 = 0, w 2 = √ 3w 1 and w 2 = − √ 3w 1 in the three cases. Another similar example is case R-II-1. Here the difference is that consistency of the derivatives with respect toŵ 1 andŵ 2 in the IRF is verified for w 1 = 0, w 1 = − √ 3w 2 and w 1 = √ 3w 2 , respectively. Since in this case we do not impose w S = 0 the constraint of Eq. (4.4) does not apply.
The high-scale validity of models based on two of these vacua, namely R-I-1 and R-II-1c, has recently been studied in Ref. [27].
Special limits. Some of the vacua listed in Table 1 can be seen as special limits 2 of another, more general case. These include • R-I-1 is contained in R-II-1a, 1b, 1c for w 2 = 0 (or x = y).

The Irreducible-Representation Framework (IRF)
As a prelude to studying the three complex minimisation equations, we may start with a simpler, linear combination of the last two. If we in the irreducible-representation framework adopt a convention where w S is real, and take with theŵ i real and non-negative, then we find the consistency condition (for w 1 = 0 and This condition, which is simpler than any of the individual derivatives (3.3)-(3.5), is a necessary, but not sufficient condition for the vacuum. If w 1 = 0, then equation (3.4) is identically zero, and as a result (3.4) and (3.5) are automatically consistent. The same does not apply to the case w 2 = 0 since this case requires λ 4 = 0 for Eq. (3.5) to be satisfied.
The generalisation of equation (4.3b) to the complex case is that the right-hand side of equation (5.2) be zero. This condition defines a hypersurface in a multidimensional parameter space. Whereas w S = 0 and λ 4 = 0 are possible solutions in the real case, they are not in the complex case, unless supplemented by additional conditions. For example, if w S = 0, we must also have Splitting this equation into real and imaginary parts, we find The solutions are (5.5b) Table 2: Complex vacua. Notation: ǫ = 1 and −1 for C-III-d and C-III-e, respectively; . With the constraints of Table 4 the vacua labelled with an asterisk ( * ) are in fact real.
Likewise, the condition λ 4 = 0 must be supplemented by Splitting this equation into real and imaginary parts, we find: Furthermore, Eq. (5.6) is obviously satisfied for On the other hand, for σ 2 = σ 1 ± nπ, n integer, (5.9) λ 2 , λ 3 and λ 7 are not constrained by equation (5.6). Finally, the real-case consistency conditionŵ 2 1 = 3ŵ 2 2 would in the complex case have to be supplemented with the above phase constraint (5.9).
We find the solutions given in Table 2. The table is organised as follows. From left to right, the first column gives the name of the vacuum, the second gives the specification in the irreducible-representation framework (IRF), and the third gives its translation to the reducible-representation framework (RRF).
The need to introduce the parameter ǫ in Table 2 results from the definitions given by Eq. (5.1), where theŵ i , i = 1, 2 are chosen to be non-negative. Naively one might expect the number of constraints to be equal to the number of free parameters of the solution. This is not the case, as can be illustrated by considering Eq. (C.7) in the limit w S = 0.
In the complex case there is a richer structure of possible vacua and again we have solutions that are similar when specified in terms of ρ's but not in terms of w's. Furthermore, in some cases solutions which can be described in an elegant way in one of the frameworks do not look so elegant in the other or may fall into a particular case of a more general one already given in that framework. As an illustration of the first remark let us consider the solution (ρ 1 , ρ 2 , ρ 3 ) = x(1, 1, e iτ ), this is a special case of C-III-a, with y = x, however the same solution after a permutation becomes a special case of C-III-h. On the other hand, the solution (ρ 1 , ρ 2 , ρ 3 ) = x(1, e iτ , e −iτ ) is a special case of C-III-i, with solution C-I-a a special case of this one again.
In Tables 3 and 4 we list the conditions on the potential parameters, in the irreduciblerepresentation framework.
The vacuum C-III-c (see Tables 2 and 3) falls in the category satisfying Eq. (5.5a). Examples of the constraints (5.7) are to be found in the vacua C-IV-b and C-IV-e (see Tables 2 and 4). The constraints (5.8) apply to the vacuum C-V (see Tables 2 and 4).
The case C-III-c of Table 2 is very interesting. It is a solution withŵ S = 0 and λ 4 = 0 with the additional constraint λ 2 + λ 3 = 0, allowing for a relative phase between the vevs of h 1 and h 2 . The fact thatŵ S is equal to zero suggests that this vacuum may provide a viable dark matter candidate. This vacuum can be specified in terms of two non-zero moduli and one single phase. Once we replace λ 2 in terms of λ 3 , the Lagrangian is left with only one term that is sensitive to the relative phase between h 1 and h 2 , to wit the term in λ 7 . The fact that the moduli of the two vevs are different might lead one to think that this vacuum violates CP spontaneously. However, in section 8 we show that this is not the case.
Cases C-IV-a, C-IV-d and C-V are listed in Table 2 for completeness and to allow for an enlightening discussion. Once one takes into consideration the constraints given in Table 4 they become real. Table 3: Constraints on complex vacua. Notation: ǫ = 1 and −1 for C-III-d and C-III-e, respectively. Where two possible signs (± or ∓) are given, they correspond to those of Table 2. Here, λ b is defined in Eq. (4.5).

Vacuum
Constraints Solution C-IV-d is more general than solution C-IV-a and reduces to C-IV-a once we fix w 2 = 0, so it suffices to discuss C-IV-d. Both of these require λ 4 = 0 and λ 7 = 0, and as a result the potential acquires symmetry for the transformation of h 1 , h 2 and h S under a unitary transformation of the form U = diag(e iτ , e iτ , 1) which allows to remove the phase σ 1 from the vacuum, making it real.
At first glance case C-V looks like the most general case, however we are assuming that it does not fall into any of the previous cases, so, as a result, full generality requires λ 2 + λ 3 = 0, λ 4 = 0 and λ 7 = 0 and there is no term in the potential sensitive to independent rephasing of each of the h fields. As a result any phase in the vevs can be rotated away. Under these circumstances, it is equivalent to a real set of vacua.
There are, in particular, two possible complex vacua that have been discussed previ- ously in the literature. One of them is: by Pakvasa and Sugawara [18]. We shall refer to this as the PS vacuum, assumingŵ = 0 andŵ S = 0. There is also a solution given by Ivanov and Nishi [22] we iσ ,ŵe iσ ,ŵ S , (5.11) which we shall refer to as the IN vacuum, assuming againŵ = 0 andŵ S = 0. By imposing the minimisation conditions it can be checked that both of these solutions require λ 4 equal to zero, corresponding to the SO(2) symmetry of the potential given by Eq. As discussed above, the conditions listed under IN-b lead to a real vacuum since they allow for the common phase of w 1 and w 2 to be rotated away.
Special limits. Some of the vacua listed in Table 2 can be seen as special limits of another, more general case. These include • C-III-a and C-III-h are equivalent in terms of the RRF.
• C-III-b, for λ 7 = 0 becomes real and falls into C-IV-a.

The Reducible-Representation Framework (RRF)
Below follow further general comments on the different vacua, as well as some constraints on the parameters in the reducible-representation framework: • Vacuum C-I-a requires • Vacuum C-III-h allows for a particular realisation in terms of the RRF with y = x, of the form C-RRF-a: (ρ 1 , ρ 2 , ρ 3 ) = x(1, e iτ , e iτ ). (5.17) Here, we have applied an overall phase rotation to the first solution presented in Table 2 for this vacuum, thus complying with the notation defined by Eq. (3.9). Depending on where we put the phase, we may have two different translations in terms of the IRF: one of them in C-III-h and the other in C-III-a. This leads to three independent minimisation conditions (as specified by the label III), two "radial" ones from ∂V /∂v 1 = 0 and ∂V /∂v 2 = 0 and one "angular" one from ∂V /∂τ = 0. The two radial equations are quadratic in cos τ , whereas the third is linear. With x and τ as input, these equations allow to constrain three parameters of the potential. Alternatively, one may remove terms quadratic in cos τ by forming linear combinations of the two radial equations. With the rescaling we find two radial equations 19b) and the angular equation Consistency of the first and the third expressions leads to Consistency of the second and third equation leads to: Invoking Eq. (5.18), we may solve for x 2 : (5.24) The conditions x 2 > 0 and | cos τ | ≤ 1 (5.25) will further constrain the potential parameters for this particular vacuum.
As presented in Table 2, Case C-III-h illustrates, once again, the fact that trivial permutations of the reducible triplet lead to different constraints for the IRF.
3 Vacuum C-I-a is of course a special case of this one, with τ = 2π/3.
There are three minimisation conditions, involving x 2 and cos τ . The constraint from the minimisation with respect to τ can be expressed as a cos τ -dependent relation among the quartic terms: whereas the others can be solved for γ and λ. Making use of Eq. (5.27), these take the form The three constraints (minimisation conditions) of Table 3 will for C-III-d, C-IIIe, and C-III-i take forms equivalent to these Eqs. (5.27)-(5.29). Equations (5.28) and (5.29) can be solved for x 2 , but the two solutions impose a cos τ -dependent consistency condition on the coefficients of the potential, given by Eq. (5.27).
• The vacua C-RRF-a and C-RRF-b have the same form as the PS and IN vacua of Eqs. (5.10)-(5.15). However, it must be stressed that they refer to the fields of the reducible-representation framework.

Complex vacua vs real vacua
The complex vacua, which are specified by three moduli and two (relative) phases are found as solutions of five conditions, whereas the real vacua are found as solutions of three.
The following questions then arise: Can the complex vacua be seen as generalisations of the real ones? Are the conditions on the moduli compatible with those for one or more of the real vacua? Are these more restrictive, or less restrictive? In order to discuss how a complex vacuum may be related to a real one, let us introduce some notations. Let us denote by C(C-X-y) the set of constraints (such as given in Tables 3  and 4) satisfied by a particular complex vacuum. Likewise, we let the real vacuum R-X ′ -y ′ satisfy the constraints C(R-X ′ -y ′ ) (see Table 1). Then, we may consider a real vacuum R-X ′ -y ′ the "origin" of a particular complex vacuum C-X-y if the following two conditions are satisfied: • the C-X-y specification, in an existing real limit for the vacuum (there may be an ambiguity of sign) coincides with that of R-X ′ -y ′ , and • the constraints are compatible, The latter condition is important due to the fact that the transition to a real vacuum is not always possible. We list in Table 5 the real vacua satisfying these two requirements.
In the following subsection, we study a particular example, how a complex vacuum is related to a real one. Table 5: Transitions from complex to real vacua in the IRF. The vacua labelled with an asterisk ( * ) were shown to be real.

Complex
Real "origin"

Transition from Vacua R-II-1b,1c to Vacuum C-III-h
An important difference between the constraints of the vacua R-II-1b,1c and C-III-h is that in the former case, the potential parameters λ 4 and λ 7 are free, i.e., they are uncorrelated.
For the vacuum C-III-h, on the other hand, they are correlated as (see Table 3) Modulo positivity and other physical constraints, the whole λ 4 -λ 7 -plane is available for the real vacuua R-II-1b,1c, whereas only the lines defined by Eq. (6.2) are available for an "extension" to a complex vacuum C-III-h. This holds even for infinitesimal phases, i.e., cos τ 2 → ±1.
The following question arises: Under what conditions is the complex vacuum deeper? It turns out that the difference can be expressed as being proportional to λ 4 or λ 7 : The question of relating complex vacua to real ones is relevant for the discussion of global minima [28,29] as well as to understand the possible correlations of different parameters of the potential.
7 The case of λ 4 = 0 As mentioned in section 2.3, in the case of λ 4 = 0 the potential has an additional, continuous SO(2) symmetry. This case was dismissed by Derman [20], as being "un-natural". This was due to the fact that this condition, when expressed in terms of the parameters of the potential written by Derman, given by Eqs. (2.9), acquires the form given by Eq. (4.10), which is not instructive and the resulting symmetry is not apparent. Spontaneous breaking of this SO(2) symmetry leads to massless particles. In this case, one way to promote this to a viable model is to break this symmetry softly, by adding a term to the bilinear part of the potential: with V 2 and V 4 as defined by equations (2.10). Choosing the minimisation conditions (3.3)-(3.5) will now become With these new conditions there will be some changes in the vacuum solutions. Notice that such a term also softly breaks the discrete symmetries h 1 going into −h 1 and h 2 going into −h 2 . Another possible choice for a term breaking softly the SO(2) symmetry is: Soft breaking terms involving h S and one h i are not consistent with λ 4 = 0. It was shown, in the context of two Higgs doublet models with a discrete symmetry, that CP can only be violated spontaneously once a soft symmetry breaking term is added to the potential [30]. Soft breaking of the S 3 symmetry of the scalar potential has been applied in [31] in order to obtain a special relation among the vevs of the three doublets that would allow to account for the observed charged lepton masses. An important implication of the type of vacuum solution and of the corresponding allowed region of parameter space is the resulting different possible spectra for the physical scalars.

Spontaneous CP violation
The S 3 -symmetric potential offers a very rich phenomenology, and can accommodate a variety of physical situations, as outlined in sections 4 and 5, where we classified the different vacua.
We assumed, for simplicity, that all parameters of the potential are real. Therefore our discussion is done in the framework of explicit CP conservation. This raises the question of whether or not CP can be violated spontaneously. For that purpose we can inspect the list of complex solutions presented in Table 2. CP can only be spontaneously violated if the Lagrangian is invariant under CP and if at the same time there is no transformation that can be identified with a CP transformation, leaving both the Lagrangian and the vacuum invariant. The idea of spontaneous CP violation was first proposed by T. D. Lee [3] in the context of two Higgs doublets. In the context of the SM, with a single Higgs doublet, a CP transformation of the scalar doublet amounts to its complex conjugation and the scalar sector cannot violate CP. In models with several Higgs doublets complex conjugation may be combined with a unitary transformation acting on the set of doublets, since this transformation leaves the kinetic energy term of the Lagrangian invariant. In this case the most general CP transformation is given by: with U an arbitrary unitary matrix 4 . This equation together with the assumption that the vacuum is CP invariant: leads to the following condition [32]: implying that there is spontaneous CP violation if none of the CP symmetries allowed by the Lagrangian satisfy this equation. For real vevs this condition is obviously verified.
If the Lagrangian has a discrete symmetry one must take it into consideration before drawing conclusions. In the discussion that follows we do not take the Yukawa sector into consideration. We now comment on each one of the cases presented in Table 2 concerning the possibility of having spontaneous CP violation: • The case C-I-a is a familiar one that has been discussed long ago in the framework of the reducible representation [32]. It was pointed out that it has complex vacuum expectation values with calculable non-trivial phases, assuming geometrical values, entirely determined by the symmetry of the scalar potential. These phases cannot be rotated away and yet they do not lead to spontaneous CP violation, since there is a matrix U satisfying the constraint of Eq. (8.3), namely: 4) which is at the same time a symmetry of the potential.
In terms of the irreducible-representation framework we can write this solution as (±iŵ 1 ,ŵ 1 , 0 ) and the matrix U satisfying the constraint (8.3) becomes U = diag(−1, 1, 1) making use of the symmetry of the potential for h 1 → −h 1 . It was shown [32] that solutions with calculable phases whose values are independent of the coupling constants of the scalar potential do not necessarily conserve CP. Characteristic features of such solutions in models with several Higgs doublets as well as the interplay between symmetries and geometrical CP violation have been analysed by several authors [33][34][35][36][37][38][39].
• Case C-III-a allows for a nontrivial phase which can be determined as a function of λ 4 , and λ 7 , as shown in Table 3. This solution violates CP spontaneously.
• Reasoning analogous to that for C-I-a can be applied to cases C-III-b, C-III-d, C-III-e where again the matrix U given above, U = diag(−1, 1, 1), satisfies Eq. (8.3) in terms of the irreducible representation framework. On the other hand, cases C-III-f and C-III-g require λ 4 = 0 and therefore the potential acquires an additional SO(2) symmetry. In these cases U can be chosen as U = diag(−1, −1, 1). Case C-IV-b also requires λ 4 = 0, as a result the potential is also symmetric under h 2 → −h 2 and one can choose U = diag(1, −1, 1).
• Case C-III-c is a very interesting one. At first sight it looks as if it may violate CP spontaneously, however, this is not the case. In order to prove that case C-IIIc does not violate CP spontaneously we start from the corresponding set of vevs (ŵ 1 e iσ ,ŵ 2 , 0) and perform a Higgs basis transformation on the Higgs doublets h 1 and h 2 by an SO(2) rotation into: such that the vevs of the new S 3 doublet fields now have the same modulus and are of the form (ae iδ 1 , ae iδ 2 , 0). This requires tan 2θ =ŵ Notice that (ae iδ , ae −iδ , 0) is a special case of the PS vacuum, given in Eq. (5.10).
We have checked that this case does not lead to spontaneous CP violation even when the two soft breaking terms discussed in section 7 are included.
• C-III-f,g are discussed above together with C-III-b, CP is not spontaneously violated.
• It is clear from this discussion that, in general, C-III-h can violate CP and as we can see from Table 3 the phase is determined by parameters of the potential.
• For C-III-i we can verify Eq. (8.3) in the reducible representation framework with U acting as a permutation between the two vevs with modulus y and there is no spontaneous CP violation.
• Solution C-IV-a is in fact real, as discussed in section 5, since it requires λ 4 and λ 7 to be zero and therefore CP is not violated.
• C-IV-b is discussed above together with C-III-b, CP is not spontaneously violated.
• Case C-IV-c has no SO(2) symmetry because λ 4 is different from zero. As a result Eq. (8.3) cannot be verified in general and therefore CP can be violated.
• For C-IV-d again λ 4 and λ 7 must be zero and the same reasoning followed for C-IV-a leads to the conclusion that CP is not violated.
• For C-IV-e the reasoning is similar to the one in case C-III-c. In order to prove that C-IV-e does not violate CP spontaneously we start with the corresponding set of vevs: (ŵ 1 e iσ 1 ,ŵ 2 e iσ 2 ,ŵ S ) whereŵ 1 = − sin 2σ 2 sin 2σ 1ŵ 2 (8.10) in this phase convention. In general one should write sin(2σ 1 − 2σ S ) and sin(2σ 2 − 2σ S ) in the latter relation, where σ S would be the phase of the third vev. We now perform an SO(2) rotation, similar to the one specified above, with tan 2θ =ŵ which once again will lead to equal moduli for the S 3 doublet fields. In this case, the vevs will acquire the form (be iγ 1 , be iγ 2 ,ŵ S ). Unlike in case C-III-c, an overall phase rotation would also affect the vev of h S . However, it turns out that condition (8.10) enforces γ 1 + γ 2 = 0. (8.12) This SO(2) rotation takes us to the PS vacuum.
• For C-IV-f there is no symmetry of the potential allowing to verify Eq. (8.3) and therefore CP can be violated.
• Solution C-V looks like the most general case but the constraints imposed on the parameters of the potential make it equivalent to a real set of vacua as discussed in section 5, so that there is no spontaneous CP violation.
The PS vacuum specified by Eq. (5.10) requires λ 4 = 0 and therefore there is symmetry under the interchange of the components of the S 3 doublet. As a result, it is possible to verify Eq. (8.3) and CP is conserved.
The IN vacuum specified by Eq. (5.11) also requires λ 4 = 0. It is clear from the previous discussion that this solution does not lead to spontaneous CP violation. In fact, from Eqs. (5.14) and (5.15) it is clear that the allowed region of parameter space where this solution minimises the potential is such that either the phase σ can be rotated away and therefore is not physical (IN-b) or it is fixed as ±π/2 (IN-a) falling into one of the cases C-III-f or C-III-g which were already discussed above.
We summarise these cases in Table 6. In this table, we also indicate whether or not λ 4 is equal to zero. One may conclude that S 3 symmetric models with λ 4 = 0 cannot violate CP spontaneously. Still, there are cases with λ 4 = 0 where spontaneous CP violation may occur in three Higgs doublet models with S 3 symmetry.
Inert dark matter has also been studied in the context of three-Higgs-doublet models without an S 3 symmetry [55][56][57]. Models with three Higgs doublets have a richer phenomenology than those with only two. A strong motivation for such an extension is the possibility of having CP violation in the scalar sector [55,56]. In these models dark matter is also stabilised via a Z 2 symmetry.
Dark matter has been proposed within S 3 -symmetric models, exploiting fields that get a vanishing vev. Models of this kind include those where the singlet plays the rôle of the SM Higgs, whereas the S 3 doublet provides dark matter [58][59][60]. An alternative way to embed dark matter could be to have the S 3 singlet as inert in a solution where it has a zero vev, such as C-III-c. For the C-III-c solution a specific example would correspond to the following S 3 representation assignments for the quarks: with a different set of coefficients (a, b, c) for the up and down quark sector. Since solution C-III-c requires λ 4 to vanish, no term in the potential will break this Z 2 symmetry. The C-III-c vacuum breaks spontaneously the SO(2) symmetry obtained from having λ 4 = 0, therefore, one way to obtain a realistic scalar spectrum is to include the additional soft breaking term given by Eq. (7.2). This has significant consequences for the solution of the new minimisation conditions. The vacuum transforms into: for a well defined region of parameter space. This was obtained by requiringŵ S = 0 and λ 4 = 0, but relaxing the condition λ 2 + λ 3 = 0. At this stage, this should be seen as a toy model. In fact it has been known since long ago that this implementation leads to an unrealistic V CKM matrix, with only two-by-two mixing [61]. A full analysis of possible realistic implementations generating the observed fermion masses and mixing is beyond the scope of this paper.

Concluding remarks
The S 3 -symmetric potential, with three doublets, is specified in terms of 10 parameters. It can accommodate 2 charged Higgs pairs and 5 neutral ones. If their masses were to be specified freely, one would need 7 parameters, leaving 10 − 7 = 3 "free". On the other hand, if we consider an arbitrary vacuum, then 5 minimisation conditions have to be satisfied, determining 5 parameters. This mis-match illustrates that the spectrum can not be chosen freely, it will be constrained. Alternatively, one might pick a vacuum for which not all 5 minimisation conditions are independent. This would be the case, for example, when one vev vanishes. Residual symmetries of the potential after spontaneous symmetry breaking play a very important rôle in constraining the possibility of having spontaneous CP violation [21,22,65]. In Ref. [21] it is proved that real vacua of S 3 symmetric 3HDM always preserve an S 2 symmetry, whilst constraints on complex-valued minima are much less severe and there are complex minima which totally break the S 3 symmetry.
The transition from real to complex vacua is not trivial. Our work is done in the context of explicit CP conservation. Table 1 illustrates a point that had already been emphasized in Ref. [21], which is that for real vacua, in the reducible representation framework, without imposing a condition equivalent to λ 4 = 0, the only allowed solutions with all three vevs different from each other are (x, −x, 0) and their permutations. This is the reason why real vacua always preserve an S 2 symmetry. Complex vacua, on the other hand, can evade this restriction as can be seen, for example, from solution C-IV-f.
In this work we focused our attention on the study of the scalar potential. The first necessary step to render such models realistic is to specify how the fermions transform under S 3 and how to generate a realistic CKM matrix [62,63] (see also Ref. [64]).

Acknowledgments
It is a pleasure to thank Gustavo Branco for discussions. PO and MNR thank the CERN Theory Division for hospitality and partial support, where part of this work was done. The authors also thank the University of Bergen and CFTP/IST Lisbon, where collaboration visits took place. PO is supported by the Research Council of Norway. This work was partially supported by Fundação para a Ciência e a Tecnologia (FCT, Portugal) through the projects CERN/FIS-NUC/0010/2015, CFTP-FCT Unit 777 (UID/FIS/00777/2013) which are partially funded through POCTI (FEDER), COMPETE, QREN and EU. DEC is currently supported by a postdoctoral fellowship from FCT Unit 777.

A Converting between the two frameworks
The potentials in the reducible-representation framework, Eqs. (2.9), and the irreduciblerepresentation framework, Eqs. (2.10), are related as follows:

B Positivity
Das and Dey [24] have discussed necessary positivity conditions. Here, we discuss necessary and sufficient positivity conditions, following the approach of refs. [55,66]. In the general case, these are rather involved. However, for the case of λ 4 = 0, they can be expressed in explicit form.

B.1 General formulation
We start by rewriting the Higgs SU(2) doublets as: where ||h i || are the norms of the spinors, andĥ i are unit spinors. We let the norms of Eq. (B.1) be parametrised as follows: The complex product between two different unit spinors will be a complex number with modulus less than or equal to unity, i.e.

B.2 The necessary conditions of Das and Dey
In Eq. (4) of Das and Dey [24], they have listed seven necessary (but not sufficient) conditions for positivity. They can be reproduced by looking at the boundaries in θ-γ space.
which is Eq. (4b) of Das and Dey.
This can be treated in the same way as was done in the 2HDM [66], and is equivalent to the three conditions: These are equivalent to Eqs. (4b), (4c), (4e) and (4f) of Das and Dey.

C Minimisation conditions in terms of moduli and phases
Here, we present explicit results for the derivatives of the potential (the minimisation conditions) in terms of moduli of the vevs, and their phases.