Reliability and Survivability Analysis of Data Center Network Topologies

Abstract

The architecture of several data centers have been proposed as alternatives to the conventional three-layer one. Most of them employ commodity equipment for cost reduction. Thus, robustness to failures becomes even more important, because commodity equipment is more failure-prone. Each architecture has a different network topology design with a specific level of redundancy. In this work, we aim at analyzing the benefits of different data center topologies taking the reliability and survivability requirements into account. We consider the topologies of three alternative data center architecture: Fat-tree, BCube, and DCell. Also, we compare these topologies with a conventional three-layer data center topology. Our analysis is independent of specific equipment, traffic patterns, or network protocols, for the sake of generality. We derive closed-form formulas for the Mean Time To Failure of each topology. The results allow us to indicate the best topology for each failure scenario. In particular, we conclude that BCube is more robust to link failures than the other topologies, whereas DCell has the most robust topology when considering switch failures. Additionally, we show that all considered alternative topologies outperform a three-layer topology for both types of failures. We also determine to which extent the robustness of BCube and DCell is influenced by the number of network interfaces per server.

Appendices

Appendix 1: MTTF Approximation

In this appendix we obtain Eq. 6, derived from the combination of Eqs. 4 and 5. First, we replace the reliability R(t) in Eq. 4 by the reliability approximation given by Eq. 5, resulting in

$$\begin{aligned} MTTF = \int _{0}^{\infty } R(t) \, dt \approx \int _{0}^{\infty } e^{-\frac{t^rc}{{E[\tau ]}^r}} \, dt. \end{aligned}$$

(19)

Hence, we find the MTTF by evaluating the integral in the rightmost term of Eq. 19. The evaluation starts by performing the following variable substitution:

$$\begin{aligned} t=x^{\frac{1}{r}} \Leftrightarrow dt = \frac{1}{r}x^{\left( \frac{1}{r}-1\right) } \, dx. \end{aligned}$$

(20)

Note that the interval of integration in Eq. 19 does not change after the variable substitution, since $t=0$ results in $x=0$ and $t \rightarrow \infty$ results in $x \rightarrow \infty$. Hence, after the variable substitution, we can write Eq. 19 as:

$$\begin{aligned} MTTF \approx \frac{1}{r}\int _{0}^{\infty } x^{\left( \frac{1}{r} - 1\right) } e^{\frac{-xc}{{E[\tau ]}^r}} \, dx. \end{aligned}$$

(21)

The integral of Eq. 21 is evaluated using the gamma function defined as [21]:

$$\begin{aligned} \varGamma (z) = k^z \int _{0}^{\infty } x^{z-1}e^{-kx} \, dx, ({\mathfrak {R}}z > 0 , {\mathfrak {R}}k > 0). \end{aligned}$$

(22)

For better clarity, we rewrite the integral of Eq. 22 as:

$$\begin{aligned} \int _{0}^{\infty } x^{z-1}e^{-kx} \, dx = \frac{\varGamma (z)}{k^z}. \end{aligned}$$

(23)

We make $z = \frac{1}{r}$ and $k=\frac{c}{{E[\tau ]}^r}$ in Eq. 23 and multiply its both sides by $\frac{1}{r}$, obtaining

$$\begin{aligned} \frac{1}{r} \int _{0}^{\infty } x^{\left( \frac{1}{r}-1\right) }e^{-\frac{xc}{{E[\tau ]}^r}} \, dx = \frac{1}{r}\frac{\varGamma \left( \frac{1}{r} \right) }{{\frac{c}{{E[\tau ]}^r}}^{\frac{1}{r}}} = \frac{E[\tau ]}{r}\root r \of {\frac{1}{c}} \varGamma \left( \frac{1}{r} \right) . \end{aligned}$$

(24)

Note that the leftmost term in Eq. 24 is the MTTF approximation given by Eq. 21. Hence, we can write the MTTF as:

$$\begin{aligned} MTTF \approx \frac{E[\tau ]}{r}\root r \of {\frac{1}{c}} \varGamma \left( \frac{1}{r} \right) . \end{aligned}$$

(25)

Appendix 2: Comparison of MTTF Equations for Link Failures

In BCube we have $MTTF_{bcube} \approx \frac{E[\tau ]}{l+1}\root l+1 \of {\frac{1}{|{\mathcal {S}}|}} \varGamma \left( \frac{1}{l+1} \right)$. Hence, we will start by showing that if we have a new configuration with $l'=l+1$ (i.e., one more server interface) we can increase the MTTF. For simplicity, we consider that $|{\mathcal {S}}|$ is equal for the configurations using both l and $l'$. Although it is not necessarily true, because the number of servers depends on the combination of l and n, we can adjust n to have a close number of servers for l and $l'$, as done on the configurations of Table 1. First, we need to state that

$$\begin{aligned} \frac{E[\tau ]}{l'+1}\root l'+1 \of {\frac{1}{|{\mathcal {S}}|}} \varGamma \left( \frac{1}{l'+1} \right) > \frac{E[\tau ]}{l+1}\root l+1 \of {\frac{1}{|{\mathcal {S}}|}} \varGamma \left( \frac{1}{l+1} \right) . \end{aligned}$$

(26)

Doing $l'=l+1$, and rearranging the terms we have the following requirements for the above formulation to be true:

$$\begin{aligned} |{\mathcal {S}}| > {\left( \frac{l+2}{l+1} \frac{\varGamma \left( \frac{1}{l+1} \right) }{\varGamma \left( \frac{1}{l+2} \right) } \right) }^{(l+1)(l+2)}. \end{aligned}$$

(27)

The right term of Eq. 27 is a decreasing function of l over the considered region ($l \ge 1$). Hence, it is sufficient to prove that Eq. 27 is true for $l=1$. Doing $l=1$ in Eq. 27, we have $|{\mathcal {S}}| > 0.955$, which is true for a feasible DC.

As DCell with $l>1$ has the same MTTF of a BCube with the same l, the above reasoning is valid for this topology. For DCell2 ($l=1$), the equation of the MTTF is the same of BCube2 ($l=1$) except that DCell2 has the value $\sqrt{\frac{1}{1.5|{\mathcal {S}}|}}$ instead of $\sqrt{\frac{1}{|{\mathcal {S}}|}}$. Consequently, the MTTF of BCube2 is greater than that of DCell2. We can thus conclude that DCell2 has the lowest MTTF among server-centric topologies. Hence, to show that the MTTF of Fat-tree is smaller than the MTTF of all server-centric topologies, we compare it to DCell2. We thus need to prove that

$$\begin{aligned} \frac{E[\tau ]}{|{\mathcal {S}}|} < \frac{E[\tau ]}{2}\sqrt{\frac{1}{1.5|{\mathcal {S}}|}} \varGamma \left( \frac{1}{2} \right) . \end{aligned}$$

(28)

The solution of this equation is $|{\mathcal {S}}| > 1.909$, which is always true considering a real DC.