Performance evaluation of P2P-TV diffusion algorithms under realistic settings

Paolo Veglia; Dario Rossi

doi:10.1007/s12083-012-0126-x

Peer-to-Peer Networking and Applications

March 2013, Volume 6, Issue 1, pp 26–45 | Cite as

Performance evaluation of P2P-TV diffusion algorithms under realistic settings

Authors
Authors and affiliations

Paolo VegliaEmail author
Dario Rossi

Article

First Online: 18 February 2012

Received: 18 March 2011

Accepted: 31 January 2012

213 Downloads
3 Citations

Abstract

Internet video and peer-to-peer television (P2P-TV) are attracting more and more users: chances are that P2P-TV is going to be the next Internet killer application. In recent years, valuable effort has been devoted to the problems of chunk-scheduling and overlay management in P2P-TV systems. However, many interesting P2P-TV proposals have been evaluated either in rather idealistic environments, or in the wild Internet. Our work sits in between these two antipodean approaches: our aim is to compare existing systems in a controlled way, but taking special care in realistic conditions for their evaluation at the same time. We carry on a simulation analysis that considers several factors, modeling the L7 overlay (e.g., chunk scheduling, topology management, overlay topology, etc.), the L3 network (e.g., end-to-end latency models, fixed vs dynamic conditions, etc.), and the interaction of both layers (e.g., measurement errors, loss of signaling messages, etc.). To depict a comprenshive system view, results are expressed in terms of both user-centric and network-centric metrics. In a nuthshell, our main finding is that P2P-TV systems are generally robust against measurement errors (e.g., propagation delay or capacity estimation), but are on the contrary deeply affected by signaling errors (e.g., loss or outdated system view), which are often overlooked without justification.

Keywords

P2P-TV Simulations QoE Network awareness

This is a preview of subscription content, to check access.

Notes

Acknowledgements

This work was supported by the European Commission under the FP7 STREP Project Network-Aware P2P-TV Application over Wise Networks (NAPA-WINE).

Supplementary material

Open image in new window

Appendix

This section describes our careful selection of some crucial parameter of our simulations campaign, such as (i) the population size and stability, (ii) the sharing ratio and (iii) the upload bandwidth. We stress that our choice was to gather simulation scenarios that are as representive as possible of the Internet and real-life: hence, we used own measurement, or other relevant measurement performed by colleagues in the scientific community, to derive a realistic set of simulation parameters.

A.1 Population size and stability in real P2P-TV systems

To justify our assumption of absence of churn in the population, we show in Fig. 9 the stability of a real user base population in the operational network of a major European ISP that we continuously monitor in the context of the NAPAWINE project [42]. For the same project, we have developed state of art P2P-TV classifiers, that are either based on the stochastic analysis of the packet payload (KISS [43]) or on the behavioral analysis of the connection pattern (Abacus [44]). The classifiers are comparably accurate [45, 46] and an open source implementation is available at [47]. We run the classifiers on several probes in different major European ISP, so that we are able to recognize the traffic of popular applications such as PPlive [48], SopCast [49], TVAnts [50], as they are used by real user in operational networks.

Fig. 9
Temporal evolution of the number of peers, received and sent traffic volume during a typical P2P-TV event (SopCast application), at a PoP of a major European ISP that we continuously monitor

While in general the usage of P2P-TV application is episodic, as it is driven by a specific program—rather typically, a sport event—during the event the population remains extremely stable. We support this statement with the help of Fig. 9, which reports the temporal evolution of the number of peers, depicted with a star point on the right y-axis, during a typical sport event streamed by the popular SopCast [49] application gathered during a Championship match in April 2009. Each point in the picture reports a measurement related to 5 s, and we sub-sample the observation points for the sake of readability. The picture also reports, on the left x-axis, the received (plus sign) and sent (cross sign) bitrate in Mbps. As it can be seen from the picture, peers arrive in a flash-crowd pattern starting from 20h30 (thus prior that the match begins), while during the whole 1h30-long soccer match the peer population keeps extremely stable to about 100 peers (i.e., the value that we actually simulated). Then, immediately after the end of the match, peers rapidly depart and the system empties. Also, noticeable from the picture, the traffic contributed by the peer behind the residential point of presence (PoP) is lower than the received traffic, which is due to the asymmetry typical of ADSL lines.

This pattern is very common in our measurements and justify our environment choice of absence of churn. First, by focusing on steady-state performance of different algorithms, we gather results that are not only more relevant (as they pertain to the whole duration of the show, rather that to a startup phase), but are also statistically more significant with respect to performance gathered during a transient phase (i.e., during arrival or departure). Second, it is likely that the algorithm exhibiting the best performance in steady-state, will also be the best candidate in the transient phase. Third, these results are also more relevant from the end-user perspective, as users are clearly interested in the QoS during the match, while they are likely less interested in the system performance prior that the match begins: indeed, notice how arrival time roughly uniformly distributes in the 30 min preceding the match, which is more likely tied to user “warm-up” of the channel (i.e., joining the P2P-TV system in advance to be sure seeing the first kick of the match) and personal habits rather than reflecting actual interactive usage of the channel. Finally, we point out that churn could not instead be disregarded in case the performance metrics of interest pertained to the transient phase (e.g., the duration of the warmup phase needed to reach a steady state) or to resilience aspects of the system (e.g., , measuring the system response in case everybody changed the channel during the break of the football match), which are however out of the scope of this paper.

A.2 Sharing ratio

Sharing ratio k is a fundamental and critical parameter in every P2P system since it describes its capacity to disseminate data to all nodes: k is defined as

$$ k=\frac{{\displaystyle{\ensuremath{\overline{BW_{U}}}}}} {\lambda_{\rm video}} =\frac{{\displaystyle\sum\limits_{i\in{N_H}}{BW_U}_i}} {N_H\cdot\lambda_{\rm video}} $$

(3)

or the ratio between the total uplink capacity (i.e. the sum of uplink capacity of all nodes) and total inbound traffic; intuitively a value of k > 1 means that nodes uplink capacity is sufficient for the system to be self-sustainable as long as algorithms are good enough to exploit capacity. On the contrary, in a scenario where k is < 1, total up-link capacity is not high enough to guarantee that every peer receives the entire stream. Notice that, as the value of k provided by common ADSL peers in actual system is between 0.2 and 0.3 [16], the correct working of the system is guaranteed by the presence of “amplifier” nodes providing the missing capacity.

Notice that scheduling algorithm may not be able to succesfully exploit the available system capacity even for k > 1. Considering for simplicity the case of a single-class homogeneous population, top plot Fig. 10 show for instance the CDF of the chunk delay for the naïve ru/r scheduler, where it can be seen that for k = 1, almost half of chunks are lost and even when the capacity is twice k = 2 the needed sustainable rate, the system still experiences a non negligible amount of 1.4% losses.

Fig. 10
Delay distribution of ru/r and lu/pa schedulers for different values of the sharing ratio k

In case more sophisticated schedulers are used, such as the power-aware lu/pa in the bottom plot of Fig. 10, we notice that the system is almost lossless for a sharing ratio of k = 1.5. Hence, in our simulation we designed the class population to match the factor k = 1.5, so that the system is self-sustainable, which allows to isolate the impact of other factors (e.g., L3 network, L7 schedulers, L3/L7 interactions) on an otherwise lossless system.

In addition, an interesting aspect emerges from this analysis: comparing the k = 1.5 homogeneous single-class system of Fig. 10 to the k = 1.5 heterogeneous multi-class system shown early in Fig. 2a, we see that peer heterogeneity plays a non marginal role in improving global efficiency [51]. Intuitively, having peers with different capacity is beneficial because high capacity peers, which can handle a greater number of active connections, will occupy the high portion of the istantaneous chunk distribution tree, close to the source, possibly as a result of topology management. On the other hand, peers with poor connectivity can occupy far positions in each chunk distribution tree, since they can serve a lower number of neighbors (or even none).

A.3 Mean upload bandwidth

In Appendix A.2 we fixed a value for parameter k, binding video rate to mean upload bandwidth: hence, we need to investigate sound values for the peer upload capacity. To the best of our knowledge, there are no studies or tools, as for instance the Meridian project for latencies, which estimate end-host bandwidth distribution. Moreover, even if there were any, the gathered data would likely be strongly dependent on countries or service provider. For these reasons we perform a survey, to gather rough boundaries for the uplink capacity, as well as a sensitivity analysis, to gather reliable simulation results between these boundaries.

Again, for the sake of simplicity, we consider an homogeneous peer population, where each peer belongs to a single class with upload bandwidth BW _U. We perform a sensitivity analysis by carrying on several simulations varying the relative magnitudo of the propagation and trasmission time, by varying BW _U. Notice moreover that latencies, chunk-size and buffer-map size are kept constant. In more detail, we denote by γ the ratio between the propagation delay and the chunk upload time:

$$ \label{eq:2} {\gamma}=\frac{{\ensuremath{\overline{M}}}}{t_{\rm TX}} ={\ensuremath{\overline{M}}}\frac{BW_U}{C} $$

(4)

where ${\overline{M}}$ is the average Meridian end-to-end latency, t _TX is the chunk transmission time and C the average chunk size. Intuitively γ indicates which component of the delay has the largest influences on the total chunk transfer time. For instance, when γ < 1 the transmission delay is greater than the propagation delay, so that the total chunk delay reduces in case of bandwidth-awareness; conversely, when γ > 1, propagation delay is the largest component of total chunk transmission time, which would thus reduce in case of latency-awareness.

To gather valid boundaries for γ we surveyed the average uplink capacity of different European countries [52], to gather upper (CZ) and lower (IT) bounds of $\overline{BW_U}$ (and, hence, of γ). Figure 11 depicts several performance metrics (i.e., chunk losses, mean and 95th percentile of the total chunk delay) as a function of γ varying in the range resulting from the survey. As γ grows, we notice a smooth decrease of the delay curves, which is an expected consequence of the transmission time reduction due to higher upload bandwidth. Clearly, this decrease is not an artifact occasioned by greater losses (as gathered early in Section 5.2 but a real gain), which is confirmed by the consistently low loss rate in the same interval.

Fig. 11
Sensitivity analysis of delay and losses as a function of the propagation-delay over transmission-delay ratio γ. *Top* x-axis represent the average per-country bandwidth reported in [52], with landmarks for the lower (Italy, IT) and *upper* (Czech republich, FR) bounds; with a *thick line*, we report the $\overline{BW_U}$ value we selected in this paper

The vertical thick line in between IT and CZ references represent the working point that we selected for our simulation, which can be thought to represents an average European country. Two considerations hold: first, notice that, as loss rate remains steady over the whole interval, we can expect the results shown earlier in this paper, to hold for a number of different European countries. Second, we gather that, as γ varies in the selected range, the delay can vary by almost one order of magnitude: hence, our simulation results should be considered as representative of an average country, and we can expect delay results to (roughly linearly) vary depending on the actual value of $\overline{BW_U}$ in the country of interest.

References

1.
Cisco visual networking index: forecast and methodology 2009–2014 (2010). http://www.cisco.com-en-US-solutions-collateral-ns341-ns525-ns537-ns705-ns827/white_paper_c11-481360_ns827_Networking_Solutions_White_Paper.html
2.
Cisco visual networking index: Global mobile data traffic forecast update, 2009–2014 (2010). http://www.cisco.com-en-US-solutions-collateral-ns341-ns525-ns537-ns705-ns827/white_paper_c11-520862.html
3.
Spotify consumes more internet capacity than all of Sweden. http://techcrunch.com/2010/03/16/live-blog-spotify-ceo-daniel-eks-keynote-interview/. Accessed 16 March 2010/Sept 2010
4.
Project pheon homepage. http://www.utorrent.com/labs/pheon. Accessed Dec 2010
5.
Adobe real time media flow protocol (rtmfp). http://labs.adobe.com/technologies/stratus/. Accessed Oct 2010
6.
Fox G, Pallickara S (2002) The Narada event brokering system: overview and extensions. In: Parallel and Distributed Processing Techniques and Applications, PDPTA ’02. Las Vegas, USAGoogle Scholar
7.
Tran D, Hua K, Do T (2004) A peer-to-peer architecture for media streaming. IEEE J Sel Areas Commun 22:121–133CrossRefGoogle Scholar
8.
Zhang J, Liu L, Ramaswamy L, Pu C (2008) PeerCast: churn-resilient end system multicast on heterogeneous overlay networks. J Netw Comput Appl 31(4):821–850CrossRefGoogle Scholar
9.
Sherwood R, Lee S, Bhattacharjee B (2006) Cooperative peer groups in NICE. Comput Networks 50(4):523–544MATHCrossRefGoogle Scholar
10.
Castro M, Druschel P, Kermarrec A, Nandi A, Rowstron A, Singh A (2003) Splitstream: high-bandwidth multicast in cooperative environments. In: Proceedings of the 19th ACM symposium on operating systems principles. Bolton Landing, USAGoogle Scholar
11.
Manzato D, da Fonseca N (2008) Incentive mechanism for the CoopNet network. PPNA 1:29–44Google Scholar
12.
Rejaie R, Ortega A (2003) PALS: peer-to-peer adaptive layered streaming. In: Proceedings of the 13th international workshop on Network and Operating Systems Support for Digital Audio and Video, NOSSDAV ’03. ACM, MontereyGoogle Scholar
13.
Li B, Xie S, Qu Y, Keung G, Lin C, Liu J, Zhang X (2008) Inside the new coolstreaming: principles, measurements and performance implications. In: Infocom 2008. IEEE, PhoenixGoogle Scholar
14.
Pianese F, Perino D, Keller J, Biersack E (2007) PULSE: an adaptive, incentive-based, unstructured P2P live streaming system. IEEE Trans Multimedia 9(8):1645–1660CrossRefGoogle Scholar
15.
Magharei N, Rejaie R (2009) Prime: Peer-to-Peer receiver-driven mesh-based streaming. IEEE/ACM Trans Netw 17(4):1052–1065Google Scholar
16.
Chang H, Jamin S, Wang W (2009) Live streaming performance of the Zattoo network. In: Proceedings of the 9th ACM SIGCOMM conference on internet measurement conference. ACMGoogle Scholar
17.
Bonald T, Massoulié L, Mathieu F, Perino D, Twigg A (2008) Epidemic live streaming: optimal performance trade-offs. In: ACM SIGMETRICS international conference on Measurement and modeling of computer systems. Annapolis, USAGoogle Scholar
18.
Bindal R, Chan W, Medved J, Suwala G, Bates T, Zhang A (2006) Improving traffic locality in bittorrent via biased neighbor selection. In: International Conference on Distributed Computing Systems, ICDCS. Lisboa, PORGoogle Scholar
19.
Silva APCD, Leonardi E, Mellia M, Meo M (2008) A bandwidth-aware scheduling strategy for P2P-TV systems. In: Peer-to-Peer computing, P2P’08. IEEE, Aachen, GermanyGoogle Scholar
20.
Ren D, Li Y, Chan S (2008) On reducing mesh delay for Peer-to-Peer live streaming. In: Infocom 2008. IEEE, PhoenixGoogle Scholar
21.
Zhao B, Lui J, Chiu D (2009) Exploring the optimal chunk selection policy for data-driven P2P streaming systems. In: Peer-to-Peer computing, P2P’09. IEEE, SeattleGoogle Scholar
22.
Magharei N, Rejaie R (2007) Mesh or multiple-tree: a comparative study of p2p live streaming services. In: Infocom 2007. IEEE, AnchorageGoogle Scholar
23.
Seibert J, Zage D, Fahmy S, Nita-Rotaru C (2008) Experimental comparison of Peer-to-Peer streaming overlays: an application perspective. In: IEEE conference on Local Computer Networks, LCN’08. Montreal, CanadaGoogle Scholar
24.
Rossi D, Veglia P (2011) Assessing the impact of signaling on the qoe of push-based p2p-tv diffusion algorithms. In: Conference on new technologies, mobility and security, NTMS’11. IEEE, ParisGoogle Scholar
25.
Pai V, Kumar K, Andy Vinay Sambamurthy KT, Mohr AE (2005) Chainsaw: eliminating trees from overlay multicast. Lecture Notes in Computer ScienceGoogle Scholar
26.
Ciullo D, Garcia MA, Akos H, Leonardi E, Mellia M, Rossi D, Telek M, Veglia P (2010) Network awareness of p2p live streaming applications: a measurement study. IEEE Trans Multimedia 12(1):54–63Google Scholar
27.
Picconi F, Massoulié L (2009) Isp friend or foe? Making p2p live streaming isp-aware. In: Proc. of IEEE International Conference on Distributed Computing Systems (ICDCS’09). IEEE, Montreal, QuebecGoogle Scholar
28.
Blond SL, Legout A, Dabbous W (2011) Pushing bittorrent locality to the limit. ComNet 55(3):541–557Google Scholar
29.
Rao A, Legout A, Dabbous W (2010) Can realistic BitTorrent experiments be performed on clusters? In: IEEE international conference on Peer-to-Peer computing (P2P’10). Delft, NetherlandsGoogle Scholar
30.
Piatek M, Isdal T, Anderson T, Krishnamurthy A, Venkataramani A (2007) Do incentives build robustness in BitTorrent? In: Symposium on Networked System Design & Implementation, NSDI’07. USENIX, CambridgeGoogle Scholar
31.
Piatek M, Madhyastha HV, John JP, Krishnamurthy A, Anderson T (2009) Pitfalls for isp-friendly p2p design. In: ACM Workshop on Hot Topics in Networks (HotNets-VIII). New York City, USAGoogle Scholar
32.
Gummadi K, Saroiu S, Gribble S (2002) King: estimating latency between arbitrary internet end hosts. In: Proceedings of the 2nd ACM SIGCOMM Workshop on Internet Measurment. Marseille, FranceGoogle Scholar
33.
Modelnet-te homepage (2010). http://www.enst.fr/~drossi/index.php?n=Software.ModelNet-TE. Accessed Feb 2011
34.
Barabási A., Albert R (1999) Emergence of scaling in random networks. Science 286(5439):509MathSciNetCrossRefGoogle Scholar
35.
Watts D, Strogatz S (1998) Collective dynamics of “small-world” networks. Nature 393(6684):440–442Google Scholar
36.
Alto ietf working group. http://datatracker.ietf.org/wg/alto/charter/. Accessed Feb 2011
37.
Meridian project. http://www.cs.cornell.edu/People/egs/meridian/. Accessed Jun 2010
38.
Lobb RJ, Couto da Silva AP, Leonardi E, Mellia M, Meo M (2009) Adaptive overlay topology for mesh-based P2P-TV systems. In: International workshop on Network and Operating Systems Support for Digital Audio and Video, NOSSDAV’09. ACM, WilliamsburgGoogle Scholar
39.
Shriram A, Murray M, Hyun Y, Brownlee N, Broido A, Fomenkov M, Claffy K (2005) Comparison of public end-to-end bandwidth estimation tools on high-speed links. Passive and Active Measurement conference, PAM’05Google Scholar
40.
Croce D, Mellia M, Leonardi E (2009) The quest for bandwidth estimation techniques for large-scale distributed systems. ACM SIGMETRICS PER 37(3):20–25CrossRefGoogle Scholar
41.
Klaue J, Rathke B, Wolisz A (2003) Evalvid a framework for video transmission and quality evaluation. In: Computer performance, vol. 2794 of lecture notes in computer science. Springer Berlin, HeidelbergGoogle Scholar
42.
Leonardi E, Mellia M, Horvath A, Muscariello L, Niccolini S, Rossi D (2008) Building a cooperative p2p-tv application over a wise network: the approach of the european fp-7 strep napa-wine. IEEE Commun Mag 64:20–22Google Scholar
43.
Finamore A, Mellia M, Meo M, Rossi D (2010) Kiss: stochastic packet inspection classifier for udp traffic. ToN 18:1505–1515Google Scholar
44.
Valenti S, Amd DR, Meo M, Mellia M, Bermolen P (2011) Abacus: accurate behavioral classification of p2p-tv traffic. ComNet 55(6):1394–1411Google Scholar
45.
Finamore A, Mellia M, Meo M, Rossi D, Valenti S (2010) Kiss to Abacus: a comparison of P2P-TV traffic classifiers. In: Traffic Measurement and Analyis (TMA’10), LNCS. Zurich, SwitzerlandGoogle Scholar
46.
Finamore A, Mellia M, Meo M, Rossi D, Valenti S (2010) Peer-to-Peer traffic classification: exploiting human communication dynamics. In: Globecom’10 Demo session. IEEE, MiamiGoogle Scholar
47.
Internet traffic classification demo webpage. http://perso.telecom-paristech.fr/ drossi/index.php?n=Software.ClassificationDemo. Accessed Sep 2010
48.
Pplive. http://www.pptv.com/en/. Accessed Jun 2010
49.
Sopcast. http://www.sopcast.org/
50.
Tvants. http://tvants.en.softonic.com/
51.
Mathieu F (2010) Heterogeneity in data-driven live streaming: blessing or curse? In: IEEE International Symposium on Parallel Distributed Processing (IPDPS), pp 1–8Google Scholar
52.
Bandwidth-test.net. Bandwidth test statistics across different countries. http://www.bandwidth-test.net/stats/country/. Accessed Feb 2011

Copyright information

Authors and Affiliations

Paolo Veglia
- 1
Email author
Dario Rossi
- 1

1.Telecom ParisTechParisFrance

Performance evaluation of P2P-TV diffusion algorithms under realistic settings

Abstract

Keywords

Notes

Acknowledgements

Supplementary material

References

Copyright information

Authors and Affiliations

Personalised recommendations

Cite article