Abstract
Passive probes continuously generate statistics on large number of metrics, that are possibly represented as probability mass functions (pmf). The need for consolidation of several pmfs arises in two contexts, namely: (i) whenever a central point collects and aggregates measurement of multiple disjoint vantage points, and (ii) whenever a local measurement processed at a single vantage point needs to be distributed over multiple cores of the same physical probe, in order to cope with growing link capacity.
In this work, we take an experimental approach and study both cases using, whenever possible, open source software and datasets. Considering different consolidation strategies, we assess their accuracy in estimating pmf deciles (from the 10th to the 90th) of diverse metrics, obtaining general design and tuning guidelines. In our dataset, we find that Monotonic Spline Interpolation over a larger set of percentiles (e.g., adding 5th, 10th, 15th, and so on) allow fairly accurate pmf consolidation in both the multiple vantage points (median error is about 1%, maximum 30%) and local processes (median 0.1%, maximum 1%) cases.
Chapter PDF
Similar content being viewed by others
Keywords
- Cumulative Distribution Function
- Vantage Point
- Consolidation Process
- Probability Mass Function
- Passive Probe
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Auckland traces, http://www.wand.net.nz/
Unibs traces, http://www.ing.unibs.it/ntw/tools/traces/
Bonelli, N., Pietro, A.D., Giordano, S., Procissi, G.: Pfq: a novel engine for multi-gigabit packet capturing with multi-core commodity hardware. In: Taft, N., Ricciato, F. (eds.) PAM 2012. LNCS, vol. 7192, pp. 64–73. Springer, Heidelberg (2012)
Chirichella, C., Rossi, D.: To the moon and back: are internet bufferbloat delays really that large. In: IEEE INFOCOM Workshop on Traffic Measurement and Analysis, TMA (2013)
del Rio, P.S., Rossi, D., Gringoli, F., Nava, L.S.L., Aracil, J.: Wire-speed statistical classification of network traffic on commodity hardware. In: ACM SIGCOMM Internet Measurement Conference, IMC (2012)
Deri, L., Cardigliano, A., Fusco, F.: 10 gbit line rate packet-to-disk using n2disk. In: IEEE INFOCOM Workshop on Traffic Monitoring and Analysis, TMA (2013)
Finamore, A., Mellia, M., Meo, M., Munafo, M., Rossi, D.: Experiences of internet traffic monitoring with tstat. IEEE Network (2011)
Fritsch, F.N., Carlson, R.E.: Monotone piecewise cubic interpolation. SIAM Journal on Numerical Analysis 17(2), 238–246 (1980)
Han, S., Jang, K., Park, K., Moon, S.: Packetshader: a gpu-accelerated software router. In: ACM SIGCOMM (2010)
Haridasan, M., van Renesse, R.: Gossip-based distribution estimation in peer-to-peer networks. In: USENIX Internet Peer-to-Peer Symposium, IPTPS (2008)
Huici, F., Di Pietro, A., Trammell, B., Gomez Hidalgo, J.M., Martinez Ruiz, D., d’Heureuse, N.: Blockmon: A high-performance composable network traffic measurement system. ACM SIGCOMM Computer Communication Review 42(4), 79–80 (2012)
Jain, R., Chlamtac, I.: The p2 algorithm for dynamic calculation of quantiles and histograms without storing observations. Communications of the ACM 28(10), 1076–1085 (1985)
Kilpi, J., Varjonen, S.: Minimizing information loss in sequential estimation of several quantiles. Technical Report (2009)
Lyra, C., Hara, C.S., Duarte Jr., E.P.: BackStreamDB: A distributed system for backbone traffic monitoring providing arbitrary measurements in real-time. In: Taft, N., Ricciato, F. (eds.) PAM 2012. LNCS, vol. 7192, pp. 42–52. Springer, Heidelberg (2012)
Pescape, A., Rossi, D., Tammaro, D., Valenti, S.: On the impact of sampling on traffic monitoring and analysis. In: International Teletraffic Congress, ITC22 (2010)
Rizzo, L.: Revisiting network i/o apis: the netmap framework. Commun. ACM 55(3), 45–51 (2012)
Wang, H., Ciucu, F., Schmitt, J.: A leftover service curve approach to analyze demultiplexing in queueing networks. In: VALUETOOLS (2012)
Wang, L., Luo, G., Yi, K., Cormode, G.: Quantiles over data streams: an experimental study. In: ACM SIGMOD (2013)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 IFIP International Federation for Information Processing
About this paper
Cite this paper
Colabrese, S., Rossi, D., Mellia, M. (2014). Aggregation of Statistical Data from Passive Probes: Techniques and Best Practices. In: Dainotti, A., Mahanti, A., Uhlig, S. (eds) Traffic Monitoring and Analysis. TMA 2014. Lecture Notes in Computer Science, vol 8406. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-54999-1_4
Download citation
DOI: https://doi.org/10.1007/978-3-642-54999-1_4
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-54998-4
Online ISBN: 978-3-642-54999-1
eBook Packages: Computer ScienceComputer Science (R0)