Abstract
To understand network behavior, researchers and enterprise network operators must interpret large amounts of network data. To understand and manage network events such as outages, route instability, and spam campaigns, they must interpret data that covers a range of networks and evolves over time. We propose a simple clustering algorithm that helps identify spatial clusters of network events based on correlations in event timing, producing 2-D visualizations. We show that these visualizations where they reveal the extent, timing, and dynamics of network outages such as January 2011 Egyptian change of government, and the March 2011 Japanese earthquake. We also show they reveal correlations in routing changes that are hidden from AS-path analysis.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Armitage G (2011) Private communications
AusNOG (2011) Discussions about Australia flooding. http://www.lists.ausnog.net/pipermail/ausnog/2011-January
CAIDA (2012) As dispersion graphs. http://www.caida.org/projects/ark/statistics/san-us/as_dispersion_by_as.html
Chang Di-Fa, Govindan R, Heidemann J (2003) The temporal and topological characteristics of BGP path changes. In: Proceedings of ICNP, Nov 2003
Cho K, Pelsser C, Bush R, Won Y (2011) The Japan earthquake: the impact on traffic and routing observed by a local ISP. In: Proceedings of ACM Special Workshop on Internet and Disasters. Tokyo, Japan, pp 2:1–2:8, ACM, Dec 2011
Dainotti A, Squarcella C, Aben E, Claffy KC, Chiesa M, Russo M, Pescap Aé (2011) Analysis of country-wide Internet outages caused by censorship. In: ACM IMC
Gao H, Hu J, Wilson C, Li Z, Chen Y, Zhao BY (2010) Detecting and characterizing social spam campaigns. In: ACM IMC, pp 35–47
Gill P, Jain N, Nagappan N (2011) Understanding network failures in data centers: measurement, analysis, and implications. In: Proceedings of the ACM SIGCOMM 2011 conference, SIGCOMM, pp 350–361
Heidemann J, Pradkin Y, Govindan R, Papadopoulos C, Bartlett G, Bannister J (2008) Census and survey of the visible internet. In: Proceedings of ACM IMC
International Business Times (2011) Optus, Telstra see service outages after Cyclone Yasi. http://www.hken.ibtimes.com/articles/108249/20110203/optus-telstra-see-service-outages-after-cyclone-yasi.htm
Katz-Bassett E, Madhyastha HV, John JP, Krishnamurthy A, Wetherall D, Anderson T (2008) Studying black holes in the Internet with Hubble. In: USENIX Symposium on Networked Systems Design & Implementation (NSDI)
Labovitz C, Ahuja A, Bose A, Jahanian F (2000) Delayed internet routing convergence. In: Proceedings of SIGCOMM 2000, Stockholm, Sweden
Labovitz C, Robert Malan G, Jahanian F (1997) Internet routing instability. In: Proceedings of the SIGCOMM
Liao TW (2005) Clustering of time series data–a survey. Pattern Recognit 38(11):1857–1874
Liu X, Yang X, Lu Y (2008) To filter or to authorize: network-layer DoS defense against multimillion-node botnets. In: Proceedings of the ACM SIGCOMM 2008 conference on Data communication, pp 195–206
Madhyastha HV, Isdal T, Piatek M, Dixon C, Anderson T, Krishnamurthy A, Venkataramani A (2006) iPlane: an information plane for distributed services. In: In: Symposium on Operating Systems Design and Implementation (OSDI 2006)
Mahajan R, Wetherall D, Anderson T (2002) Understanding BGP misconfiguration. In: Proceedings of the SIGCOMM
Malik O (2011) In Japan, many undersea cables are damaged. GigaOM blog. http://www.gigaom.com/broadband/in-japan-many-under-sea-cables-are-damaged/
Markopoulou A, Iannaccone G, Bhattacharyya S, Chuah C-N, Diot C (2004) Characterization of Failures in an IP Backbone. In: Proceedings of IEEE Infocom
New York Times (2011) Egypt cuts off most Internet and cell service. http://www.nytimes.com/2011/01/29/technology/internet/29cutoff.html
Open Layers (2012) Openlayers: free maps for the web. http://www.openlayers.org
Quan L, Heidemann J, Pradkin Y (2012) Detecting internet outages with precise active probing (extended). Technical, Report ISI-TR-2012-678
Quan L, Heidemann J, Pradkin Y (2012) Supplemental data and visualizations. http://www.isi.edu/ant/pubs/network-vis/
Schulman A, Spring N (2011) Pingin’ in the rain. In: Proceedings of ACM IMC, pp 19–28
Turner D, Levchenko K, Snoeren AC, Savage S (2010) California fault lines: understanding the causes and impact of network failures. In: Proceedings of the SIGCOMM
Xu R, Wunsch D II (2005) Survey of clustering algorithms. IEEE Trans Neural Netw 16(3):645–678
Yan H, Oliveira R, Burnett K, Matthews D, Zhang L, Massey D (2009) BGPmon: a real-time, scalable, extensible monitoring system. In: Proceedings of IEEE CATCH
Acknowledgments
We thank Jim Koda (ISI), Brian Yamaguchi (USC), and CSU network operations for providing BGP feeds to assist our evaluation, and Dan Massey, Christos Papadopoulos, Mikhail Strizhov for assisting with BGPmon and at CSU. We also thank Katsuhiro Horiba (WIDE) for providing probing infrastructure and BGP feeds. This work was reviewed by USC’s IRB (IIR00000975) and identified as non-human subjects research.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Quan, L., Heidemann, J. & Pradkin, Y. Visualizing sparse internet events: network outages and route changes. Computing 96, 39–51 (2014). https://doi.org/10.1007/s00607-013-0283-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00607-013-0283-7