Computing

, Volume 96, Issue 1, pp 27–38

Visualizing big network traffic data using frequent pattern mining and hypergraphs

  • Eduard Glatz
  • Stelios Mavromatidis
  • Bernhard Ager
  • Xenofontas Dimitropoulos
Article

DOI: 10.1007/s00607-013-0282-8

Cite this article as:
Glatz, E., Mavromatidis, S., Ager, B. et al. Computing (2014) 96: 27. doi:10.1007/s00607-013-0282-8

Abstract

Visualizing communication logs, like NetFlow records, is extremely useful for numerous tasks that need to analyze network traffic traces, like network planning, performance monitoring, and troubleshooting. Communication logs, however, can be massive, which necessitates designing effective visualization techniques for large data sets. To address this problem, we introduce a novel network traffic visualization scheme based on the key ideas of (1) exploiting frequent itemset mining (FIM) to visualize a succinct set of interesting traffic patterns extracted from large traces of communication logs; and (2) visualizing extracted patterns as hypergraphs that clearly display multi-attribute associations. We demonstrate case studies that support the utility of our visualization scheme and show that it enables the visualization of substantially larger data sets than existing network traffic visualization schemes based on parallel-coordinate plots or graphs. For example, we show that our scheme can easily visualize the patterns of more than 41 million NetFlow records. Previous research has explored using parallel-coordinate plots for visualizing network traffic flows. However, such plots do not scale to data sets with thousands of even millions of flows.

Keywords

VisualizationBig dataNetwork trafficFrequent item-set miningNetwork securityNetFlow

Mathematics Subject Classification

00A66

Copyright information

© Springer-Verlag Wien 2013

Authors and Affiliations

  • Eduard Glatz
    • 1
  • Stelios Mavromatidis
    • 2
  • Bernhard Ager
    • 1
  • Xenofontas Dimitropoulos
    • 1
  1. 1.ETH ZurichZurichSwitzerland
  2. 2.Open University of CyprusLatsiaCyprus