Chapter

Intelligence and Security Informatics

Volume 3975 of the series Lecture Notes in Computer Science pp 201-212

Finding Hidden Group Structure in a Stream of Communications

  • J. BaumesAffiliated withCS Department, RPI
  • , M. GoldbergAffiliated withCS Department, RPI
  • , M. HayvanovychAffiliated withCS Department, RPI
  • , M. Magdon-IsmailAffiliated withCS Department, RPI
  • , W. WallaceAffiliated withDSES Department, RPI
  • , M. ZakiAffiliated withCS Department, RPI

* Final gross prices may vary according to local VAT.

Get Access

Abstract

A hidden group in a communication network is a group of individuals planning an activity over a communication medium without announcing their intentions. We develop algorithms for separating non-random planning-related communications from random background communications in a streaming model. This work extends previous results related to the identification of hidden groups in the cyclic model. The new statistical model and new algorithms do not assume the existence of a planning time-cycle in the stream of communications of a hidden group. The algorithms construct larger hidden groups by building them up from smaller ones. To illustrate our algorithms, we apply them to the Enron email corpus in order to extract the evolution of Enron’s organizational structure.