Computational and Mathematical Organization Theory

, Volume 18, Issue 3, pp 280–299 | Cite as

Rapid modeling and analyzing networks extracted from pre-structured news articles

  • Jürgen Pfeffer
  • Kathleen M. Carley
SI: Data to Model


In the face of uprisings and revolutions happening in several countries within short period of time (Arab Spring 2011), the need for fast network assessments is compelling. In this article we present a rapid network assessment approach which uses a vast amount of pre-indexed news data to provide up-to-date overview and orientation in emerging and ongoing incidents. We describe the fully automated process of preparing the data and creating the dynamic meta-networks. We also describe the network analytical measures that we are using to identify important topics, persons, organizations, and locations in these networks. With our rapid network modeling and analysis approach first results can be provided within hours. In the explorative study of this article we use 108,000+ articles from 600+ English written news sources discussing Egypt, Libya, and Sudan within a time period of 18 months to show an application scenario of our approach. In particular we are looking at the involvement of other countries and their politicians during time periods of major incidents.


Rapid network analysis Rapid assessment Network text analysis Dynamic networks Two mode networks Weighted networks 



This work is supported in part by the Office of Naval Research (ONR), United States Navy (ONR MURI N000140811186, ONR MMV N00014060104). The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Office of Naval Research or the U.S. government. The authors wish to acknowledge Jeff Reminga who has been instrumental in developing much of the related technology and Bradley Schmerl for his endeavor to include the data-to-network process of this article into SORASCS.


  1. Apte C, Damerau F, Weiss S (1994) Automated learning of decision rules for text categorization. ACM Trans Inf Syst 12(3):233–251 CrossRefGoogle Scholar
  2. Batagelj V, Mrvar A, Zaversnik M (2002) Network analysis of texts. In: Proceedings B of the 5th international multi-conference information science, pp 143–148 Google Scholar
  3. Bertrand A, Cellier JM (1995) Psychological approach to indexing: effects of the operator’s expertise upon indexing behaviour. J Inf Sci 21(6):459–472 CrossRefGoogle Scholar
  4. Bonacich P (1972) Factoring and weighting approaches to status scores and clique identification. J Math Sociol 2:113–120 CrossRefGoogle Scholar
  5. Borgatti SP, Everett MG (1997) Network analysis of 2-mode data. Soc Netw 19(3):243–269 CrossRefGoogle Scholar
  6. Brandes U (2001) A faster algorithm for betweenness centrality. J Math Sociol 25(2):163–177 CrossRefGoogle Scholar
  7. Carley KM (1997) Network text analysis: the network position of concepts. In: Text analysis for the social sciences: methods for drawing statistical inferences from texts and transcripts. Roberts CW (ed) Lawrence Erlbaum Associates, Mahwah Google Scholar
  8. Carley KM (2002a) Summary of key network measures for characterizing organizational architectures. Unpublished document, CMU 2002, Carnegie Mellon University, SCS/ISR Google Scholar
  9. Carley KM (2002b) Smart agents and organizations of the future. In: Lievrouw LA, Livingstone S (eds) The handbook of new media. Sage, Thousand Oaks Google Scholar
  10. Carley KM, Columbus D (2011) Basic lessons in ORA and AutoMap 2011. Technical report, CMU-ISR-11-109, Carnegie Mellon University, SCS/ISR Google Scholar
  11. Carley KM, Bigrigg M, Papageorgiou C, Johnson J, Kunkel F, Lanham M, Martin M, Morgan G, Schmerl B, van Holt T (2011a) Rapid ethnographic assessment: data-to-model. In: Proceedings of HSCB focus 2011: integrating social science theory and analytic methods for operational use Google Scholar
  12. Carley KM, Reminga J, Storrick J, Columbus D (2011b) ORA user’s guide 2011. Technical report, CMU-ISR-11-107, Carnegie Mellon University, SCS/ISR Google Scholar
  13. Chickering D, Heckerman D, Meek C (1997) A Bayesian approach for learning Bayesian networks with local structure. In: Proceedings of thirteenth conference on uncertainty in artificial intelligence Google Scholar
  14. Davis A, Gardner B, Gardner M (1941) Deep south: a social anthropological study of caste and class. University of Chicago Press, Chicago Google Scholar
  15. Diesner J, Carley KM (2004) Using network text analysis to detect the organizational structure of covert networks. In: Proceedings of the North American association for computational social and organizational science NAACSOS Google Scholar
  16. Diesner J, Carley KM (2005) Revealing social structure from texts: meta-matrix text analysis as a novel method for network text analysis. In: Narayanan VK, Armstrong DJ (eds) Causal mapping for information systems and technology research. Idea Group Publishing, Harrisburg Google Scholar
  17. Diesner J, Carley KM (2008) Conditional random fields for entity extraction and ontological text coding. J Comput Math Organ Theory 14:248–262 CrossRefGoogle Scholar
  18. Diesner J, Carley KM (2010) Mapping socio-cultural networks of Sudan from open-source, large-scale text data. In: Proceedings of the 29th annual conference of the Sudan studies association Google Scholar
  19. Dumais S, Platt J, Heckerman D, Sahami M (1998) Inductive learning algorithms and representations for text categorization. In: Proceedings of the 7th international conference on information and knowledge management Google Scholar
  20. Faust K (1997) Centrality in affiliation networks. Soc Netw 19(2):157–191 CrossRefGoogle Scholar
  21. Freeman LC (1977) A set of measures of centrality based on betweenness. Sociometry 40:35–41 CrossRefGoogle Scholar
  22. Freeman LC (1979) Centrality in social networks: conceptual clarification. Soc Netw 1:215–239 CrossRefGoogle Scholar
  23. Garlan D, Carley KM, Schmerl B, Bigrigg M, Celiku O (2009) Using service-oriented architectures for socio-cultural analysis. In: Proceedings of the 21st international conference on software engineering and knowledge engineering Google Scholar
  24. Harris KJ, Jerome NW, Fawcett SB (1997) Rapid assessment procedures: a review and critique. Human Organ 56(3):375–378 Google Scholar
  25. Holsti OR (1969) Content analysis for the social sciences and humanities. Addison-Wesley, Reading Google Scholar
  26. Hotho A, Jäschke R, Schmitz C, Stumme G (2006) BibSonomy: a social bookmark and publication sharing system. In: Proceedings of the conceptual structures tool interoperability workshop at the 14th international conference on conceptual structures, pp 87–102 Google Scholar
  27. Howe J (2008) Crowdsourcing: why the power of the crowd is driving the future of business. Three Rivers, New York Google Scholar
  28. Mayfield J, McNamee P, Costello C, Piatko C, Banerjee A (2002) JHU/APL at TREC 2001: experiments in filtering and in Arabic, video, and web retrieval. In: Proceedings of the tenth text retrieval conference, NIST special publication Google Scholar
  29. Tsvetovat M, Reminga J, Carley KM (2004) DyNetML: interchange format for rich social network data. Technical report, CMU-ISRI-04-105, Carnegie Mellon University, SCS/ISR Google Scholar
  30. Wasserman S, Faust K (1995) Social network analysis, methods and applications. Cambridge University Press, Cambridge Google Scholar
  31. Wei W, Pfeffer J, Reminga J, Carley KM (2011) Handling weighted, asymmetric, self-looped, and disconnected networks in ORA. Technical report, CMU-ISR-11-113, Carnegie Mellon University, SCS/ISR Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  1. 1.CASOS, ISR, SCSCarnegie Mellon UniversityPittsburghUSA

Personalised recommendations