Evaluation and Development of Data Mining Tools for Social Network Analysis

  • Dhiraj Murthy
  • Alexander Gross
  • Alexander Takata
  • Stephanie Bond
Part of the Lecture Notes in Social Networks book series (LNSN)


This chapter reviews existing data mining tools for scraping data from heterogeneous online social networks. It introduces not only the complexities of scraping data from these sources (which include diverse data forms), but also presents currently available tools including their strengths and weaknesses. The chapter introduces our solution to effectively mining online social networks through the development of VoyeurServer, a tool we designed which builds upon the open-source Web-Harvest framework. We have shared details of how VoyeurServer was developed and how it works so that data mining developers can develop their own customized data mining solutions built upon the Web-Harvest framework. We conclude the chapter with future directions of our data mining project so that developers can incorporate relevant features into their data mining applications.


Data mining Online social networks Web content extraction Web-Harvest Network analysis 


  1. 1.
    Bollen J (2011) Twitter mood as a stock market predictor. Computer 44(10):91–94 CrossRefGoogle Scholar
  2. 2.
    Boyd DM, Ellison NB (2008) Social network sites: definition, history, and scholarship. J Comput-Mediat Commun 13(1):210–230 CrossRefGoogle Scholar
  3. 3.
    Doan S, Vo B-KH, Collier N (2011) An analysis of twitter messages in the 2011 Tohoku earthquake. arXiv:1109.1618v1 [cs.SI]
  4. 4.
    Graczyk M et al. (2009) Comparative analysis of premises valuation models using KEEL, RapidMiner, and WEKA computational collective intelligence. In: Semantic web, social networks and multiagent systems. Springer, Berlin, pp 800–812 Google Scholar
  5. 5.
    Golder SA, Macy MW (2011) Diurnal and seasonal mood vary with work, sleep, and daylength across diverse cultures. Science 333(6051):1878–1881 ADSCrossRefGoogle Scholar
  6. 6.
    Han J, Rodriguez JC, Beheshti M (2008) Diabetes data analysis and prediction model discovery using RapidMiner. In: Proceedings of the 2008 second international conference on future generation communication and networking, vol 03. IEEE Comput Soc, Los Alamitos Google Scholar
  7. 7.
    Häsel M (2011) OpenSocial: an enabler for social applications on the web. Commun ACM 54(1):139–144 CrossRefGoogle Scholar
  8. 8.
    Hughes AL, Palen L, Sutton J, Liu SB, Vieweg S (2008) “Site-seeing” in disaster: an examination of on-line social convergence. In: 5th international ISCRAM conference Google Scholar
  9. 9.
    Katzdobler F-J, Filho HPB (1999) Knowledge extraction from web. In: Book knowledge extraction from Web. Cited 15 Jan 1999
  10. 10.
    Kokkoras F et al. (2008) MOpiS: a multiple opinion summarizer artificial intelligence: theories, models and applications. Springer, Berlin, pp 110–122 CrossRefGoogle Scholar
  11. 11.
    Morzy M (2011) Internet forums: what knowledge can be mined from online discussions. In: Knowledge discovery practices and emerging applications of data mining: trends and new domains. IGI Global, Hershey, pp 315–336 Google Scholar
  12. 12.
    Nagel T, Duval E (2010) Muse: visualizing the origins and connections of institutions based on co-authorship of publications. In: Proceeding of 2nd international workshop on research 20 at the 5th European conference on technology enhanced learning sustaining TEL, pp 48–52 Google Scholar
  13. 13.
    Sheng VS, Provost F, Ipeirotis PG (2008) Get another label? improving data quality and data mining using multiple, noisy labelers. In: Proceedings of the 14th ACM SIGKDD international conference on knowledge discovery and data mining. ACM, Las Vegas Google Scholar
  14. 14.
    Snow R et al. (2008) Cheap and fast—but is it good?: evaluating non-expert annotations for natural language tasks. In: Proceedings of the conference on empirical methods in natural language processing. Association for Computational Linguistics, Honolulu Google Scholar
  15. 15.
    van Wel L, Rotakkers L (2004) Ethical issues in web data mining. Ethics Inf Technol 6(2):129–140 CrossRefGoogle Scholar
  16. 16.
    Yang L et al. (2009) Discovering topics from dark websites. In: Computational intelligence in cyber security. CICS ’09. IEEE symposium. IEEE: Univ. of Tennessee at Chattanooga, Chattanooga, pp 175–179 CrossRefGoogle Scholar
  17. 17.
    Yin RM (2007) TagCrawler: a web crawler focused on data extraction from collaborative tagging communities. University of British Columbia, Vancouver Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2013

Authors and Affiliations

  • Dhiraj Murthy
    • 1
  • Alexander Gross
    • 1
  • Alexander Takata
    • 1
  • Stephanie Bond
    • 1
  1. 1.Social Network Innovation LabBowdoin CollegeBrunswickUSA

Personalised recommendations