ParSoDA: high-level parallel programming for social data mining

  • Loris Belcastro
  • Fabrizio MarozzoEmail author
  • Domenico Talia
  • Paolo Trunfio
Original Article


Software systems for social data mining provide algorithms and tools for extracting useful knowledge from user-generated social media data. ParSoDA (Parallel Social Data Analytics) is a high-level library for developing parallel data mining applications based on the extraction of useful knowledge from large data set gathered from social media. The library aims at reducing the programming skills needed for implementing scalable social data analysis applications. To reach this goal, ParSoDA defines a general structure for a social data analysis application that includes a number of configurable steps and provides a predefined (but extensible) set of functions that can be used for each step. User applications based on the ParSoDA library can be run on both Apache Hadoop and Spark clusters. The paper describes the ParSoDA library and presents two social data analysis applications to assess its usability and scalability. Concerning usability, we compare the programming effort required for coding a social media application using versus not using the ParSoDA library. The comparison shows that ParSoDA leads to a drastic reduction (i.e., about 65%) of lines of code, since the programmer only has to implement the application logic without worrying about configuring the environment and related classes. About scalability, using a cluster with 300 cores and 1.2 TB of RAM, ParSoDA is able to reduce the execution time of such applications up to 85%, compared to a cluster with 25 cores and 100 GB of RAM.


Social data analysis Scalability Parallel library Big data Social media Social networks 



This work has been partially supported by the SMART Project, CUP J28C17000150006, funded by Regione Calabria (POR FESR-FSE 2014-2020), and by the ASPIDE Project funded by the European Unions Horizon 2020 research and innovation program under grant agreement No. 801091.


  1. Amer-Yahia S, Ibrahim N, Kengne CK, Ulliana F, Rousset MC (2014) SOCLE: towards a framework for data preparation in social applications. Ingénierie des Systèmes d’Information 19(3):49–72CrossRefGoogle Scholar
  2. Belcastro L, Marozzo F, Talia D, Trunfio P (2017a) Appraising spark on large-scale social media analysis. In: Euro-Par workshops. Lecture notes in computer science. Santiago de Compostela, Spain, pp 483–495. ISBN:978-3-319-75178-8Google Scholar
  3. Belcastro L, Marozzo F, Talia D, Trunfio P (2017b) Big data analysis on clouds. In: Sakr S, Zomaya A (eds) Handbook of big data technologies. Springer, Berlin, pp 101–142. ISBN:978-3-319-49339-8Google Scholar
  4. Belcastro L, Marozzo F, Talia D, Trunfio P (2017c) A parallel library for social media analytics. In: The 2017 international conference on high performance computing & simulation (HPCS 2017), Genoa, ItalyGoogle Scholar
  5. Casalino G, Castiello C, Del Buono N, Mencar C (2018) A framework for intelligent twitter data analysis with nonnegative matrix factorization. Int J Web Inf Syst 14(3):334–356CrossRefGoogle Scholar
  6. Cesario E, Iannazzo A R, Marozzo F, Morello F, Riotta G, Spada A, Talia D, Trunfio P (2016) Analyzing social media data to discover mobility patterns at EXPO 2015: methodology and results. In: The 2016 international conference on high performance computing and simulation (HPCS 2016), Innsbruck, AustriaGoogle Scholar
  7. Chodorow K (2013) MongoDB: the definitive guide. O’Reilly Media, Inc., NewtonGoogle Scholar
  8. Chu C, Kim SK, Lin YA, Yu Y, Bradski G, Ng AY, Olukotun K (2007) Map-reduce for machine learning on multicore. Adv Neural Inf Process. Syst. 19:281Google Scholar
  9. Cuesta Á, Barrero DF, R-Moreno MD (2014) A framework for massive Twitter data extraction and analysis. Malays J Comput Sci 27:50–67Google Scholar
  10. Dean J, Ghemawat S (2004) MapReduce: simplified data processing on large clusters. In: Proceedings of the 6th conference on symposium on operating systems design & implementation, OSDI’04, Berkeley, USA, p 10Google Scholar
  11. ECMA (2009) ECMA-262: ECMAscript language specification, 5th edn. ECMA (European Association for Standardizing Information and Communication Systems), GenevaGoogle Scholar
  12. Han J, Pei J, Yin Y, Mao R (2004) Mining frequent patterns without candidate generation: a frequent-pattern tree approach. Data Min Knowl Discov 8(1):53–87MathSciNetCrossRefGoogle Scholar
  13. Hussain A, Vatrapu R (2014) Social data analytics tool (SODATO). Springer International Publishing, Cham, pp 368–372Google Scholar
  14. Li H, Wang Y, Zhang D, Zhang M, Chang EY (2008) PFP: parallel FP-growth for query recommendation. In: Proceedings of the 2008 ACM conference on recommender systems, New York, NY, USA, pp 107–114Google Scholar
  15. Marozzo F, Bessi A (2018) Analyzing polarization of social media users and news sites during political campaigns. Soc Netw Anal Min 8(1):1CrossRefGoogle Scholar
  16. Pang B, Lee L (2008) Opinion mining and sentiment analysis. Found Trends Inf Retr 2(12):1–135CrossRefGoogle Scholar
  17. Pei J, Han J, Mortazavi-Asl B, Wang J, Pinto H, Chen Q, Dayal U, Hsu MC (2004) Mining sequential patterns by pattern-growth: the prefixSpan approach. IEEE Trans Knowl Data Eng 16(11):1424–1440CrossRefGoogle Scholar
  18. Talia D, Trunfio P, Marozzo F (2015) Data analysis in the cloud. Elsevier, AmsterdamGoogle Scholar
  19. White T (2012) Hadoop: the definitive guide. O’Reilly Media, Inc., NewtonGoogle Scholar
  20. You L, Motta G, Sacco D, Ma T (2014) Social data analysis framework in cloud and mobility analyzer for smarter cities. In: Proceedings of 2014 IEEE international conference on service operations and logistics, and informatics, Qingdao, China, pp 96–101Google Scholar
  21. Zaharia M, Xin RS, Wendell P, Das T, Armbrust M, Dave A, Meng X, Rosen J, Venkataraman S, Franklin MJ et al (2016) Apache spark: a unified engine for big data processing. Commun ACM 59(11):56–65CrossRefGoogle Scholar
  22. Zhou D, Chen L, He Y (2015) An unsupervised framework of exploring events on twitter: filtering, extraction and categorization. In: Proceedings of the 29th AAAI conference on artificial intelligence, Austin, Texas, USA, pp 2468–2475Google Scholar

Copyright information

© Springer-Verlag GmbH Austria, part of Springer Nature 2018

Authors and Affiliations

  1. 1.DIMES DepartmentUniversity of CalabriaRendeItaly
  2. 2.DtoK Lab SrlRendeItaly

Personalised recommendations