Tweets Analysis with Big Data Technology and Machine Learning to Evaluate Smart and Sustainable Urban Mobility Actions in Barcelona

Abstract

Social media can become a valuable data source to gather user opinions on topics of interest. Setting the focus on Twitter as one of the most popular social media channels to share opinions, three challenges have been identified in this work: to obtain users’ sentiment: to classify the topics of interest, to decide whether the opinion is positive or negative by applying sentiment analysis to natural language in written form, and to handle in an efficient manner the huge volume of data generated by Twitter. This paper shows how machine learning and big data technologies have been applied to classify user opinions on electromobility in Barcelona. Supervised and unsupervised approaches have been compared in terms of accuracy and a big data framework based on Spark has been implemented for real time processing combined with batch modelling. The results obtained show potential to apply them as a complementary mechanism to surveys.