Skip to main content

Advertisement

SpringerLink
Log in

Employing BERT-DCNN with sentic knowledge base for social media sentiment analysis

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

Machine learning is considered advantageous smart cities through sentiment analysis using social media reviews. Social media reviews can be helpful inside smart cities for various purposes. Primary convolutional neural networks (CNNs) are hard to implement for parallelizing applications and inadequate to understand the contextual semantics in long-term sequences for emotion classification. In this context, this paper presents a Bidirectional Encoder Representations from Transformers (BERT) based Dilated Convolutional Neural Network (BERT-DCNN) model, which leverages BERT as a pre-trained language model to generate word embeddings. Additionally, three parallel layers of Dilated Convolutional Neural Network (DCNN) stacked with a global average pooling layer helps in fine-tuning the model. Our implemented BERT-DCNN model performs dimensionality reduction and assimilates an increase of related dimensions withstanding any information loss. Furthermore, the model can capture long-term dependencies by utilizing various dilation rates. Moreover, the sentic knowledge base is incorporated in our model, enabling it to achieve concept-level sentiment analysis. Our experimental study demonstrates the importance of the implemented model in terms of F-measure, recall, precision, and accuracy with different machine learning models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  • Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, Corrado G. S., Davis A, Dean J, Devin M, Ghemawat S, Goodfellow I, Harp A, Irving G, Isard M, Jia Y, Jozefowicz R, Kaiser L, Kudlur M, Levenberg J, Mané D, Monga R, Moore S, Murray D, Olah C, Schuster M, Shlens J, Steiner B, Sutskever I, Talwar K, Tucker P, Vanhoucke V, Vasudevan V, Viégas F, Vinyals O, Warden P, Wattenberg M, Wicke M, Yu Y, Zheng X (2015) TensorFlow: Large-scale machine learning on heterogeneous systems. Software available from tensorflow.org

  • Abualigah L, Alfar HE, Shehab M, Hussein AMA (2020) Sentiment analysis in healthcare: a brief review. Recent advances in NLP: the case of Arabic language. Springer, pp 129–141

    Chapter  Google Scholar 

  • Alam M, Abid F, Guangpei C, Yunrong L (2020) Social media sentiment analysis through parallel dilated convolutional neural network for smart city applications. Comput Commun 154:129–137

    Article  Google Scholar 

  • Ambartsoumian A, Popowich F (2018) Self-attention: A better building block for sentiment analysis neural network classifiers. In: Proceedings of the 9th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, Association for Computational Linguistics, Brussels, Belgium, p 130–139

  • Anand PB, Navío-Marco J (2018) Governance and economics of smart cities: opportunities and challenges. Telecommunications Policy 42(10):795–799

    Article  Google Scholar 

  • Anastasi G, Antonelli M, Bechini A, Brienza S, D’Andrea E, De Guglielmo D, Ducange P, Lazzerini B, Marcelloni F, Segatori A (2013) Urban and social sensing for sustainable mobility in smart cities. In: 2013 Sustainable Internet and ICT for Sustainability (SustainIT), p 1–4

  • Ang L-M, Seng KP, Zungeru AM, Ijemaru GK (2017) Big sensor data systems for smart cities. IEEE Internet of Things Journal 4(5):1259–1271

    Article  Google Scholar 

  • Avvenuti M, Cresci S, La Polla M. N., Marchetti A, Tesconi M (2014) Earthquake emergency management by social sensing. In: 2014 IEEE International Conference on Pervasive Computing and Communication Workshops (PERCOM WORKSHOPS), pages 587–592

  • Balaji T, Annavarapu CSR, Bablani A (2021) Machine learning algorithms for social media analysis: a survey. Comput Sci Rev 40:100395

    Article  Google Scholar 

  • Barnes J, Klinger R, Schulte im Walde S (2017) Assessing state-of-the-art sentiment models on state-of-the-art sentiment datasets. In: Proceedings of the 8th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, Association for Computational Linguistics, , Copenhagen, Denmark, p 2–12

  • Bencke L, Cechinel C, Munoz R (2020) Automated classification of social network messages into smart cities dimensions. Fut Gener Comput Syst 109:218–237

    Article  Google Scholar 

  • Bourg L, Chatzidimitris T, Chatzigiannakis I, Gavalas D, Giannakopoulou K, Kasapakis V, Konstantopoulos C, Kypriadis D, Pantziou G, Zaroliagis C (2021) Enhancing shopping experiences in smart retailing. Journal of Ambient Intelligence and Humanized Computing, p 1–19

  • Bravo-Marquez F, Mendoza M, Poblete B (2013) Combining strengths, emotions and polarities for boosting twitter sentiment analysis. In: Proceedings of the Second International Workshop on Issues of Sentiment Discovery and Opinion Mining, p 1–9

  • Cambria E, Speer R, Havasi C, Hussain A (2010) Senticnet: A publicly available semantic resource for opinion mining. In: AAAI fall symposium: commonsense knowledge, 10

  • Cambria E, Livingstone A, Hussain A (2012) The hourglass of emotions. Cognitive behavioural systems. Springer, pp 144–157

    Chapter  Google Scholar 

  • Cambria E, Poria S, Hazarika D, Kwok K (2018) SenticNet 5: Discovering conceptual primitives for sentiment analysis by means of context embeddings. In: Proceedings of the AAAI conference on artificial intelligence, Vol. 32, No. 1

  • Chen Y, Xu L, Liu K, Zeng D, Zhao J (2015) Event extraction via dynamic multi-pooling convolutional neural networks. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), p 167–176

  • Chin J, Callaghan V, Lam I (2017) Understanding and personalising smart city services using machine learning, the internet-of-things and big data. In: 2017 IEEE 26th International Symposium on Industrial Electronics (ISIE), p 2050–2055

  • Collobert R, Weston J (2008) A unified architecture for natural language processing: Deep neural networks with multitask learning. In: Proceedings of the 25th international conference on Machine learning, p 160–167

  • Collovini S, Pereira B, dos Santos HD, Vieira R (2018) Annotating relations between named entities with crowdsourcing. International Conference on Applications of Natural Language to Information Systems. Springer, pp 290–297

    Google Scholar 

  • D’Andrea E, Ducange P, Lazzerini B, Marcelloni F (2015) Real-time detection of traffic from twitter stream analysis. IEEE Trans Intell Trans Syst 16(4):2269–2283

    Article  Google Scholar 

  • Devlin J, Chang M.-W., Lee K, Toutanova K (2019) BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Association for Computational Linguistics, , Minneapolis, Minnesota, p 4171–4186

  • Dizon E, Pranggono B (2021) Smart streetlights in Smart City: a case study of Sheffield. J Ambient Intell Human Comput, pp 1–16

  • Dos Santos C, Gatti M (2014) Deep convolutional neural networks for sentiment analysis of short texts. In: Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, p 69–78

  • Finin T, Murnane W, Karandikar A, Keller N, Martineau J, Dredze M (2010) Annotating named entities in twitter data with crowdsourcing. In: Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon’s Mechanical Turk, p 80–88

  • Flekova L, Ferschke O, Gurevych I (2014) Ukpdipf: A lexical semantic approach to sentiment polarity prediction in Twitter data

  • Ghahramani M, Galle NJ, Ratti C, Pilla F (2021) Tales of a city: sentiment analysis of urban green space in Dublin. Cities 119:103395

    Article  Google Scholar 

  • Gu J, Wang Z, Kuen J, Ma L, Shahroudy A, Shuai B, Liu T, Wang X, Wang G, Cai J et al (2018) Recent advances in convolutional neural networks. Pattern Recogn 77:354–377

    Article  Google Scholar 

  • Heaton J (2008) The number of hidden layers. Heaton Research Inc, pp 157–158

    Google Scholar 

  • Howard J, Ruder S (2018) Universal language model fine-tuning for text classification. arXiv preprint arXiv:1801.06146

  • Huai S, Van de Voorde T (2022) Which environmental features contribute to positive and negative perceptions of urban parks? a cross-cultural comparison using online reviews and natural language processing methods. Lands Urban Plan 218:104307

    Article  Google Scholar 

  • Jain PK, Pamula R (2020) Content-based airline recommendation prediction using machine learning techniques. Machine Learning Algorithms for Industrial Applications. Springer, pp 185–194

    Google Scholar 

  • Jain PK, Pamula R, Srivastava G (2021a) A systematic literature review on machine learning applications for consumer sentiment analysis using online reviews. Comput Sci Rev 41:100413

    Article  Google Scholar 

  • Jain PK, Quamer W, Pamula R, Saravanan V (2021b) SpSAN: Sparse self-attentive network-based aspect-aware model for sentiment analysis. J Ambient Intell Human Comput, pp 1–18

  • Kalchbrenner N, Espeholt L, Simonyan K, Oord A. v. d., Graves A, Kavukcuoglu K (2016) Neural machine translation in linear time. arXiv preprint arXiv:1610.10099

  • Kim Y (2014) Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882

  • Lee K, Palsetia D, Narayanan R, Patwary M. M. A., Agrawal A, Choudhary A (2011) Twitter trending topic classification. In: 2011 IEEE 11th International Conference on Data Mining Workshops, p 251–258

  • Lin X-M, Ho C-H, Xia L-T, Zhao R-Y (2021) Sentiment analysis of low-carbon travel app user comments based on deep learning. Sustain Energy Technol Assess 44:101014

    Google Scholar 

  • Liu B (2020) Sentiment analysis: mining opinions, sentiments, and emotions. Cambridge University Press

    Book  Google Scholar 

  • Liu B et al (2010) Sentiment analysis and subjectivity. Handb Nat Lang Process 2(2010):627–666

    Google Scholar 

  • Lytras MD, Mathkour HI, Abdalla H, Al-Halabi W, Yanez-Marquez C, Siqueira SWM (2015) An emerging–Social and emerging computing enabled philosophical paradigm for collaborative learning systems: toward high effective next-generation learning systems for the knowledge society. Comput Human Behav 51:557–561

    Article  Google Scholar 

  • Lytras M, Aljohani NR, Hussain A, Luo J, Zhang JX (2018) Cognitive computing track chairs’ welcome & organization. In: Companion Proceedings of the The Web Conference 2018:247–250

  • Mainka A, Hartmann S, Stock WG, Peters I (2015) Looking for friends and followers: a global investigation of governmental social media use. Transforming Government: People, Process and Policy

  • Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J (2013) Distributed representations of words and phrases and their compositionality. In: Burges CJC, Bottou L, Welling M, Ghahramani Z, Weinberger KQ (eds) Advances in Neural Information Processing Systems 26. Curran Associates Inc, pp 3111–3119

    Google Scholar 

  • Mora H, Pérez-delHoyo R, Paredes-Pérez JF, Mollá-Sirvent RA (2018) Analysis of social networking service data for smart urban planning. Sustainability 10(12):4732

    Article  Google Scholar 

  • Nguyen T. H., Grishman R (2015) Event detection and domain adaptation with convolutional neural networks. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), p 365–371

  • Oord A. v. d., Dieleman S, Zen H, Simonyan K, Vinyals O, Graves A, Kalchbrenner N, Senior A, Kavukcuoglu K (2016) Wavenet: A generative model for raw audio. arXiv preprint arXiv:1609.03499, page may 5

  • Peng X, Feng J, Xiao S, Yau W-Y, Zhou JT, Yang S (2018) Structured autoencoders for subspace clustering. IEEE Trans Image Process 27(10):5076–5086

    Article  MathSciNet  Google Scholar 

  • Pennington J, Socher R, Manning CD (2014) Glove: Global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1532–1543

  • Pereira J. F. F. (2017) Social media text processing and semantic analysis for smart cities. arXiv preprint arXiv:1709.03406

  • Peters M, Neumann M, Iyyer M, Gardner M, Clark C, Lee K, Zettlemoyer L (2018a) Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), Association for Computational Linguistics, New Orleans, Louisiana, p 2227–2237

  • Peters M. E., Neumann M, Iyyer M, Gardner M, Clark C, Lee K, Zettlemoyer L (2018b) Deep contextualized word representations. arXiv preprint arXiv:1802.05365

  • Poria S, Cambria E, Winterstein G, Huang G-B (2014) Sentic patterns: dependency-based rules for concept-level sentiment analysis. Knowl Based Syst 69:45–63

    Article  Google Scholar 

  • Priyadarshini I, Cotton C (2021) A novel LSTM–CNN–grid search-based deep neural network for sentiment analysis. J Supercomput, pp 1–22

  • Radford A, Narasimhan K, Salimans T, Sutskever I (2018) Improving language understanding by generative pre-training

  • Roberts K, Roach, MA, Johnson J, Guthrie J, Harabagiu SM (2012) EmpaTweet: annotating and detecting emotions on twitter. In: Lrec, vol 12, pp 3806–3813

  • Rose G, Willis A (2019) Seeing the smart city on twitter: colour and the affective territories of becoming smart. Environ Plan D Soc Space 37(3):411–427

    Article  Google Scholar 

  • Seliverstov Y, Seliverstov S, Malygin I, Korolev O (2020) Traffic safety evaluation in northwestern federal district using sentiment analysis of internet users’ reviews. Trans Res Proc 50:626–635

    Google Scholar 

  • Shekar C, Wakade S, Liszka K. J., Chan C.-C. (2010) Mining pharmaceutical spam from twitter. In: 2010 10th International Conference on Intelligent Systems Design and Applications, p 813–817

  • Sheth A (2009) Citizen sensing, social signals, and enriching human experience. IEEE Internet Comput 13(4):87–92

    Article  Google Scholar 

  • Signorini A, Segre AM, Polgreen PM (2011) The use of twitter to track levels of disease activity and public concern in the us during the influenza a h1n1 pandemic. PloS One 6(5):e19467

    Article  Google Scholar 

  • Tai K. S., Socher R, Manning C. D. (2015) Improved semantic representations from tree-structured long short-term memory networks. arXiv preprint arXiv:1503.00075

  • Trindade EP, Hinnig MPF, Moreira da Costa E, Marques JS, Bastos RC, Yigitcanlar T (2017) Sustainable development of smart cities: a systematic review of the literature. J Open Innov Technol Market Complex 3(3):11

    Google Scholar 

  • Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, u., and Polosukhin, I. (2017) Attention is all you need. In: Guyon I, Luxburg UV, Bengio S, Wallach H, Fergus R, Vishwanathan S, Garnett R (eds) Advances in Neural Information Processing Systems 30. Curran Associates Inc, p 5998–6008

  • Visvizi A, Lytras MD, Damiani E, Mathkour H (2018) Policy making for smart cities: innovation and social inclusive economic growth for sustainability. J Sci Technol Policy Manage

  • Xu B, Wang N, Chen T, Li M (2015) Empirical evaluation of rectified activations in convolutional network. arXiv preprint arXiv:1505.00853, page may 5

  • Yadav A, Vishwakarma DK (2020) Sentiment analysis using deep learning architectures: a review. Artif Intell Rev 53(6):4335–4385

    Article  Google Scholar 

  • Yin C, Xiong Z, Chen H, Wang J, Cooper D, David B (2015) A literature survey on smart cities. Sci China Inform Sci 58(10):1–18

    Article  Google Scholar 

  • Yin W, Schütze H (2016) Multichannel variable-size convolution for sentence classification. arXiv preprint arXiv:1603.04513

  • Yu F, Koltun V (2015) Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122

  • Zeng D, Liu K, Lai S, Zhou G, Zhao J (2014) Relation classification via convolutional deep neural network. In: Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, p 2335–2344

  • Zubiaga A, Spina D, Fresno V, Martínez R (2011) Classifying trending topics: a typology of conversation triggers on twitter. In: Proceedings of the 20th ACM international conference on Information and knowledge management, p 2461–2464

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Indian Institute of Technology (Indian School of Mines), Dhanbad, 826004, JH, India

    Praphula Kumar Jain, Waris Quamer & Rajendra Pamula

  2. University of South Dakota, Vermillion, USA

    Vijayalakshmi Saravanan

Authors
  1. Praphula Kumar Jain
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Waris Quamer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vijayalakshmi Saravanan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rajendra Pamula
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Praphula Kumar Jain.

Ethics declarations

Conflict of interest

All authors declare that they do not any conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jain, P.K., Quamer, W., Saravanan, V. et al. Employing BERT-DCNN with sentic knowledge base for social media sentiment analysis. J Ambient Intell Human Comput 14, 10417–10429 (2023). https://doi.org/10.1007/s12652-022-03698-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-022-03698-z

Keywords

Search

Navigation