Applied Intelligence

, Volume 49, Issue 11, pp 3947–3964 | Cite as

Adaptive deep Q-learning model for detecting social bots and influential users in online social networks

  • Greeshma LingamEmail author
  • Rashmi Ranjan Rout
  • D. V. L. N. Somayajulu


In an online social network (like Twitter), a botmaster (i.e., leader among a group of social bots) establishes a social relationship among legitimate participants to reduce the probability of social bot detection. Social bots generate fake tweets and spread malicious information by manipulating the public opinion. Therefore, the detection of social bots in an online social network is an important task. In this paper, we consider social attributes, such as tweet-based attributes, user profile-based attributes and social graph-based attributes for detecting the social bots among legitimate participants. We design a deep Q-network architecture by incorporating a Deep Q-Learning (DQL) model using the social attributes in the Twitter network for detection of social bots based on updating Q-value function (i.e., state-action value function). We consider each social attribute of a user as a state and the learning agent’s movement from one state to another state is considered as an action. For Q-value function, we consider all the state-action pairs in order to construct the state transition probability values between the state-action pairs. In the proposed DQL algorithm, the learning agent chooses a specific learning action with an optimal Q-value in each state for social bot detection. Further, we also propose an approach that identifies the most influential users (which are influenced by the social bots) based on tweets and the users’ interactions. The experimentation using the datasets collected from Twitter network illustrates the efficacy of proposed model.


Social bot Deep Q-learning Twitter Online social networks 



  1. 1.
    Ala’M AZ, Faris H, Hassonah MA, et al. (2018) Evolving support vector machines using whale optimization algorithm for spam profiles detection on online social networks in different lingual contexts. Knowl-Based Syst 153:91–104CrossRefGoogle Scholar
  2. 2.
    Albladi SM, Weir GR (2018) User characteristics that influence judgment of social engineering attacks in social networks. Human-centric Comput Inf Sci 8(1):5CrossRefGoogle Scholar
  3. 3.
    Alshahrani M, Zhu F, Zheng L, Mekouar S, Huang S (2018) Selection of top-k influential users based on radius-neighborhood degree, multi-hops distance and selection threshold. J Big Data 5(1):28CrossRefGoogle Scholar
  4. 4.
    Ashfaq AB, Abaid Z, Ismail M, Aslam MU, Syed AA, Khayam SA (2016) Diagnosing bot infections using Bayesian inference. J Comput Virol Hacking Techniq, 1–18Google Scholar
  5. 5.
    Baltazar J, Costoya J, Flores R (2009) The real face of koobface: the largest web 2.0 botnet explained. Trend Micro Res 5(9):10Google Scholar
  6. 6.
    Barushka A, Hajek P (2018) Spam filtering using integrated distribution-based balancing approach and regularized deep neural networks. Appl Intell, 1–19Google Scholar
  7. 7.
    Blei DM, Ng AY, Jordan MI (2003) Latent Dirichlet allocation. J Mach Learn Res 3:993–1022zbMATHGoogle Scholar
  8. 8.
    Boshmaf Y, Muslukhov I, Beznosov K, Ripeanu M (2011) The socialbot network: when bots socialize for fame and money. In: Proceedings of the 27th annual computer security applications conference. ACM, pp 93–102Google Scholar
  9. 9.
    Cesarano C, Dorr B, Picariello A, Reforgiato D, Sagoff A, Subrahmanian V (2004) Oasys: an opinion analysis system. In: AAAI Spring symposium on computational approaches to analyzing weblogs (CAAW 2006), pp 21–26Google Scholar
  10. 10.
    Cheng Z, Caverlee J, Lee K (2013) A content-driven framework for geolocating microblog users. ACM Trans Intell Syst Technol (TIST) 4(1):2Google Scholar
  11. 11.
    Chowdhury S, Khanzadeh M, Akula R, Zhang F, Zhang S, Medal H, Marufuzzaman M, Bian L (2017) Botnet detection using graph-based feature clustering. J Big Data 4(1):14CrossRefGoogle Scholar
  12. 12.
    Chu Z, Gianvecchio S, Wang H, Jajodia S (2012) Detecting automation of twitter accounts: are you a human, bot, or cyborg? IEEE Trans Depend Secur Comput 9(6):811–824CrossRefGoogle Scholar
  13. 13.
    Cresci S, Di Pietro R, Petrocchi M, Spognardi A, Tesconi M (2017) The paradigm-shift of social spambots: evidence, theories, and tools for the arms race. In: Proceedings of the 26th international conference on world wide web companion. International World Wide Web Conferences Steering Committee, pp 963–972Google Scholar
  14. 14.
    Dickerson JP, Kagan V, Subrahmanian V (2014) Using sentiment to detect bots on twitter: are humans more opinionated than bots? In: 2014 IEEE/ACM International conference on advances in social networks analysis and mining (ASONAM), pp 620–627Google Scholar
  15. 15.
    Ferrara E, Varol O, Davis C, Menczer F, Flammini A (2016) The rise of social bots. Commun ACM 59(7):96–104CrossRefGoogle Scholar
  16. 16.
    François-Lavet V, Fonteneau R, Ernst D (2015) How to discount deep reinforcement learning: towards new dynamic strategies. arXiv:1512.02011
  17. 17.
    Freitas C, Benevenuto F, Veloso A, Ghosh S (2016) An empirical study of socialbot infiltration strategies in the twitter social network. Soc Netw Anal Min 6(1):23CrossRefGoogle Scholar
  18. 18.
    Gadaleta M, Chiariotti F, Rossi M, Zanella A (2017) D-dash: a deep q-learning framework for dash video streaming. IEEE Trans Cogn Commun Network 3(4):703–718CrossRefGoogle Scholar
  19. 19.
    Gilani Z, Wang L, Crowcroft J, Almeida M, Farahbakhsh R (2016) Stweeler: a framework for twitter bot analysis. In: Proceedings of the 25th international conference companion on world wide web. International World Wide Web Conferences Steering Committee, pp 37–38Google Scholar
  20. 20.
    Halfaker A, Riedl J (2012) Bots and cyborgs: Wikipedia’s immune system. Computer 45(3):79–82CrossRefGoogle Scholar
  21. 21.
    Islam R, Henderson P, Gomrokchi M, Precup D (2017) Reproducibility of benchmarked deep reinforcement learning tasks for continuous control. arXiv:1708.04133
  22. 22.
    Ji Y, He Y, Jiang X, Cao J, Li Q (2016) Combating the evasion mechanisms of social bots. Comput Secur 58:230–249CrossRefGoogle Scholar
  23. 23.
    Kaufmann M, Kalita J (2010) Syntactic normalization of twitter messages. In: International conference on natural language processing. KharagpurGoogle Scholar
  24. 24.
    Kudugunta S, Ferrara E (2018) Deep neural networks for bot detection. Inform Sci 467:312–322CrossRefGoogle Scholar
  25. 25.
    Kusy M, Zajdel R (2015) Application of reinforcement learning algorithms for the adaptive computation of the smoothing parameter for probabilistic neural network. IEEE Trans Neural Netw Learn Syst 26(9):2163–2175MathSciNetCrossRefGoogle Scholar
  26. 26.
    Lee K, Eoff BD, Caverlee J (2011) Seven months with the devils: a long-term study of content polluters on twitter. In: ICWSM, pp 185–192Google Scholar
  27. 27.
    Lee S, Kim J (2013) Fluxing Botnet command and control channels with url shortening services. Comput Commun 36(3):320–332CrossRefGoogle Scholar
  28. 28.
    Lee S, Kim J (2013) Warningbird: a near real-time detection system for suspicious urls in twitter stream. IEEE Trans Depend Sec Comput 10(3):183–195CrossRefGoogle Scholar
  29. 29.
    Leibo JZ, Zambaldi V, Lanctot M, Marecki J, Graepel T (2017) Multi-agent reinforcement learning in sequential social dilemmas. In: Proceedings of the 16th conference on autonomous agents and multiagent systems, pp 464–473Google Scholar
  30. 30.
    Lingam G, Rout RR, Somayajulu DV (2018) Learning automata-based trust model for user recommendations in online social networks. Comput Electric Eng 66:174–188CrossRefGoogle Scholar
  31. 31.
    Liu W, Zhang L, Tao D, Cheng J (2017) Reinforcement online learning for emotion prediction by using physiological signals. Pattern Recognition LettersGoogle Scholar
  32. 32.
    Ma Q, Ma J (2017) A robust method to discover influential users in social networks. Soft Comput, 1–13Google Scholar
  33. 33.
    Perera RD, Anand S, Subbalakshmi K, Chandramouli R (2010) Twitter analytics: architecture, tools and analysis. In: Military communications conference, 2010-MILCOM 2010. IEEE, pp 2186–2191Google Scholar
  34. 34.
    Sheikhahmadi A, Nematbakhsh MA, Zareie A (2017) Identification of influential users by neighbors in online social networks. Physica A: Stat Mech Appl 486:517–534CrossRefGoogle Scholar
  35. 35.
    Sirivianos M, Kim K, Yang X (2011) Socialfilter: introducing social trust to collaborative spam mitigation. In: INFOCOM, 2011 proceedings. IEEE, pp 2300–2308Google Scholar
  36. 36.
    Subrahmanian V, Azaria A, Durst S, Kagan V, Galstyan A, Lerman K, Zhu L, Ferrara E, Flammini A, Menczer F (2016) The darpa twitter bot challenge. Computer 49(6):38–46CrossRefGoogle Scholar
  37. 37.
    Subrahmanian VS, Reforgiato D (2008) Ava: adjective-verb-adverb combinations for sentiment analysis. IEEE Intell Syst 23(4):43–50CrossRefGoogle Scholar
  38. 38.
    Tang R, Fong S, Deb S, Vasilakos AV, Millham RC (2018) Dynamic group optimisation algorithm for training feed-forward neural networks. Neurocomputing 314:1–19CrossRefGoogle Scholar
  39. 39.
    Teng TH, Tan AH, Zurada JM (2015) Self-organizing neural networks integrating domain knowledge and reinforcement learning. IEEE Trans Neural Netw Learn Syst 26(5):889–902MathSciNetCrossRefGoogle Scholar
  40. 40.
    Venkatachalam N, Anitha R (2017) A multi-feature approach to detect stegobot: a covert multimedia social network botnet. Multimed Tools Appl 76(4):6079–6096CrossRefGoogle Scholar
  41. 41.
    Wagner C, Mitter S, Körner C, Strohmaier M (2012) When social bots attack: modeling susceptibility of users in online social networks. Making Sense Microposts (# MSM2012) 2(4):1951–1959Google Scholar
  42. 42.
    Wei Q, Lewis FL, Sun Q, Yan P, Song R (2017) Discrete-time deterministic q-learning: a novel convergence analysis. IEEE Trans Cybern 47(5):1224–1237CrossRefGoogle Scholar
  43. 43.
    Wei W, Xu F, Tan CC, Li Q (2012) Sybildefender: defend against sybil attacks in large social networks. In: INFOCOM, 2012 proceedings. IEEE, pp 1951–1959Google Scholar
  44. 44.
    Wu J, Sha Y, Li R, Liang Q, Jiang B, Tan J, Wang B (2017) Identification of influential users based on topic-behavior influence tree in social networks. In: National CCF conference on natural language processing and Chinese computing. Springer, pp 477–489Google Scholar
  45. 45.
    Yan G (2013) Peri-watchdog: hunting for hidden Botnets in the periphery of online social networks. Comput Netw 57(2):540–555CrossRefGoogle Scholar
  46. 46.
    Zhang J, Zhang R, Sun J, Zhang Y, Zhang C (2016) Truetop: a sybil-resilient system for user influence measurement on twitter. IEEE/ACM Trans Network 24(5):2834–2846CrossRefGoogle Scholar
  47. 47.
    Zhang J, Zhang R, Zhang Y, Yan G (2016) The rise of social botnets: attacks and countermeasures. IEEE Transactions on Dependable and Secure ComputingGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • Greeshma Lingam
    • 1
    Email author
  • Rashmi Ranjan Rout
    • 1
  • D. V. L. N. Somayajulu
    • 1
    • 2
  1. 1.Computer Science and EngineeringNational Institute of TechnologyWarangalIndia
  2. 2.Information Technology Design and Manufacturing (IIITDM)KurnoolIndia

Personalised recommendations