Abstract
Today, implications of automation in social media, specifically whether social bots can be used to manipulate people’s thoughts and behaviors are discussed. Some believe that social bots are simple tools that amplify human-created content, while others claim that social bots do not exist at all and that the research surrounding them is a conspiracy theory. This paper discusses the potential of automation in online media and the challenges that may arise as technological advances continue. The authors believe that automation in social media exists, but acknowledge that there is room for improvement in current scientific methodology for investigating this phenomenon. They focus on the evolution of social bots, the state-of-the-art content generation technologies, and the perspective of content generation in games. They provide a background discussion on the human perception of content in computer-mediated communication and describe a new automation level, from which they derive interdisciplinary challenges.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
In the most general sense, we understand automation to mean technically controlled processes that ensure a specified target achievement largely without human intervention. In closed-loop systems, target achievement is controlled by feedback mechanisms and through self-regulating control mechanisms. In open-loop systems, no feedback mechanism is implemented [31].
- 2.
“Convincingly” in the sense that social media users are not aware of messaging with an automaton or consuming artificially generated content. This does not relate to direct change of opinion.
- 3.
- 4.
- 5.
- 6.
- 7.
The readers may ask themselves whether they can judge who wrote the abstract of this paper - the authors or GPT-3. In fact, the abstract has been generated automatically by GPT-3 using only the introduction chapter of this paper as input. No editing has been done by the authors.
- 8.
- 9.
References
Adelani, D.I., Mai, H., Fang, F., Nguyen, H.H., Yamagishi, J., Echizen, I.: Generating sentiment-preserving fake online reviews using neural language models and their human- and machine-based detection. In: Barolli, L., Amato, F., Moscato, F., Enokido, T., Takizawa, M. (eds.) AINA 2020. AISC, vol. 1151, pp. 1341–1354. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-44041-1_114
Alabdulkarim, A., Li, S., Peng, X.: Automatic story generation: challenges and attempts (2021). https://doi.org/10.48550/ARXIV.2102.12634. https://arxiv.org/abs/2102.12634
Alam, F., et al.: A survey on multimodal disinformation detection. In: The 29th International Conference on Computational Linguistics (COLING 2022). ACL (2022)
Alhayan, F., Pennington, D.R., Ruthven, I.: “She seems more human’’: understanding twitter users’ credibility assessments of dementia-related information. In: Smits, M. (ed.) Information for a Better World: Shaping the Global Future. LNCS, vol. 13193, pp. 292–313. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-96960-8_20
Arsenyan, J., Mirowska, A.: Almost human? A comparative case study on the social media presence of virtual influencers. Int. J. Hum. Comput. Stud. 155, 102694 (2021). https://doi.org/10.1016/j.ijhcs.2021.102694
Assenmacher, D., Clever, L., Frischlich, L., Quandt, T., Trautmann, H., Grimme, C.: Demystifying social bots: on the intelligence of automated social media actors. Soc. Media + Soc. 6(3) (2020). https://doi.org/10.1177/2056305120939264
Assenmacher, D., et al.: Benchmarking crisis in social media analytics: a solution for the data-sharing problem. Soc. Sci. Comput. Rev. (2021). https://doi.org/10.1177/08944393211012268
Bessi, A., Ferrara, E.: Social bots distort the 2016 U.S. Presidential election online discussion. First Monday 21(11) (2016). https://doi.org/10.5210/fm.v21i11.7090. https://firstmonday.org/ojs/index.php/fm/article/view/7090
Bilewicz, M., et al.: Artificial intelligence against hate: intervention reducing verbal aggression in the social network environment. Aggress. Behav. 47(3), 260–266 (2021)
Boneh, D., Grotto, A.J., McDaniel, P., Papernot, N.: How relevant is the Turing test in the age of sophisbots? IEEE Secur. Priv. 17(6), 64–71 (2019)
Brown, T., et al.: Language models are few-shot learners. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.F., Lin, H. (eds.) Advances in Neural Information Processing Systems, vol. 33, pp. 1877–1901. Curran Associates, Inc. (2020)
Chen, D., Dolan, W.: Collecting highly parallel data for paraphrase evaluation. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Portland, Oregon, USA, pp. 190–200. Association for Computational Linguistics (2011). https://aclanthology.org/P11-1020
Ciechanowski, L., Przegalinska, A., Magnuski, M., Gloor, P.: In the shades of the uncanny valley: an experimental study of human-chatbot interaction. Futur. Gener. Comput. Syst. 92, 539–548 (2019). https://doi.org/10.1016/j.future.2018.01.055
Cresci, S.: A decade of social bot detection. Commun. ACM 63(10), 72–83 (2020)
Cresci, S., Di Pietro, R., Petrocchi, M., Spognardi, A., Tesconi, M.: The paradigm-shift of social spambots: evidence, theories, and tools for the arms race. In: The 26th International Conference on World Wide Web Companion (WWW 2017), pp. 963–972 (2017)
Cresci, S., Petrocchi, M., Spognardi, A., Tognazzi, S.: The coming age of adversarial social bot detection. First Monday 26(7) (2021)
Cresci, S., Trujillo, A., Fagni, T.: Personalized interventions for online moderation. In: The 33rd ACM Conference on Hypertext and Social Media (HT 2022), pp. 248–251. ACM (2022)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, Minnesota, pp. 4171–4186. Association for Computational Linguistics (2019). https://doi.org/10.18653/v1/N19-1423
Di Pietro, R., Caprolu, M., Raponi, S., Cresci, S.: New Dimensions of Information Warfare. Advances in Information Security, vol. 84. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-60618-3
Di Pietro, R., Cresci, S.: Metaverse: security and privacy issues. In: The 3rd IEEE International Conference on Trust, Privacy and Security in Intelligent Systems, and Applications (TPS 2021), pp. 281–288. IEEE (2021)
Echeverría, J., De Cristofaro, E., Kourtellis, N., Leontiadis, I., Stringhini, G., Zhou, S.: LOBO: evaluation of generalization deficiencies in Twitter bot classifiers. In: The 34th Annual Computer Security Applications Conference (ACSAC 2018), pp. 137–146 (2018)
Edwards, C., Beattie, A.J., Edwards, A., Spence, P.R.: Differences in perceptions of communication quality between a twitterbot and human agent for information seeking and learning. Comput. Hum. Behav. 65, 666–671 (2016). https://doi.org/10.1016/j.chb.2016.07.003
Edwards, C., Edwards, A., Spence, P.R., Shelton, A.K.: Is that a bot running the social media feed? Testing the differences in perceptions of communication quality for a human agent and a bot agent on twitter. Comput. Hum. Behav. 33, 372–376 (2014). https://doi.org/10.1016/j.chb.2013.08.013
Eichenberg, C., Black, S., Weinbach, S., Parcalabescu, L., Frank, A.: Magma-multimodal augmentation of generative models through adapter-based finetuning. arXiv preprint arXiv:2112.05253 (2021). https://doi.org/10.48550/arXiv.2112.05253
Fagni, T., Falchi, F., Gambini, M., Martella, A., Tesconi, M.: TweepFake: about detecting deepfake tweets. PLoS ONE 16(5), e0251415 (2021)
Ferrara, E., Varol, O., Davis, C., Menczer, F., Flammini, A.: The rise of social bots. Commun. ACM 59(7), 96–104 (2016)
Gallwitz, F., Kreil, M.: The rise and fall of ‘social bot’ research. SSRN 3814191 (2021). https://ssrn.com/abstract=3814191
Gillespie, T.: Custodians of the Internet: Platforms, Content Moderation, and the Hidden Decisions That Shape Social Media. Yale University Press, New Haven (2018)
Grimme, C., Assenmacher, D., Adam, L.: Changing perspectives: is it sufficient to detect social bots? In: Meiselwitz, G. (ed.) SCSM 2018. LNCS, vol. 10913, pp. 445–461. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-91521-0_32
Grimme, C., Preuss, M., Adam, L., Trautmann, H.: Social bots: human-like by means of human control? Big Data 5(4), 279–293 (2017). https://doi.org/10.1089/big.2017.0044
Groover, M.: Fundamentals of Modern Manufacturing: Materials, Processes, and Systems. Wiley, Hoboken (2010)
He, B., Ahamad, M., Kumar, S.: PETGEN: personalized text generation attack on deep sequence embedding-based classification models. In: The 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining (KDD 2021), pp. 575–584 (2021)
Ho, A., Hancock, J., Miner, A.S.: Psychological, relational, and emotional effects of self-disclosure after conversations with a chatbot. J. Commun. 68(4), 712–733 (2018). https://doi.org/10.1093/joc/jqy026
Im, J., Tandon, S., Chandrasekharan, E., Denby, T., Gilbert, E.: Synthesized social signals: computationally-derived social signals from account histories. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA. ACM (2020). https://doi.org/10.1145/3313831.3376383
Ippolito, D., Duckworth, D., Callison-Burch, C., Eck, D.: Automatic detection of generated text is easiest when humans are fooled. In: Proceedings of the 58th Annual Meeting of the ACL, pp. 1808–1822. ACL (2020). https://doi.org/10.18653/v1/2020.acl-main.164
Jeon, Y.A.: Reading social media marketing messages as simulated self within a metaverse: an analysis of gaze and social media engagement behaviors within a metaverse platform. In: 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), pp. 301–303 (2022). https://doi.org/10.1109/VRW55335.2022.00068
Jhaver, S., Boylston, C., Yang, D., Bruckman, A.: Evaluating the effectiveness of deplatforming as a moderation strategy on Twitter. In: The 24th ACM Conference On Computer-Supported Cooperative Work And Social Computing (CSCW 2021). ACM (2021)
Johnson, M.R.: Collecting highly parallel data for paraphrase evaluation. In: Proceedings of the 6th Workshop on Procedural Content Generation. Dundee, Scotland, UK (2016). https://www.pcgworkshop.com
Knox, J.: The metaverse, or the serious business of tech frontiers. Postdigit. Sci. Educ. 4(2), 207–215 (2022). https://doi.org/10.1007/s42438-022-00300-9
Lee, L.H., et al.: All One Needs to Know about Metaverse: A Complete Survey on Technological Singularity, Virtual Ecosystem, and Research Agenda (2021). arXiv Preprint. https://doi.org/10.48550/ARXIV.2110.05352
Liapis, A., Yannakakis, G.N., Nelson, M.J., Preuss, M., Bidarra, R.: Orchestrating game generation. IEEE Trans. Games 11(1), 48–68 (2019). https://doi.org/10.1109/TG.2018.2870876
Lin, C.Y.: ROUGE: a package for automatic evaluation of summaries. In: Text Summarization Branches Out, Barcelona, Spain, pp. 74–81. ACL (2004). https://aclanthology.org/W04-1013
Livingstone, D.: Turing’s test and believable AI in games. Comput. Entertain. 4(1), 6 (2006). https://doi.org/10.1145/1111293.1111303
Mendoza, M., Tesconi, M., Cresci, S.: Bots in social and interaction networks: detection and impact estimation. ACM Trans. Inf. Syst. 39(1), 1–32 (2020)
Meta Platforms: The Facebook Company Is Now Meta (2021). https://about.fb.com/news/2021/10/facebook-company-is-now-meta/. Accessed 22 May 2022
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015). https://doi.org/10.1038/nature14236
Mori, M.: The uncanny valley. Energy 7(4), 33–35 (1970). https://doi.org/10.1109/MRA.2012.2192811
Mozgovoy, M., Preuss, M., Bidarra, R.: Guest editorial special issue on team AI in games. IEEE Trans. Games 13(4), 327–329 (2021). https://doi.org/10.1109/TG.2021.3127967
Nass, C., Steuer, J., Tauber, E.R.: Computers are social actors. In: Conference Companion on Human Factors in Computing Systems, Boston, MA, USA, CHI 1994, p. 204. Association for Computing Machinery (1994). https://doi.org/10.1145/259963.260288
Nizzoli, L., Tardelli, S., Avvenuti, M., Cresci, S., Tesconi, M.: Coordinated behavior on social media in 2019 UK general election. In: The 15th International AAAI Conference on Web and Social Media (ICWSM 2021), pp. 443–454. AAAI (2021)
Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL 2002, Philadelphia, Pennsylvania, p. 311. Association for Computational Linguistics (2001). https://doi.org/10.3115/1073083.1073135
Pohl, J.S., Assenmacher, D., Seiler, M.V., Trautmann, H., Grimme, C.: Artificial social media campaign creation for benchmarking and challenging detection approaches. In: Proceedings of the 16th International Conference on Web and Social Media. NEATCLasS, Association for the Advancement of Artificial Intelligence (AAI), Hybrid: Atlanta, Georgia, US and Online (2022)
Rabkin, M.: Connect 2021 Recap: Horizon Home, the Future of Work, Presence Platform, and More (2021). https://www.oculus.com/blog/connect-2021-recap-horizon-home-the-future-of-work-presence-platform-and-more/. Accessed 22 May 2022
Radford, A., Narasimhan, K., Salimans, T., Sutskever, I.: Improving language understanding by generative pre-training. Technical report, OpenAI (2018)
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., Chen, M.: Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125 (2022). https://doi.org/10.48550/ARXIV.2204.06125
Rauchfleisch, A., Kaiser, J.: The false positive problem of automatic bot detection in social science research. PLoS ONE 15(10), e0241045 (2020)
Rauschnabel, P.A., Felix, R., Hinsch, C., Shahab, H., Alt, F.: What is XR? Towards a framework for augmented and virtual reality. Comput. Hum. Behav. 133, 107289 (2022). https://doi.org/10.1016/j.chb.2022.107289
Reed, S., et al.: A generalist agent (2022). https://doi.org/10.48550/ARXIV.2205.06175. https://arxiv.org/abs/2205.06175
Reeves, B., Nass, C.: The Media Equation: How People Treat Computers, Television, and New Media Like Real People and Pla. Bibliovault OAI Repository, the University of Chicago Press (1996)
Riedl, M.O.: Computational narrative intelligence: a human-centered goal for artificial intelligence. arXiv preprint arXiv:1602.06484 (2016)
Risi, S., Preuss, M.: From chess and atari to StarCraft and beyond: how game AI is driving the world of AI. KI - Künstliche Intelligenz 34(1), 7–17 (2020). https://doi.org/10.1007/s13218-020-00647-w
Saygin, A.P., Chaminade, T., Ishiguro, H., Driver, J., Frith, C.: The thing that should not be: predictive coding and the uncanny valley in perceiving human and humanoid robot actions. Soc. Cogn. Affect. Neurosci. 7(4), 413–422 (2012). https://doi.org/10.1093/scan/nsr025
Shin, M., Song, S.W., Chock, T.M.: Uncanny valley effects on friendship decisions in virtual social networking service. Cyberpsychol. Behav. Soc. Netw. 22(11), 700–705 (2019). https://doi.org/10.1089/cyber.2019.0122
Silver, D., et al.: Mastering the game of go with deep neural networks and tree search. Nature 529(7587), 484–489 (2016). https://doi.org/10.1038/nature16961
Skjuve, M., Haugstveit, I., Følstad, A., Brandtzaeg, P.: Help! Is my chatbot falling into the uncanny valley? An empirical study of user experience in human-chatbot interaction. Hum. Technol. 15, 30–54 (2019). https://doi.org/10.17011/ht/urn.201902201607
Spence, P.R., Edwards, A., Edwards, C., Jin, X.: ‘The bot predicted rain, grab an umbrella’: few perceived differences in communication quality of a weather twitterbot versus professional and amateur meteorologists. Behav. Inf. Technol. 38(1), 101–109 (2019). https://doi.org/10.1080/0144929X.2018.1514425
Stephenson, N.: Snow Crash. Metropolis Media (1992)
Sundar, S.S.: The MAIN Model: A Heuristic Approach to Understanding Technology Effects on Credibility. Digital Media, p. 29 (2008)
Tardelli, S., Avvenuti, M., Tesconi, M., Cresci, S.: Characterizing social bots spreading financial disinformation. In: Meiselwitz, G. (ed.) HCII 2020. LNCS, vol. 12194, pp. 376–392. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-49570-1_26
Tekiroglu, S., Bonaldi, H., Fanton, M., Guerini, M.: Using pre-trained language models for producing counter narratives against hate speech: a comparative study. In: Findings of the Association for Computational Linguistics (ACL 2022), pp. 3099–3114. ACL (2022)
Togelius, J.: We tried learning AI from games. How about learning from players? (2022). https://modl.ai/learning-ai-from-players. modl.ai blog
Togelius, J., et al.: Procedural content generation: goals, challenges and actionable steps. In: Lucas, S.M., Mateas, M., Preuss, M., Spronck, P., Togelius, J. (eds.) Artificial and Computational Intelligence in Games, Dagstuhl Follow-Ups, vol. 6, pp. 61–75. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, Dagstuhl, Germany (2013). https://doi.org/10.4230/DFU.Vol6.12191.61. http://drops.dagstuhl.de/opus/volltexte/2013/4336
Trujillo, A., Cresci, S.: Make reddit great again: assessing community effects of moderation interventions on r/The_Donald. In: The 25th ACM Conference On Computer-Supported Cooperative Work And Social Computing (CSCW 2022). ACM (2022)
Tsimpoukelli, M., Menick, J., Cabi, S., Eslami, S.M.A., Vinyals, O., Hill, F.: Multimodal few-shot learning with frozen language models. In: Beygelzimer, A., Dauphin, Y., Liang, P., Vaughan, J.W. (eds.) Advances in Neural Information Processing Systems (2021)
Vaswani, A., et al.: Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS 2017, pp. 6000–6010. Curran Associates Inc., Red Hook (2017)
Walther, J.B.: Computer-mediated communication: impersonal, interpersonal, and hyperpersonal interaction. Commun. Res. 23(1), 3–43 (1996). https://doi.org/10.1177/009365096023001001
Weber, D., Neumann, F.: Amplifying influence through coordinated behaviour in social networks. Soc. Netw. Anal. Min. 11(1), 1–42 (2021). https://doi.org/10.1007/s13278-021-00815-2
Xu, K., Liao, T.: Explicating cues: a typology for understanding emerging media technologies. J. Comput.-Mediat. Commun. 25(1), 32–43 (2020). https://doi.org/10.1093/jcmc/zmz023
Yang, C., Harkreader, R., Gu, G.: Empirical evaluation and new design for fighting evolving Twitter spammers. IEEE Trans. Inf. Forensics Secur. 8(8), 1280–1293 (2013)
Yang, K.C., Varol, O., Hui, P.M., Menczer, F.: Scalable and generalizable social bot detection through data selection. In: Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA, vol. 34 (2020). https://doi.org/10.1609/aaai.v34i01.5460
Yannakakis, G.N., Togelius, J.: Experience-driven procedural content generation. IEEE Trans. Affect. Comput. 2(3), 147–161 (2011). https://doi.org/10.1109/T-AFFC.2011.6
Zhang, T., Kishore, V., Wu, F., Weinberger, K.Q., Artzi, Y.: BERTScore: evaluating text generation with BERT. arXiv preprint arXiv:1904.09675 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Grimme, C., Pohl, J., Cresci, S., Lüling, R., Preuss, M. (2022). New Automation for Social Bots: From Trivial Behavior to AI-Powered Communication. In: Spezzano, F., Amaral, A., Ceolin, D., Fazio, L., Serra, E. (eds) Disinformation in Open Online Media. MISDOOM 2022. Lecture Notes in Computer Science, vol 13545 . Springer, Cham. https://doi.org/10.1007/978-3-031-18253-2_6
Download citation
DOI: https://doi.org/10.1007/978-3-031-18253-2_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-18252-5
Online ISBN: 978-3-031-18253-2
eBook Packages: Computer ScienceComputer Science (R0)