Skip to main content
SpringerLink
Log in

Lost in Transformation: Rediscovering LLM-Generated Campaigns in Social Media

  • Conference paper
  • First Online:
Disinformation in Open Online Media (MISDOOM 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14397))

  • 622 Accesses

Abstract

This paper addresses new challenges of detecting campaigns in social media, which emerged with the rise of Large Language Models (LLMs). LLMs particularly challenge algorithms focused on the temporal analysis of topical clusters. Simple similarity measures can no longer capture and map campaigns that were previously broadly similar in content. Herein, we analyze whether the classification of messages over time can be profitably used to rediscover poorly detectable campaigns at the content level. Thus, we evaluate classical classifiers and a new method based on siamese neural networks. Our results show that campaigns can be detected despite the limited reliability of the classifiers as long as they are based on a large amount of simultaneously spread artificial content.

The authors acknowledge support by the European Research Center in Information Systems (ERCIS) and by the project HybriD (FKZ: 16KIS1531K) funded by the German Federal Ministry of Education and Research.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 49.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Alamleh, H., Al Qahtani, A., ElSaid, A.: Distinguishing human-written and ChatGPT-generated text using machine learning. In: Systems and Information Engineering Design Symposium, pp. 154–158. IEEE, Charlottesville, USA (2023)

    Google Scholar 

  2. Antoun, W., Mouilleron, V., Sagot, B., Seddah, D.: Towards a Robust Detection of Language Model Generated Text. arXiv 2306.05871 (2023)

    Google Scholar 

  3. Assenmacher, D., Adam, L., Trautmann, H., Grimme, C.: Towards real-time and unsupervised campaign detection in social media. In: Proceedings of the Florida Artificial Intelligence Research Society Conference. AAAI Press, Florida, USA (2020)

    Google Scholar 

  4. Assenmacher, D., Clever, L., Pohl, J.S., Trautmann, H., Grimme, C.: A two-phase framework for detecting manipulation campaigns in social media. In: Meiselwitz, G. (ed.) HCII 2020. LNCS, vol. 12194, pp. 201–214. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-49570-1_14

    Chapter  Google Scholar 

  5. Assenmacher, D., et al.: Demystifying social bots: on the intelligence of automated social media actors. Soc. Med. Soc. 6(3), 1–14 (2020)

    Google Scholar 

  6. Assenmacher, D., Trautmann, H.: Textual one-pass stream clustering with automated distance threshold adaption. In: Nguyen, N.T., Tran, T.K., Tukayev, U., Hong, TP., Trawiń,ski B., Szczerbicki, E. (eds.) Intelligent Information and Database Systems. ACIIDS 2022. LNCS, vol. 13757, pp. 3–16. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-21743-2_1

  7. Bellutta, D., Carley, K.M.: Investigating coordinated account creation using burst detection and network analysis. J. Big Data 10(1), 1–17 (2023)

    Article  Google Scholar 

  8. Bromley, J., Guyon, I., LeCun, Y., Säckinger, E., Shah, R.: Signature verification using a siamese time delay neural network. In: Cowan, J.D., Tesauro, G., Alspector, J. (eds.) Advances in Neural Information Processing Systems, vol. 6, NIPS, pp. 737–744. Morgan Kaufmann (1993)

    Google Scholar 

  9. Chakraborty, S., Bedi, A.S., Zhu, S., An, B., Manocha, D., Huang, F.: On the Possibilities of AI-Generated Text Detection (2023). arXiv:2304.04736

  10. Cinelli, M., Cresci, S., Quattrociocchi, W., Tesconi, M., Zola, P.: Coordinated inauthentic behavior and information spreading on Twitter. Decis. Support Syst. 160, 1–28 (2022). https://doi.org/10.1016/j.dss.2022.113819

    Article  Google Scholar 

  11. Cresci, S., Di Pietro, R., Petrocchi, M., Spognardi, A., Tesconi, M.: DNA-inspired online behavioral modeling and its application to spambot detection. IEEE Intell. Syst. 31(5), 58–64 (2016)

    Article  Google Scholar 

  12. Crothers, E., Japkowicz, N., Viktor, H.: Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methods (2023). arXiv:2210.07321

  13. Davis, C.A., Varol, O., Ferrara, E., Flammini, A., Menczer, F.: BotOrNot: a system to evaluate social bots. In: Proceedings of the 25th International Conference Companion on World Wide Web, pp. 273–274 (2016)

    Google Scholar 

  14. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding (2018). arXiv:1810.04805

  15. EleutherAI: GPT Neo - An Implementation of Model & Data Parallel GPT3-Like Models Using the Mesh-Tensorflow Library (2022). https://github.com/EleutherAI/gpt-neo. Accessed 02 July 2023

  16. Erhardt, K., Albassam, D.: Detecting the hidden dynamics of networked actors using temporal correlations. In: Companion Proceedings of the ACM Web Conference 2023, pp. 1214–1217. WWW 2023 Companion, ACM, Austin, TX, USA (2023)

    Google Scholar 

  17. Fagni, T., Falchi, F., Gambini, M., Martella, A., Tesconi, M.: TweepFake: about detecting deepfake Tweets. PLOS ONE. 16(5), e0251415 (2021)

    Article  Google Scholar 

  18. Fernández-Delgado, M., Cernadas, E., Barro, S., Amorim, D.: Do we need hundreds of classifiers to solve real world classification problems? J. Mach. Learn. Res. 15(1), 3133–3181 (2014)

    MathSciNet  MATH  Google Scholar 

  19. Ferrara, E.: Social bot detection in the age of ChatGPT: challenges and opportunities. First Monday. 28(6), 1–30 (2023). https://doi.org/10.5210/fm.v28i6.13185

    Article  Google Scholar 

  20. Gao, L., et al.: The Pile: An 800 GB Dataset of Diverse Text for Language Modeling (2020). arXiv:2101.00027

  21. Grimme, C., Preuss, M., Adam, L., Trautmann, H.: Social bots: human-like by means of human control? Big Data 5(4), 279–293 (2017)

    Article  Google Scholar 

  22. Grimme, C., Assenmacher, D., Adam, L.: Changing perspectives: is it sufficient to detect social bots? In: Proceedings of the International Conference on Human-Computer Interaction. Las Vegas, United States of America (2018)

    Google Scholar 

  23. Grimme, C., Pohl, J., Cresci, S., Lüling, R., Preuss, M.: New automation for social bots: from trivial behavior to AI-powered communication. In: Spezzano, F., Amaral, A., Ceolin, D., Fazio, L., Serra, E. (eds.) Disinformation in Open Online Media. MISDOOM 2022. LNCS, vol. 13545, pp. 79–99. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-18253-2_6

  24. Guo, B., et al.: How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection (2023). arXiv:2301.07597

  25. Ippolito, D., Duckworth, D., Callison-Burch, C., Eck, D.: Automatic detection of generated text is easiest when humans are fooled. In: Proceedings of the 58th Annual Meeting of the ACL, pp. 1808–1822. ACL, Online (2020)

    Google Scholar 

  26. James, G., Witten, D., Hastie, T., Tibshirani, R.: An Introduction to Statistical Learning, 2nd edn. Springer, New York (2021). https://doi.org/10.1007/978-1-4614-7138-7

    Book  MATH  Google Scholar 

  27. Kirchner, J.H., Ahmad, L., Aaronson, S., Leike, J.: New AI classifier for indicating AI-written text. https://openai.com/blog/new-ai-classifier-for-indicating-ai-written-text (2022). Accessed 30 June 2023

  28. Kumarage, T., Garland, J., Bhattacharjee, A., Trapeznikov, K., Ruston, S., Liu, H.: Stylometric Detection of AI-Generated Text in Twitter Timelines (2023). arXiv:2303.03697

  29. Michail, D., Kanakaris, N., Varlamis, I.: Detection of fake news campaigns using graph convolutional networks. Int. J. Inf. Manage. Data Insights 2(2), 100104 (2022)

    Google Scholar 

  30. Mitchell, E., Lee, Y., Khazatsky, A., Manning, C.D., Finn, C.: DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature (2023). arXiv:2301.11305

  31. Mitrović, S., Andreoletti, D., Ayoub, O.: ChatGPT or Human? Detect and Explain. Explaining Decisions of Machine Learning Model for Detecting Short ChatGPT-generated Text (2023). arXiv:2301.13852

  32. Pohl, J., Assenmacher, D., Seiler, M., Trautmann, H., Grimme, C.: Artificial social media campaign creation for benchmarking and challenging detection approaches. In: Workshop Proceedings of the 16\(^{th}\) International Conference on Web and Social Media. AAAI Press, Atlanta, GA, USA (2022)

    Google Scholar 

  33. Radford, A., Narasimhan, K., Salimans, T., Sutskever, I.: Improving Language Understanding by Generative Pre-Training. Technical report, OpenAI (2018)

    Google Scholar 

  34. Rauchfleisch, A., Kaiser, J.: The false positive problem of automatic bot detection in social science research. PLoS ONE 15(10), e0241045 (2020)

    Article  Google Scholar 

  35. Sadasivan, V.S., Kumar, A., Balasubramanian, S., Wang, W., Feizi, S.: Can AI-Generated Text be Reliably Detected? (2023). arXiv:2303.11156

  36. Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 815–823 (2015)

    Google Scholar 

  37. Singh, A.: A comparison study on AI language detector. In: 2023 IEEE 13th Annual Computing and Communication Workshop and Conference (CCWC), pp. 489–493. IEEE, Las Vegas, NV, USA (2023)

    Google Scholar 

  38. Tang, R., Chuang, Y.N., Hu, X.: The Science of Detecting LLM-Generated Texts (2023). arXiv:2303.07205

  39. Vaswani, A., et al.: Attention is all you need. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems. vol. 30. Curran Associates, Inc. (2017)

    Google Scholar 

  40. Verma, V., Fleisig, E., Tomlin, N., Klein, D.: Ghostbuster: Detecting Text Ghostwritten by Large Language Models (2023). arXiv:2305.15047

  41. Weber, D., Neumann, F.: Amplifying influence through coordinated behaviour in social networks. Soc. Netw. Anal. Min. 11(1), 1–42 (2021). https://doi.org/10.1007/s13278-021-00815-2

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, TU Dortmund University, Dortmund, Germany

    Britta Grimme

  2. Computational Social Science and Systems Analysis, University of Münster, Münster, Germany

    Janina Pohl, Lucas Stampe & Christian Grimme

  3. Practical Computer Science, University of Münster, Münster, Germany

    Hendrik Winkelmann

Authors
  1. Britta Grimme
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Janina Pohl
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hendrik Winkelmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lucas Stampe
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Christian Grimme
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christian Grimme .

Editor information

Editors and Affiliations

  1. Centrum Wiskunde & Informatica, Amsterdam, The Netherlands

    Davide Ceolin

  2. University of Groningen, Groningen, The Netherlands

    Tommaso Caselli

  3. University of Amsterdam, Amsterdam, The Netherlands

    Marina Tulin

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Grimme, B., Pohl, J., Winkelmann, H., Stampe, L., Grimme, C. (2023). Lost in Transformation: Rediscovering LLM-Generated Campaigns in Social Media. In: Ceolin, D., Caselli, T., Tulin, M. (eds) Disinformation in Open Online Media. MISDOOM 2023. Lecture Notes in Computer Science, vol 14397. Springer, Cham. https://doi.org/10.1007/978-3-031-47896-3_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-47896-3_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-47895-6

  • Online ISBN: 978-3-031-47896-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation