Skip to main content

YTTREX: Crowdsourced Analysis of YouTube’s Recommender System During COVID-19 Pandemic

  • Conference paper
  • First Online:
Information Management and Big Data (SIMBig 2020)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1410))

Included in the following conference series:


Algorithmic personalization is difficult to approach because it entails studying many different user experiences, with a lot of variables outside of our control. Two common biases are frequent in experiments: relying on corporate service API and using synthetic profiles with small regards of regional and individualized profiling and personalization. In this work, we present the result of the first crowdsourced data collections of YouTube’s recommended videos via YouTube Tracking Exposed (YTTREX). Our tool collects evidence of algorithmic personalization via an HTML parser, anonymizing the users. In our experiment we used a BBC video about COVID-19, taking into account 5 regional BBC channels in 5 different languages and we saved the recommended videos that were shown during each session. Each user watched the first five second of the videos, while the extension captured the recommended videos. We took into account the top20 recommended videos for each completed session, looking for evidence of algorithmic personalization. Our results showed that the vast majority of videos were recommended only once in our experiment. Moreover, we collected evidence that there is a significant difference between the videos we could retrieve using the official API and what we collected with our extension. These findings show that filter bubbles exist and that they need to be investigated with a crowdsourced approach.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others


  1. 1.

    Site:, AGPL3 code:

  2. 2.

    For reference see:

  3. 3.

  4. 4.

  5. 5.

  6. 6.

  7. 7.

  8. 8.

  9. 9.


  1. Fernandez, M., Harith, A.: Online misinformation: challenges and future directions. In: Companion Proceedings of the Web Conference 2018, WWW 2018, International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, pp. 595–602 (2018).

  2. Zollo, F., Bessi, A., Del Vicario, M., et al.: Debunking in a world of tribes. PLoS One 12(7) (2017).

  3. Del Vicario, M., Vivaldo, G., Bessi, A., et al.: Echo chambers: emotional contagion and group polarization on Facebook. Sci. Rep. 6, 37825 (2016).

    Article  Google Scholar 

  4. Pariser, E.: The Filter Bubble: What the Internet is Hiding from You. Penguin, London (2011)

    Google Scholar 

  5. Zimmer, F., Scheibe, K., Stock, M., et al.: Fake news in social media: bad algorithms or biased users? J. Inf. Sci. Theory Pract. 7(2), 40–53 (2019).

    Article  Google Scholar 

  6. Bruns, A.: Filter bubble. Internet Policy Rev. 8(4). (2019)

  7. Covington, P., Adams, J., Sargin, E.: Deep neural networks for YouTube recommendations. In: Proceedings of the 10th ACM Conference on Recommender Systems. ACM (2016).

  8. Zhe, Z., Lichan, H., Li, W., Jilin, et al.: Recommending what video to watch next: a multitask ranking system. In: Proceedings of the 13th ACM Conference on Recommender Systems (RecSys 2019), pp. 43–51. Association for Computing Machinery, New York (2019).

  9. Trielli, D., Diakopoulos, N.: Partisan search behavior and Google results in the 2018 U.S. midterm elections. Inf. Commun. Soc. (2020).

  10. McKay, D., Makri, S., Guiterrez-Lopez, M., et al.: We are the change that we seek: information interactions during a change of viewpoint. In: Proceedings of ACM Conference on Human Information Interaction and Retrieval (CHIIR 2020), p. 10. ACM, New York (2019).

  11. Robertson, R.E., Jiang, S., Joseph, K., et al.: Auditing partisan audience bias within google search. Proc. ACM Hum.-Comput. Interact. 2(CSCW), 22 (2018). Article 148

  12. Hargreaves, E., Agosti, C., Menasché, D., et al.: Biases in the Facebook news feed: a case study on the Italian elections. In: International Conference on Advances in Social Networks Analysis and Mining, Barcelona, August 2018. arXiv: 1807.08346 (2018)

  13. Arthurs, J., Drakopoulou, S., Gandini, A.: Researching YouTube. Convergence 24(1), 3–15 (2018).

    Article  Google Scholar 

  14. Song, M., Yun, J., Anatoliy, G.: Examining sentiments and popularity of pro-and anti-vaccination videos on YouTube. In: Proceedings of the 8th International Conference on Social Media & Society, pp. 1–8 (2017).

  15. Abisheva, A., Garcia, D., Schweitzer, F.: When the filter bubble bursts: collective evaluation dynamics in online communities. In: Proceedings of the 8th ACM Conference on Web Science, pp. 307–308 (2016).

  16. Bishop, S.: Anxiety, panic and self-optimization: inequalities and the YouTube algorithm. Convergence 24(1), 69–84 (2018).

    Article  Google Scholar 

  17. Rieder, B., Matamoros-Fernández, A., Coromina, O.: From ranking algorithms to ‘ranking cultures’: investigating the modulation of visibility in YouTube search results. Convergence 24(1), 50–68 (2018).

    Article  Google Scholar 

  18. Sandvig, C., Hamilton, K., Karahalios, K., Langbort, C.: Auditing algorithms: research methods for detecting discrimination on internet platforms. In: Data and Discrimination: Converting Critical Concerns into Productive Inquiry, a Preconference at the 64th Annual Meeting of the International Communication Association, 22 May 2014, Seattle, WA, USA (2014)

    Google Scholar 

  19. Bastian, M., Heymann, S., Jacomy, M.: Gephi: an open source software for exploring and manipulating networks. In: Third international AAAI Conference on Weblogs and Social Media (2009)

    Google Scholar 

  20. Six, J.M., Tollis, I.G.: A framework and algorithms for circular drawings of graphs. J. Discrete Algorithms 4(1), 25–50 (2006).

    Article  MathSciNet  MATH  Google Scholar 

  21. Brbić, M., Rožić, E., Žarko, I.P.: Recommendation of YouTube Videos. In: 2012 Proceedings of the 35th International Convention MIPRO, pp. 1775–1779. IEEE (2012)

    Google Scholar 

  22. Ledwich, M., Zaitsev, A.: Algorithmic extremism: examining YouTube’s rabbit hole of radicalization. arXiv preprint arXiv:1912.11211 (2019)

  23. Marchal, N., Au, H., Howard, P.N.: Coronavirus news and information on YouTube. Health 1(1), 0–3 (2020).

  24. Airoldi, M., Beraldo, D., Gandini, A.: Follow the algorithm: an exploratory investigation of music on YouTube. Poetics 57, 1–13 (2016).

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to Leonardo Sanna .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sanna, L., Romano, S., Corona, G., Agosti, C. (2021). YTTREX: Crowdsourced Analysis of YouTube’s Recommender System During COVID-19 Pandemic. In: Lossio-Ventura, J.A., Valverde-Rebaza, J.C., Díaz, E., Alatrista-Salas, H. (eds) Information Management and Big Data. SIMBig 2020. Communications in Computer and Information Science, vol 1410. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-76227-8

  • Online ISBN: 978-3-030-76228-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics