Skip to main content
SpringerLink
Log in

An Eye-Tracking Study of Differences in Reading Between Automated and Human-Written News

  • Conference paper
  • First Online:
Information Systems and Neuroscience (NeuroIS 2020)

Part of the book series: Lecture Notes in Information Systems and Organisation ((LNISO,volume 43))

Included in the following conference series:

Abstract

An eye-tracking experiment (N = 24) was conducted to study differences in reading between automated and human-written news. This work adopted expectation-confirmation theory to examine readers’ prior expectations and actual perceptions of both human-written news and automated news. Results revealed that nine eye-tracking variables were significantly different when people read automated news vs. human-written news. Findings also showed promising classification results of 31 eye-tracking-derived features. Self-reported results showed that the readability of human-written news was perceived as significantly higher than that of automated news.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Napoli, P.M.: Automated media: an institutional theory perspective on algorithmic media production and consumption. Commun. Theory 24, 340–360 (2014). https://doi.org/10.1111/comt.12039

    Article  Google Scholar 

  2. Graefe, A.: Guide to Automated Journalism. Columbia Journalism School, Tow Center for Digital Journalism, New York (2016)

    Google Scholar 

  3. Celeste LeCompte: Automation in the newsroom: How algorithms are helping reporters expand coverage, engage audiences, and respond to breaking news (2015). https://niemanreports.org/articles/automation-in-the-newsroom/

  4. Dubay, W.H.: The Principles of Readability. Impact Information, Costa Mesa (2004)

    Google Scholar 

  5. Dalecki, L., Lasorsa, D.L., Lewis, S.C.: The news readability problem. Journal. Pract. 3, 7–8 (2009)

    Google Scholar 

  6. Diakopoulos, N.: Algorithmic accountability. Digital Journal. 3(3), 398–415 (2015). https://doi.org/10.1080/21670811.2014.976411

    Article  Google Scholar 

  7. Clerwall, C.: Enter the robot journalist. Journal. Practice 5, 519–531 (2014). https://doi.org/10.1080/17512786.2014.883116

    Article  Google Scholar 

  8. Haim, M., Graefe, A.: Automated news: better than expected? Digital Journal. 5, 1044–1059 (2017)

    Article  Google Scholar 

  9. Jia, C.: Chinese automated journalism: a comparison between expectations and perceived quality. Int. J. Commun. 14, 2611–2632 (2020)

    Google Scholar 

  10. Carlson, M.: The robotic reporter. Digital Journal. 3(3), 416–431 (2015). https://doi.org/10.1080/21670811.2014.976412

    Article  Google Scholar 

  11. van der Kaa, H., Krahmer, E.: Journalist versus news consumer: the perceived credibility of machine written news. BMJ 2(5147), 305 (2014)

    Google Scholar 

  12. Shyam Sundar, S.: Exploring receivers’ criteria for perception of print and online news. Journal. Mass Commun. Q. 76(2), 373–386 (1999). https://doi.org/10.1177/107769909907600213

    Article  Google Scholar 

  13. Graefe, A., Haim, M., Haarmann, B., Brosius, H.-B.: Readers’ perception of computer-generated news: credibility, expertise, and readability. Journalism 19(5), 595–610 (2018). https://doi.org/10.1177/1464884916641269

    Article  Google Scholar 

  14. Seib, C.: Papers need to work on handling the English language, Austin American-Statesman, p. B25 (1976)

    Google Scholar 

  15. Fishkin, S.F.: From Fact to Fiction: Journalism & Imaginative Writing in America. Oxford University Press, Oxford (1985)

    Google Scholar 

  16. Flesch, R.F.: A new readability yardstick. J. Appl. Psychol. 32, 221–233 (1948)

    Article  Google Scholar 

  17. Flesch, R.F.: The Art of Readable Writing. Harper, New York (1949)

    Google Scholar 

  18. Gedeon, T., Caldwell, S.: Effects of text difficulty and readers on predicting reading comprehension from eye movements. In: 2015 6th IEEE International Conference on Cognitive Infocommunications (CogInfoCom), Gyor, pp. 407–412 (2015). https://doi.org/10.1109/coginfocom.2015.7390628

  19. Rayner, K., Chace, K.H., Slattery, T.J., Ashby, J.: Eye movements as reflections of comprehension processes in reading. Sci. Stud. Read. 10, 241–255 (2006)

    Article  Google Scholar 

  20. Martínez-Gómez, P., Aizawa, A.: Recognition of understanding level and language skill using measurements of reading behavior. In: Proceedings of the 19th International Conference on Intelligent User Interfaces 2014, pp. 95–104 (2014). http://dx.doi.org/10.1145/2557500.2557546

  21. Gwizdka, J., Hosseini, R., Cole, M., Wang, S.: Temporal dynamics of eye-tracking and EEG during reading and relevance decisions. J. Assoc. Inf. Sci. Technol. 68(10), 2299–2312 (2017). https://doi.org/10.1002/asi.23904

    Article  Google Scholar 

  22. Gwizdka, J.: Differences in reading between word search and information relevance decisions: evidence from eye-tracking. In: Davis, F.D., Riedl, R., vom Brocke, J., Léger, P.-M., Randolph, A.B. (eds.) Information Systems and Neuroscience. LNISO, vol. 16, pp. 141–147. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-41402-7_18

    Chapter  Google Scholar 

  23. Cole, M.J., Gwizdka, J., Liu, C., Bierig, R., Belkin, N.J., Zhang, X.: Task and user effects on reading patterns in information search. Interact. Comput. 23(4), 346–362 (2011). https://doi.org/10.1016/j.intcom.2011.04.007

    Article  Google Scholar 

  24. Shojaeizadeh, M., Djamasbi, S.: Eye movements and reading behavior of younger and older users: an exploratory eye-tacking study. In: Zhou, J., Salvendy, G. (eds.) ITAP 2018. LNCS, vol. 10926, pp. 377–391. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-92034-4_29

    Chapter  Google Scholar 

  25. Sundar, R.P., Becker, M.W., Bello, N.M., Bix, L.: Quantifying age-related differences in information processing behaviors when viewing prescription drug labels. PLoS ONE 7(6) (2012) http://dx.doi.org.ezproxy.lib.utexas.edu/10.1371/journal.pone.0038819

  26. Angela, S., Tanya, B.: The influence of science reading comprehension on South African township learners’ learning of science. S. Afr. J. Sci. 115(1), 72–80 (2019)

    Google Scholar 

  27. Bhattacherjee, A.: Understanding information systems continuous: an expectation-confirmation model. MIS Q. 25(3), 351–370 (2001). https://doi.org/10.2307/3250921

    Article  Google Scholar 

  28. Lin, C.S., Sheng, W., Tsai, R.J.: Integrating perceived playfulness into expectation-confirmation model for web portal context. Inf. Manag. 42(5), 683–693 (2005)

    Article  Google Scholar 

  29. Oliver, R.L.: A cognitive model of the antecedents and consequences of satisfaction decisions. J. Mark. Res. 17, 460–469 (1980). https://doi.org/10.2307/3150499

    Article  Google Scholar 

  30. Oliver, R.L.: Cognitive, affective, and attribute bases of the satisfaction response. J. Consum. Res. 20, 418–430 (1993). http://dx.doi.org/10.1086/209358

  31. Oliver, R.L.: Satisfaction. A Behavioral Perspective on the Consumer, 2nd edn. Routledge, London (2015)

    Google Scholar 

  32. Salvucci, D.D., Goldberg, J.H.: Identifying fixations and saccades in eye-tracking protocols. In: Proceedings of the 2000 Symposium on Eye Tracking Research & Applications (ETRA 2000), 71–78 (2000). https://doi.org/10.1145/355017.355028

  33. Spreng, R.A., Olshavsky, R.W.: A desires congruency model of consumer satisfaction. J. Acad. Mark. Sci. 21(3), 169–177 (1993)

    Article  Google Scholar 

  34. Witten, I.H., Frank, E., Hall, M.A.: Data Mining, Fourth Edition: Practical Machine Learning Tools and Techniques. Morgan Kaufmann Publishers Inc., San Francisco (2016)

    Google Scholar 

  35. Chawla, N.V., Bowyer, K.W., Hall, L.O., Philip Kegelmeyer,W.: SMOTE: synthetic minority over-sampling technique. arXiv:1106.1813 Cs (2011)

  36. Breiman, L.: Random forests. Mach. Learn. 45, 5–32 (2011)

    Article  Google Scholar 

  37. Rayner, K., Pollatsek, A.: The Psychology of Reading. Lawrence Erlbaum Associates, Mahwah (1989)

    Google Scholar 

Download references

Acknowledgements

We thank Lan Li, who is a master student in the School of Information at the University of Texas at Austin, for contributing to the data collection.

Author information

Authors and Affiliations

  1. School of Journalism, University of Texas at Austin, Austin, USA

    Chenyan Jia

  2. School of Information, University of Texas at Austin, Austin, USA

    Jacek Gwizdka

Authors
  1. Chenyan Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jacek Gwizdka
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chenyan Jia .

Editor information

Editors and Affiliations

  1. Information Systems and Quantitative Sciences (ISQS), Texas Tech University, Lubbock, TX, USA

    Fred D. Davis

  2. University of Applied Sciences Upper Austria, Steyr, Austria

    René Riedl

  3. Department of Information Systems, University of Liechtenstein, Vaduz, Liechtenstein

    Jan vom Brocke

  4. Department of Information Technology, HEC Montréal, Montreal, QC, Canada

    Pierre-Majorique Léger

  5. Department of Information Systems, Kennesaw State University, Kennesaw, GA, USA

    Adriane B. Randolph

  6. Department for Digital Business, University of Applied Sciences Upper Austria, Steyr, Austria

    Thomas Fischer

Rights and permissions

Reprints and permissions

Copyright information

© 2020 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Jia, C., Gwizdka, J. (2020). An Eye-Tracking Study of Differences in Reading Between Automated and Human-Written News. In: Davis, F.D., Riedl, R., vom Brocke, J., Léger, PM., Randolph, A.B., Fischer, T. (eds) Information Systems and Neuroscience. NeuroIS 2020. Lecture Notes in Information Systems and Organisation, vol 43. Springer, Cham. https://doi.org/10.1007/978-3-030-60073-0_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-60073-0_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-60072-3

  • Online ISBN: 978-3-030-60073-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation