An Evaluation Environment for User Studies in the Public Transport Domain

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12213)


User studies to evaluate public transport systems are often hard to set up. While field tests provide important insight into real-world usability of public transport systems, they are also complex and expensive. Especially in early development stages of public transport related systems, field tests are not appropriate. However, usability of public transport systems is often depending on “real-life” context factors that are hard to reproduce in lab-based user studies. We have developed a mockup of a tram or train compartment that can be flexibly used to create a public transport experience in user studies. In this paper we will describe our experiences and recurring challenges with user studies in public transport, the design and set-up of our mockup, as well as give an insight into its applications in studies we conducted and lessons we learned.


User studies Public transport Evaluation Passenger information system Smart public transport 



This work was carried out as part of the research project “SmartMMI - model- and context-based mobility information on smart public displays and mobile devices in public transport” and was funded by the German Federal Ministry of Transport and Digital Infrastructure as part of the mFund initiative (Funding ID: 19F2042A). We would like to thank Johannes Bauer, Fabian Boschert-Hennrich, Kai-Lukas Schwägerl, Michael Wagner and Felix Werner for their contributions.


  1. 1.
    Abowd, G.D., Mynatt, E.D.: Charting past, present, and future research in ubiquitous computing. ACM Trans. Comput.-Hum. Interact. 7(1), 29–58 (2000)CrossRefGoogle Scholar
  2. 2.
    Alt, F., Schneegaundefined, S., Schmidt, A., Müller, J., Memarovic, N.: How to evaluate public displays. In: Proceedings of the 2012 International Symposium on Pervasive Displays (PerDis 2012). Association for Computing Machinery, New York (2012).
  3. 3.
    de Amorim, D.M., Dias, T.G., Ferreira, M.C.: Usability evaluation of a public transport mobile ticketing solution. In: Ahram, T., Karwowski, W., Taiar, R. (eds.) IHSED 2018. AISC, vol. 876, pp. 345–351. Springer, Cham (2019). Scholar
  4. 4.
    Beul-Leusmann, S., Habermann, A.L., Ziefle, M., Jakobs, E.M.: Unterwegs im öv. mobile fahrgastinformationssysteme in der usability evaluation. In: Prinz, W., Borchers, J., Jarke, M. (eds.) Mensch und Computer 2016 - Tagungsband. Gesellschaft für Informatik e.V., Aachen (2016).
  5. 5.
    Bezerra, C., et al.: Challenges for usability testing in ubiquitous systems. In: Proceedings of the 26th Conference on l’Interaction Homme-Machine (IHM 2014), pp. 183–188. Association for Computing Machinery, New York (2014).
  6. 6.
    Camacho, T., Foth, M., Rakotonirainy, A., Rittenbruch, M., Bunker, J.: The role of passenger-centric innovation in the future of public transport. Public Transp. 8(3), 453–475 (2016). Scholar
  7. 7.
    Chow, V.T.F., et al.: Utilizing real-time travel information, mobile applications and wearable devices for smart public transportation. In: 2016 7th International Conference on Cloud Computing and Big Data (CCBD), pp. 138–144, November 2016.
  8. 8.
    Chowdhury, S., Giacaman, N.: En-route planning of multi-destination public-transport trips using smartphones. J. Public Transp. 18(4), 31–45 (2015)CrossRefGoogle Scholar
  9. 9.
    Consolvo, S., Arnstein, L., Franza, B.R.: User study techniques in the design and evaluation of a ubicomp environment. In: Borriello, G., Holmquist, L.E. (eds.) UbiComp 2002. LNCS, vol. 2498, pp. 73–90. Springer, Heidelberg (2002). Scholar
  10. 10.
    Czaja, S., et al.: Factors predicting the use of technology: findings from the Center for Research and Education on Aging and Technology Enhancement (CREATE). Psychol. Aging 21(2), 333–352 (2006)CrossRefGoogle Scholar
  11. 11.
    García, C.R., Candela, S., Ginory, J., Quesada-Arencibia, A., Alayón, F.: On route travel assistant for public transport based on Android technology. In: 2012 Sixth International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing, pp. 840–845, July 2012.
  12. 12.
    Habermann, A.L., Kasugai, K., Ziefle, M.: Mobile app for public transport: a usability and user experience perspective. In: Mandler, B., et al. (eds.) IoT360 2015. LNICST, vol. 170, pp. 168–174. Springer, Cham (2016). Scholar
  13. 13.
    Handte, M., Foell, S., Wagner, S., Kortuem, G., Marrón, P.J.: An Internet-of-Things enabled connected navigation system for urban bus riders. IEEE Internet Things J. 3(5), 735–744 (2016). Scholar
  14. 14.
    Keller, C., Struwe, S., Titov, W., Schlegel, T.: Understanding the usefulness and acceptance of adaptivity in smart public transport. In: Krömker, H. (ed.) HCII 2019. LNCS, vol. 11596, pp. 307–326. Springer, Cham (2019). Scholar
  15. 15.
    Keller, C., Titov, W., Sawilla, S., Schlegel, T.: Evaluation of a smart public display in public transport. In: Mensch und Computer 2019 - Workshopband. Gesellschaft für Informatik e.V., Bonn (2019)Google Scholar
  16. 16.
    Kientz, J.A., Patel, S.N., Jones, B., Price, E., Mynatt, E.D., Abowd, G.D.: The Georgia Tech aware home. In: CHI’08 Extended Abstracts on Human Factors in Computing Systems (CHI EA 2008), pp. 3675–3680. Association for Computing Machinery, New York (2008).
  17. 17.
    Kim, S.H., Kim, S.W., Park, H.: Usability challenges in ubicomp environment. In: Proceedings of the International Ergonomics Association (2003)Google Scholar
  18. 18.
    Kjeldskov, J., Paay, J.: A longitudinal review of mobile HCI research methods. In: Proceedings of the 14th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI 2012), pp. 69–78. Association for Computing Machinery, New York (2012).
  19. 19.
    Kukka, H., Oja, H., Kostakos, V., Gonçalves, J., Ojala, T.: What makes you click: exploring visual signals to entice interaction on public displays. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2013), pp. 1699–1708. Association for Computing Machinery, New York (2013).
  20. 20.
    Kurniawan, S.: Older people and mobile phones: a multi-method investigation. Int. J. Hum.-Comput. Stud. 66(12), 889–901 (2008). Mobile human-computer interaction
  21. 21.
    Leung, R., McGrenere, J., Graf, P.: Age-related differences in the initial usability of mobile device icons. Behav. Inf. Technol. 30(5), 629–642 (2011).
  22. 22.
    Lyytinen, K., Yoo, Y.: Issues and challenges in ubiquitous computing. Commun. ACM 45(12), 62–65 (2002).
  23. 23.
    Mayas, C., Steinert, T., Krömker, H.: Interactive public displays for paperless mobility stations. In: Kurosu, M. (ed.) HCI 2018. LNCS, vol. 10902, pp. 542–551. Springer, Cham (2018). Scholar
  24. 24.
    Murata, A., Iwase, H.: Usability of touch-panel interfaces for older adults. Hum. Factors 47(4), 767–776 (2005). pMID: 16553065
  25. 25.
    Page, T.: Touchscreen mobile devices and older adults: a usability study. Int. J. Hum. Factors Ergon. (IJHFE) 3(1), 65–85 (2014)CrossRefGoogle Scholar
  26. 26.
    Poppe, R., Rienks, R., van Dijk, B.: Evaluating the future of HCI: challenges for the evaluation of emerging applications. In: Huang, T.S., Nijholt, A., Pantic, M., Pentland, A. (eds.) Artifical Intelligence for Human Computing. LNCS (LNAI), vol. 4451, pp. 234–250. Springer, Heidelberg (2007). Scholar
  27. 27.
    Prensky, M.: Digital Natives, Digital Immigrants Part 1, vol. 9, pp. 1–6. MCB University Press, Bingley (2001)Google Scholar
  28. 28.
    Reis, R.A.C., de Fontão, A.L., Gomes, L.L., Dias-Neto, A.C.: Usability evaluation approaches for (ubiquitous) mobile applications: a systematic mapping study. In: 9th International Conference on Mobile Ubiquitous Computing, Systems, Services and Technologies (2015)Google Scholar
  29. 29.
    Santos, R.M., de Oliveira, K.M., Andrade, R.M.C., Santos, I.S., Lima, E.R.: A quality model for human-computer interaction evaluation in ubiquitous systems. In: Collazos, C., Liborio, A., Rusu, C. (eds.) CLIHC 2013. LNCS, vol. 8278, pp. 63–70. Springer, Cham (2013). Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Institute of Ubiquitous Mobility SystemsKarlsruhe University of Applied SciencesKarlsruheGermany

Personalised recommendations