Automatic Acquisition of User Models of Interaction to Evaluate the Usability of Virtual Environments

  • Nader Hanna
  • Deborah Richards
  • Michael J. Jacobson
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7457)


Evaluation is an essential step in the research and development of software, particularly for new technologies such as Virtual Environments. One of the challenges to such evaluation is to collect data needed for analyzing the behavior of the users of the virtual environment. Conventional acquisition of evaluation data is time-consuming and prone to bias. In this paper, we present a taxonomy to assist identification and collection of appropriate variables for automatic data collection. We further show how these variables, such as navigation paths and characters met, can be used to capture the behavioral interaction of learners in a virtual ecosystem and to produce a user-model to evaluate the usability of the world under development.


Virtual Environment Automatic Data Acquisition Usability Evaluation User-model Data Collection 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Hix, D., Hartson, H.R.: Developing User Interfaces: Ensuring Usability through Product & Process. John Wiley and Sons, New York (1993)MATHGoogle Scholar
  2. 2.
    Johnson, C.R.: Evaluating the Contribution of DesktopVR for Safety-Critical Applications. In: Felici, M., Kanoun, K., Pasquini, A. (eds.) SAFECOMP 1999. LNCS, vol. 1698, pp. 67–78. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  3. 3.
    Jacko, J.A., Sears, A.: The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies, and Emerging Applications. Psychology Press (2003)Google Scholar
  4. 4.
    Teixeira, L., Rebelo, F., Filgueiras, E.: Human Interaction Data Acquisition Software for Virtual Reality: A User-Centered Design Approach, pp. 793–801. CRC Press/Taylor & Francis, Ltd. (2010)Google Scholar
  5. 5.
    Bowman, D.A., Gabbard, J.L., Hix, D.: A Survey of Usability Evaluation in Virtual Environments: Classification and Comparison of Methods. PRESENCE: Teleoperators and Virtual Environments 11, 404–424 (2002)CrossRefGoogle Scholar
  6. 6.
    Slater, M.: How Colorful was Your Day? Why Questionnaires Cannot Assess Presence in Virtual Environments. Presence: Teleoperators and Virtual Environments 13, 484–493 (2004)CrossRefGoogle Scholar
  7. 7.
    Zhang, Y., Yu, X., Dang, Y., Chen, H.: An Integrated Framework for Avatar Data Collection from the Virtual World. IEEE Intelligent Systems 25, 17–23 (2010)MATHCrossRefGoogle Scholar
  8. 8.
    de Leeuw, E., Nicholls II, W.: Technological Innovations in Data Collection: Acceptance, Data Quality and Costs. Sociological Research Online 1 (1996)Google Scholar
  9. 9.
    Lampton, D., Bliss, J., Morris, C.: Human Performance Measurement in Virtual Environments. In: Stanney, K.M. (ed.) Handbook of Virtual Environments: Design, Implementation, and Applications, pp. 701–720. Lawrence Erlbaum Associates, Mahwah (2002)Google Scholar
  10. 10.
    Siochi, A.C., Hix, D.: A study of computer-supported user interface evaluation using maximal repeating pattern analysis. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: Reaching Through Technology, pp. 301–305. ACM, 108926 (1991)CrossRefGoogle Scholar
  11. 11.
    Andreas, K., Tsiatsos, T., Terzidou, T., Pomportsis, A.: Fostering Collaborative Learning in Second Life: Metaphors and Affordances. Computers & Education 55, 603–615 (2010)CrossRefGoogle Scholar
  12. 12.
    Yee, N., Bailenson, J.: A Method for Longitudinal Behavioral Data Collection in Second Life. PRESENCE: Teleoperators and Virtual Environments 17, 594–596 (2008)CrossRefGoogle Scholar
  13. 13.
    Duarte, E., Vilar, E., Rebelo, F., Teles, J., Almeida, A.: Some Evidences of the Impact of Environment’s Design Features in Routes Selection in Virtual Environments. In: Shumaker, R. (ed.) Virtual and Mixed Reality 2011, Part I. LNCS, vol. 6773, pp. 154–163. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  14. 14.
    Laaki, H., Kaurila, K., Ots, K., Nuckchady, V., Belimpasakis, P.: Augmenting virtual worlds with real-life data from mobile devices. In: 2010 IEEE Virtual Reality Conference (VR), pp. 281–282 (2010)Google Scholar
  15. 15.
    Grillon, H., Thalmann, D.: Eye contact as trigger for modification of virtual character behavior. Virtual Rehabilitation, 205–211 (2008)Google Scholar
  16. 16.
    White, C.: Classifying Human Motion in Virtual Environments (Unpublished paper describing the system), pp. 1–6 (2006)Google Scholar
  17. 17.
    Sebok, A., Nystad, E.: Design and Evaluation of Virtual Reality Systems: A Process to Ensure Usability. In: Proceedings of the Virtual Reality Design and Evaluation Workshop (2004)Google Scholar
  18. 18.
    Holm, R., Priglinger, M., Stauder, E., Volkert, J., Wagner, R.: Automatic Data Acquisition and Visualization for Usability Evaluation of Virtual Reality Systems. In: Eurographics-Short Presentations (2002)Google Scholar
  19. 19.
    Chernova, S., De Palma, N., Morant, E., Breazeal, C.: Crowdsourcing human-robot interaction: Application from virtual to physical worlds. In: 2011 IEEE RO-MAN, pp. 21–26. IEEE (2011)Google Scholar
  20. 20.
    Bonebright, T.L., Miner, N.E., Goldsmith, T.E., Caudell, T.P.: Data Collection and Analysis Techniques for Evaluating the Perceptual Qualities of Auditory Stimuli. ACM Transactions on Applied Perception (TAP) 2, 505–516 (2005)CrossRefGoogle Scholar
  21. 21.
    Borner, K., Lin, Y.C.: Visualizing Chat Log Data Collected In 3-D Virtual Worlds. In: Proceedings. Fifth International Conference on Information Visualisation, pp. 141–146 (2001)Google Scholar
  22. 22.
    Borner, K., Hazlewood, W.R., Sy-Miaw, L.: Visualizing the spatial and temporal distribution of user interaction data collected in three-dimensional virtual worlds. In: 2002 Proceedings of the Sixth International Conference on Information Visualisation, pp. 25–31 (2002)Google Scholar
  23. 23.
    Teixeira, L., Vilar, E., Duarte, E., Rebelo, F.: Comparing Two Types of Navigational Interfaces for Virtual Reality. A Journal of Prevention, Assessment and Rehabilitation, 2195–2200 (2012)Google Scholar
  24. 24.
    Griffiths, G., Nichols, S.S.n., Wilson, J.R.: Performance of New Participants in Virtual Environments: The Nottingham Tool for Assessment of Interaction in Virtual Environments (NAIVE). International Journal of Human-Computer Studies 64, 240–250 (2006)CrossRefGoogle Scholar
  25. 25.
    Bruckman, A.: Chapter 58: Analysis of Log File Data to Understand Behavior and Learning in an Online Community. In: Weiss, J., Nolan, J., Hunsinger, J., Trifonas, P. (eds.) The International Handbook of Virtual Learning Environments, pp. 1449–1465. Springer, Dordrecht (2006)CrossRefGoogle Scholar
  26. 26.
    Orkin, J., Roy, D.: The Restaurant Game: Learning Social Behavior and Language from Thousands of Players Online. Journal of Game Development 3, 39–60 (2007)Google Scholar
  27. 27.
    Hartson, H.R., Castillo, J.C., Kelso, J., Neale, W.C.: Remote Evaluation: The Network as an Extension of the Usability Laboratory. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: Common Ground (CHI 1996), pp. 228–235. ACM (1996), 238511Google Scholar
  28. 28.
    Preece, J.: Online Communities: Designing Usability and Supporting Sociability. John Wiley & Sons, NewYork (2000)Google Scholar
  29. 29.
    Mikovec, Z., Maly, I., Slavik, P., Curin, J.: Visualization of Users’ Activities in a Specific Environment. In: The 2007 Winter Simulation Conference, pp. 738–746 (2007)Google Scholar
  30. 30.
    Schroeder, R., Heldal, I., Tromp, J.: The Usability of Collaborative Virtual Environments and Methods for the Analysis of Interaction. Journal of Presence 15, 655–667 (2006)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Nader Hanna
    • 1
  • Deborah Richards
    • 1
  • Michael J. Jacobson
    • 2
  1. 1.Department of ComputingMacquarie UniversityAustralia
  2. 2.Centre for Computer Supported Learning and CognitionThe University of SydneyAustralia

Personalised recommendations