Abstract
Evaluation is an essential step in the research and development of software, particularly for new technologies such as Virtual Environments. One of the challenges to such evaluation is to collect data needed for analyzing the behavior of the users of the virtual environment. Conventional acquisition of evaluation data is time-consuming and prone to bias. In this paper, we present a taxonomy to assist identification and collection of appropriate variables for automatic data collection. We further show how these variables, such as navigation paths and characters met, can be used to capture the behavioral interaction of learners in a virtual ecosystem and to produce a user-model to evaluate the usability of the world under development.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Hix, D., Hartson, H.R.: Developing User Interfaces: Ensuring Usability through Product & Process. John Wiley and Sons, New York (1993)
Johnson, C.R.: Evaluating the Contribution of DesktopVR for Safety-Critical Applications. In: Felici, M., Kanoun, K., Pasquini, A. (eds.) SAFECOMP 1999. LNCS, vol. 1698, pp. 67–78. Springer, Heidelberg (1999)
Jacko, J.A., Sears, A.: The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies, and Emerging Applications. Psychology Press (2003)
Teixeira, L., Rebelo, F., Filgueiras, E.: Human Interaction Data Acquisition Software for Virtual Reality: A User-Centered Design Approach, pp. 793–801. CRC Press/Taylor & Francis, Ltd. (2010)
Bowman, D.A., Gabbard, J.L., Hix, D.: A Survey of Usability Evaluation in Virtual Environments: Classification and Comparison of Methods. PRESENCE: Teleoperators and Virtual Environments 11, 404–424 (2002)
Slater, M.: How Colorful was Your Day? Why Questionnaires Cannot Assess Presence in Virtual Environments. Presence: Teleoperators and Virtual Environments 13, 484–493 (2004)
Zhang, Y., Yu, X., Dang, Y., Chen, H.: An Integrated Framework for Avatar Data Collection from the Virtual World. IEEE Intelligent Systems 25, 17–23 (2010)
de Leeuw, E., Nicholls II, W.: Technological Innovations in Data Collection: Acceptance, Data Quality and Costs. Sociological Research Online 1 (1996)
Lampton, D., Bliss, J., Morris, C.: Human Performance Measurement in Virtual Environments. In: Stanney, K.M. (ed.) Handbook of Virtual Environments: Design, Implementation, and Applications, pp. 701–720. Lawrence Erlbaum Associates, Mahwah (2002)
Siochi, A.C., Hix, D.: A study of computer-supported user interface evaluation using maximal repeating pattern analysis. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: Reaching Through Technology, pp. 301–305. ACM, 108926 (1991)
Andreas, K., Tsiatsos, T., Terzidou, T., Pomportsis, A.: Fostering Collaborative Learning in Second Life: Metaphors and Affordances. Computers & Education 55, 603–615 (2010)
Yee, N., Bailenson, J.: A Method for Longitudinal Behavioral Data Collection in Second Life. PRESENCE: Teleoperators and Virtual Environments 17, 594–596 (2008)
Duarte, E., Vilar, E., Rebelo, F., Teles, J., Almeida, A.: Some Evidences of the Impact of Environment’s Design Features in Routes Selection in Virtual Environments. In: Shumaker, R. (ed.) Virtual and Mixed Reality 2011, Part I. LNCS, vol. 6773, pp. 154–163. Springer, Heidelberg (2011)
Laaki, H., Kaurila, K., Ots, K., Nuckchady, V., Belimpasakis, P.: Augmenting virtual worlds with real-life data from mobile devices. In: 2010 IEEE Virtual Reality Conference (VR), pp. 281–282 (2010)
Grillon, H., Thalmann, D.: Eye contact as trigger for modification of virtual character behavior. Virtual Rehabilitation, 205–211 (2008)
White, C.: Classifying Human Motion in Virtual Environments (Unpublished paper describing the system), pp. 1–6 (2006)
Sebok, A., Nystad, E.: Design and Evaluation of Virtual Reality Systems: A Process to Ensure Usability. In: Proceedings of the Virtual Reality Design and Evaluation Workshop (2004)
Holm, R., Priglinger, M., Stauder, E., Volkert, J., Wagner, R.: Automatic Data Acquisition and Visualization for Usability Evaluation of Virtual Reality Systems. In: Eurographics-Short Presentations (2002)
Chernova, S., De Palma, N., Morant, E., Breazeal, C.: Crowdsourcing human-robot interaction: Application from virtual to physical worlds. In: 2011 IEEE RO-MAN, pp. 21–26. IEEE (2011)
Bonebright, T.L., Miner, N.E., Goldsmith, T.E., Caudell, T.P.: Data Collection and Analysis Techniques for Evaluating the Perceptual Qualities of Auditory Stimuli. ACM Transactions on Applied Perception (TAP) 2, 505–516 (2005)
Borner, K., Lin, Y.C.: Visualizing Chat Log Data Collected In 3-D Virtual Worlds. In: Proceedings. Fifth International Conference on Information Visualisation, pp. 141–146 (2001)
Borner, K., Hazlewood, W.R., Sy-Miaw, L.: Visualizing the spatial and temporal distribution of user interaction data collected in three-dimensional virtual worlds. In: 2002 Proceedings of the Sixth International Conference on Information Visualisation, pp. 25–31 (2002)
Teixeira, L., Vilar, E., Duarte, E., Rebelo, F.: Comparing Two Types of Navigational Interfaces for Virtual Reality. A Journal of Prevention, Assessment and Rehabilitation, 2195–2200 (2012)
Griffiths, G., Nichols, S.S.n., Wilson, J.R.: Performance of New Participants in Virtual Environments: The Nottingham Tool for Assessment of Interaction in Virtual Environments (NAIVE). International Journal of Human-Computer Studies 64, 240–250 (2006)
Bruckman, A.: Chapter 58: Analysis of Log File Data to Understand Behavior and Learning in an Online Community. In: Weiss, J., Nolan, J., Hunsinger, J., Trifonas, P. (eds.) The International Handbook of Virtual Learning Environments, pp. 1449–1465. Springer, Dordrecht (2006)
Orkin, J., Roy, D.: The Restaurant Game: Learning Social Behavior and Language from Thousands of Players Online. Journal of Game Development 3, 39–60 (2007)
Hartson, H.R., Castillo, J.C., Kelso, J., Neale, W.C.: Remote Evaluation: The Network as an Extension of the Usability Laboratory. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: Common Ground (CHI 1996), pp. 228–235. ACM (1996), 238511
Preece, J.: Online Communities: Designing Usability and Supporting Sociability. John Wiley & Sons, NewYork (2000)
Mikovec, Z., Maly, I., Slavik, P., Curin, J.: Visualization of Users’ Activities in a Specific Environment. In: The 2007 Winter Simulation Conference, pp. 738–746 (2007)
Schroeder, R., Heldal, I., Tromp, J.: The Usability of Collaborative Virtual Environments and Methods for the Analysis of Interaction. Journal of Presence 15, 655–667 (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hanna, N., Richards, D., Jacobson, M.J. (2012). Automatic Acquisition of User Models of Interaction to Evaluate the Usability of Virtual Environments. In: Richards, D., Kang, B.H. (eds) Knowledge Management and Acquisition for Intelligent Systems. PKAW 2012. Lecture Notes in Computer Science(), vol 7457. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-32541-0_4
Download citation
DOI: https://doi.org/10.1007/978-3-642-32541-0_4
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-32540-3
Online ISBN: 978-3-642-32541-0
eBook Packages: Computer ScienceComputer Science (R0)