Same Task, Different Place: Developing Novel Simulation Environments with Equivalent Task Difficulties

Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 958)


We introduce a novel framework for creating and evaluating multiple virtual reality environments (VEs) that are naturalistic and similar in navigational complexity. We developed this framework in support of a spatial-learning study using a within-subjects design. We generated three interior environments and used graph-theoretic methods to ensure similar complexity. We then developed a scavenger-hunt task that ensured participants would visit all parts of the environments. Here, we describe VE development and a user study evaluating the relative task difficulty in the environments. Our results showed that our techniques were generally successful: the average time to complete the task was similar across environments. Some participants took longer to complete the task in one of the environments, indicating room for refinement of our framework. The methods described here should be of use for future studies using VEs, especially in within-subjects design.


Virtual environments Graph-theoretic measures Within-subjects designs Task development Floorplan design Bayesian modelling 



This work was funded by the US Army Research Laboratory’s Human sciences campaign. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Army Research Laboratory or U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein. The authors thank Debbie Patton, Mark Ericson, and the entire Training Effectiveness/Immersion group for their comments and suggestions on this work. Bianca Dalangin helped conduct the user study.


  1. 1.
    Witmer, B.G., Bailey, J.H., Knerr, B.W., Parsons, K.C.: Virtual spaces and real world places: transfer of route knowledge. Int. J. Hum.-Comput. Stud. 45, 413–428 (1996)CrossRefGoogle Scholar
  2. 2.
    Cummings, J.J., Bailenson, J.N.: How immersive is enough? a meta-analysis of the effect of immersive technology on user presence. Media Psychol. 19, 272–309 (2016)CrossRefGoogle Scholar
  3. 3.
    Grajewski, D., Górski, F., Zawadzki, P., Hamrol, A.: Application of virtual reality techniques in design of ergonomic manufacturing workplaces. Proc. Comput. Sci. 25, 289–301 (2013)CrossRefGoogle Scholar
  4. 4.
    Zimmons, P., Panter, A.: The influence of rendering quality on presence and task performance in a virtual environment. In: Proceedings of IEEE Virtual Reality 2003, pp. 293–294 (2003)Google Scholar
  5. 5.
    Steadman, P.: Graph theoretic representation of architectural arrangement. Archit. Res. Teach. 2, 161–172 (1973)Google Scholar
  6. 6.
    Roth, J., Hashimshony, R.: Algorithms in graph theory and their use for solving problems in architectural design. Comput.-Aided Des. 20, 373–381 (1988)CrossRefGoogle Scholar
  7. 7.
    Levy, R.M., O’Brien, M.G., Aorich, A.: Prediciting the behavior of game players - space syntax and urban planning theory as a predictive tool in game design. In: 2009 15th International Conference on Virtual Systems and Multimedia, pp. 203–208 (2009)Google Scholar
  8. 8.
    Rubinov, M., Sporns, O.: Complex network measures of brain connectivity: uses and interpretations. NeuroImage 52, 1059–1069 (2010)CrossRefGoogle Scholar
  9. 9.
    Sinatra, A.M., et al.: Development of cognitive transfer tasks for virtual environments and applications for adaptive instructional systems. In: Lecture Notes in Computer Science. Springer, Orlando (2019, forthcoming)Google Scholar
  10. 10.
    Carpenter, B., et al.: Stan : a probabilistic programming language. J. Stat. Softw. 76, 1–32 (2017)Google Scholar
  11. 11.
    Stan Development Team: RStan: the R interface to Stan (2018)Google Scholar
  12. 12.
    Vehtari, A., Gelman, A., Gabry, J.: Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. Stat. Comput. 27, 1413–1432 (2017)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Benedikt, M.L.: To take hold of space: isovists and isovist fields. Environ. Plan. B Plan. Des. 6, 47–65 (1979)CrossRefGoogle Scholar

Copyright information

© This is a U.S. government work and not under copyright protection in the U.S.; foreign copyright protection may apply 2020

Authors and Affiliations

  1. 1.US Army Research LaboratoryLos AngelesUSA
  2. 2.DCS CorporationLos AngelesUSA
  3. 3.Psychological and Brain SciencesUniversity of California at Santa BarbaraSanta BarbaraUSA
  4. 4.Computer Science, University of MinnesotaMinneapolisUSA
  5. 5.Natick Soldier Research, Development & Engineering Center – Simulation & Training Technology CenterOrlandoUSA

Personalised recommendations