Neural Computing and Applications

, Volume 23, Issue 6, pp 1711–1728 | Cite as

Neural network-based detection of virtual environment anomalies

  • Alfredo NantesEmail author
  • Ross Brown
  • Frederic Maire
Original Article


The increasingly widespread use of large-scale 3D virtual environments has translated into an increasing effort required from designers, developers and testers. While considerable research has been conducted into assisting the design of virtual world content and mechanics, to date, only limited contributions have been made regarding the automatic testing of the underpinning graphics software and hardware. In the work presented in this paper, two novel neural network-based approaches are presented to predict the correct visualization of 3D content. Multilayer perceptrons and self-organizing maps are trained to learn the normal geometric and color appearance of objects from validated frames and then used to detect novel or anomalous renderings in new images. Our approach is general, for the appearance of the object is learned rather than explicitly represented. Experiments were conducted on a game engine to determine the applicability and effectiveness of our algorithms. The results show that the neural network technology can be effectively used to address the problem of automatic and reliable visual testing of 3D virtual environments.


Anomaly detection Visual correctness 3D virtual environment Virtual environment testing Computer game testing 


  1. 1.
    Association EES (2012) Industry facts. World Wide Web electronic publication. June 2012,
  2. 2.
    Balamuralithara B, Woods PC (2009) Virtual laboratories in engineering education: the simulation lab and remote lab. Comput Appl Eng Edu 17(1):108–118CrossRefGoogle Scholar
  3. 3.
    Bishop CM (1996) Neural networks for pattern recognition. Oxford University Press, New YorkzbMATHGoogle Scholar
  4. 4.
    Bogacz R, Brown MW, Giraud-Carrier C (1999) High capacity neural networks for familiarity discrimination. In: Proceedings of the 9th international conference on artificial neural networks (ICANN’99), Edinburgh, pp 773–778Google Scholar
  5. 5.
    Breiman L, Friedman J, Olshen R, Stone C (1984) Classification and regression trees. Wadsworth and Brooks, MontereyzbMATHGoogle Scholar
  6. 6.
    Browne C, Maire F (2010) Evolutionary game design. IEEE Trans Comput Intell AI Games 2(1):1–16CrossRefGoogle Scholar
  7. 7.
    Chandola V, Banerjee A, Kumar V (2009) Anomaly detection: a survey. ACM Comput Surv 41(3):1–58CrossRefGoogle Scholar
  8. 8.
    Demuth H, Beale M (1993) Neural network toolbox: for use with MATLAB: user’s guide. The Mathworks, MassachusettsGoogle Scholar
  9. 9.
    Desurvire H, Caplan M, Toth JA (2004) Using heuristics to evaluate the playability of games. In: Extended abstracts of conference on human factors in computing systems (CHI’04), New York, pp 1509–1512Google Scholar
  10. 10.
    Dickheiser M (2006) Game programming gems 6 (game development series). Charles River Media, RocklandGoogle Scholar
  11. 11.
    Fink A, Denzinger J, Aycock J (2007) Extracting npc behavior from computer games using computer vision and machine learning techniques. In: IEEE symposium on computational intelligence and games, IEEE Press, Hawaii, pp 24–31Google Scholar
  12. 12.
    Flanagan T, Thornton C, Denzinger J (2009) Testing harbour patrol and interception policies using particle-swarm-based learning of cooperative behavior. In Proceedings of the 2nd IEEE international conference on computational intelligence for security and defense applications (CISDA’09), IEEE Press, Piscataway, pp 215–222Google Scholar
  13. 13.
    Fredrickson S, Roberts S, Townsend N, Tarassenko L (1994) Speaker identification using networks of radial-basis functions. In: 7th European signal processing conference (EUSIPCO’94), Edinburgh, pp 812–815Google Scholar
  14. 14.
    Harris T (1993) Neural network in machine health monitoring. Professional engineering. July/August, pp 23–28Google Scholar
  15. 15.
    Hawkins S, He H, Williams G, Baxter R (2002) Outlier detection using replicator neural networks. In: 5th international conference on data warehousing and knowledge discovery (DaWaK’02), Provence, pp 170–180Google Scholar
  16. 16.
    Hebb D (2002) The organization of behavior: a neuropsychological theory. Lawrence Erlbaum Associates, Mahwah (reprint)Google Scholar
  17. 17.
    Ho TV, Rouat J (1997) Novelty detection based on relaxation time of a network of integrate-and-fire neurons. In: Proceedings of the 7th international conference on artificial neural networks, Losanna, Switzerland, pp 103–108Google Scholar
  18. 18.
    Hogg R, Ledolter J (1987) Engineering statistics. Mathematics and statistics. Macmillan, New YorkGoogle Scholar
  19. 19.
    I T, D A (2003) Sonic intelligence as a virtual therapeutic environment. CyberPsychol Behav 6(3):309–314Google Scholar
  20. 20.
    Irish D (2005) The game producer’s handbook. Course Technology Press, BostonGoogle Scholar
  21. 21.
    Jagota A (1991) Novelty detection on a very large number of memories stored in a hopfield-style network. In: Proceedings of the international joint conference on neural networks, vol 2. Seattle, p 905Google Scholar
  22. 22.
    Kohonen T (1989) Self-organization and associative memory. Springer, New YorkCrossRefGoogle Scholar
  23. 23.
    Laird JE (2000) An exploration into computer games and computer generated forces. In: 9th Conference on computer generated forces and behavioural representations, Orlando, pp 241–250Google Scholar
  24. 24.
    Li Y, Pont MJ, Jones NB (2002) Improving the performance of radial basis function classifiers in condition monitoring and fault diagnosis applications where ‘unknown’ faults may occur. Pattern Recognit Lett 23(5):569–577CrossRefzbMATHGoogle Scholar
  25. 25.
    Lindley C, Nacke L (2008) Boredom, immersion, flow—a pilot study investigating player experience. In: Proceedings of the IADIS gaming 2008: design for engaging experience and social interaction, IADIS, Amsterdam, pp 103–107Google Scholar
  26. 26.
    Lloyd SP (1982) Least squares quantization in pcm. IEEE Trans Inf Theory 28(2):129–137MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Macleod A (2005) Game design through self-play experiments. In: Proceedings of the 2005 ACM SIGCHI international conference on advances in computer entertainment technology (ACE’05), ACM, New York, pp 421–428Google Scholar
  28. 28.
    Mahlmann T, Drachen A, Togelius J, Canossa A, Yannakakis GN (2010) Predicting player behavior in tomb raider: Underworld. In: IEEE symposium on computational intelligence and games (CIG’10), Copenhagen, pp 178–185Google Scholar
  29. 29.
    Markou M, Singh S (2003) Novelty detection: a review—part 2: neural network based approaches. Signal Process 83(12):2499–2521CrossRefzbMATHGoogle Scholar
  30. 30.
    Marsland S (2003) Novelty detection in learning systems. Neural Comput Surv 3:157–195Google Scholar
  31. 31.
    Martinez D (1998) Neural tree density estimation for novelty detection. IEEE Trans Neural Netw 9(2):330–338CrossRefGoogle Scholar
  32. 32.
    Mikolajczyk K, Schmid C (2004) Scale & affine invariant interest point detectors. Int J Comput Vis 60(1):63–86CrossRefGoogle Scholar
  33. 33.
    Nantes A, Brown R, Maire F (2008) A framework for the semi-automatic testing of video games. In: Proceedings of the 4th conference on artificial intelligence and interactive digital entertainment (AIIDE’08), Stanford, pp 197–202Google Scholar
  34. 34.
    Nantes A, Brown R, Maire F (2010) Measuring visual consistency in 3d rendering systems. In: Proceedings of the 33rd Australian computer science conference (ACSC’10), Brisbane, pp 43–51Google Scholar
  35. 35.
    Pinelle D, Wong N, Stach T (2008) Heuristic evaluation for games: usability principles for video game design. In: Proceedings of the 26th annual SIGCHI conference on human factors in computing systems (CHI’08), Florence, pp 1453–1462Google Scholar
  36. 36.
    Rojas R (1996) Neural networks: a systematic introduction, 1st edn. Springer, New YorkCrossRefGoogle Scholar
  37. 37.
    Ruppert B (2011) New directions in virtual environments and gaming to address obesity and diabetes: Industry perspective. J Diabetes Sci Technol 5(2):277–282Google Scholar
  38. 38.
    Stellmach S, Nacke LE, Dachselt R, Lindley CA (2009) Trends and techniques in visual gaze analysis. In: Proceedings of the 5th conference on communication by gaze interaction (COGAIN’09), Lyngby, pp 89–93Google Scholar
  39. 39.
    Thurau C, Bauckhage C (2010) Analyzing the evolution of social groups in world of warcraft. In: Proceedings of the IEEE conference on computational intelligence and games (CIG’10), Copenhagen, pp 170–177Google Scholar
  40. 40.
    Togelius J, De Nardi R, Lucas SM (2007) Towards automatic personalised content creation in racing games. In: Proceedings of the IEEE symposium on computational intelligence and games (CIG’07), Honolulu, pp 252–259Google Scholar
  41. 41.
    Xiao G, Southey F, Holte RC, Wilkinson D (2005) Software testing by active learning for commercial games. In: Proceedings of the 20th national conference on artificial intelligence (AAAI’05), vol 2, Pittsburgh, pp 898–903Google Scholar
  42. 42.
    Yannakakis GN, Hallam J (2007) Towards optimizing entertainment in computer games. Appl Artif Intell 21:933–971CrossRefGoogle Scholar
  43. 43.
    Yannakakis GN, Togelius J (2011) Experience-driven procedural content generation. IEEE Trans Affect Comput 2:147–161CrossRefGoogle Scholar
  44. 44.
    Yee YH, Pattanaik SN, Greenberg DP (2001) Spatiotemporal sensitivity and visual attention for efficient rendering of dynamic environments. ACM Trans Graph (TOG) 20(1):39–65CrossRefGoogle Scholar
  45. 45.
    Yuan M, Farbiz F, Manders CM, Tang KY (2008) Robust hand tracking using a simple color classification technique. In: Proceedings of the 7th ACM SIGGRAPH international conference on virtual-reality continuum and its applications in industry (VRCAI’08), ACM, New York, pp 1–5Google Scholar

Copyright information

© Springer-Verlag London Limited 2012

Authors and Affiliations

  1. 1.Science and Engineering FacultyQueensland University of TechnologyBrisbaneAustralia

Personalised recommendations