Abstract
This chapter aims to provide the reader with basic knowledge about the experiment as a general method that can be applied towards serious games research and evaluation. It explains the main terms and rules of the experimental design, as well as points out the main risks and difficulties to avoid. The chapter also explains the differences in possible conclusions between true experiments, quasi-experiments, and correlational studies.
Keywords
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Adams, K.A., Lawrence, E.K.: Research Methods, Statistics and Applications. SAGE Publications, London (2014)
Adejumobi, B., Franck, N., Janzen, M.: Designing and testing a racing car serious game module. In: Ma, M., Oliveira, M.F., Baalsrud Hauge, J. (eds.) SGDA 2014. LNCS, vol. 8778, pp. 192–198. Springer, Heidelberg (2014)
Anderson, C.A., Dill, K.E.: Video games and aggressive thoughts, feelings, and behavior in the laboratory and in life. J. Person. Soc. Psychol. 78(4), 772–790 (2000)
Anguera, J.A., Boccanfuso, J., Rintoul, J.L., Al-Hashimi, O., Faraji, F., Janowich, J., Kong, E., Larraburo, Y., Rolle, C., Johnston, E., Gazzaley, A.: Video game training enhances cognitive control in older adults. Nature 501, 97–101 (2013)
Baek, Y., Ko, R., Marsh, T. (eds.): Trends and Applications of Serious Gaming and Social Media. Gaming Media and Social Effects. Springer, New York (2014)
Bidarra, R., Gambon, D., Kooij, R., Nagel, D., Schutjes, M., Tziouvara, I.: Gaming at the Dentist’s–serious game design for pain and discomfort distraction. In: Schouten, B., Fedtke, S., Bekker, T., Schijven, M., Gekker, A. (eds.) Games for Health 2013. Proceeding of the 3rd Conference on Gaming and Playful Interaction in Healthcare, pp. 207–215. Springer, Heidelberg (2013)
Boinodiris, P., Fingar, P.: Serious Games for Business: Using Gamification to Fully Engage Customers, Employees and Partners. Meghan-Kiffer Press, Tampa (2014)
Botella, C., Breton-Lópeza, J., Queroa, S., Bañosb, R.M., García-Palaciosa, A., Zaragozac, I., Alcanizc, M.: Treating cockroach phobia using a serious game on a mobile phone and augmented reality exposure: a single case study. Comput. Hum. Behav. 27(1), 217–227 (2011). Current Research Topics in Cognitive Load Theory. Third International Cognitive Load Theory Conference
Brom, C., Buchtová, M., Šisler, V., Děchtěrenko, F., Palme, R., Glenk, L.M.: Flow, social interaction anxiety and salivary cortisol responses in serious games: a quasi-experimntal study. Comput. Educ. 79, 69–100 (2014)
Bushman, B.J., Anderson, C.: Violent video games and hostile expectations: a test of the general aggression model. Pers. Soc. Psychol. Bull. 28(12), 1679–1686 (2002)
Campbell, D.T., Cook, D.T.: Quasy-Experimental Design and Analysis Issues for Field Setting. Rand McNally, Chicago (1979)
Casti, J.L.: Reality Rules, Picturing the World in Mathematics–The Fundamentals, vol. I. Wiley, New York (1997)
Choo, A., Tong, X., Gromala, D., Hollander, A.: Virtual reality and mobius floe: cognitive distraction as non-pharmacological analgesic for pain management. In: Schouten, B., Fedtke, S., Schijven, M., Vosmeer, M., Gekker, A. (eds.) Games for Health 2014 Proceeding of the 4th conference on gaming and playful interaction in healthcare, pp. 8–12. Springer Fachmedien Wiesbaden, Heidelberg (2014)
Adams, K.A.: Cram101 Textbooks Reviews, Just the facts101 Textbook Key Facts: Research Methods, Statistics, and Applications. Content Technologies (2014). http://cram101.com
De Freitas, S., Liarokapis, F.: Serious games: a new paradigm for education? In: Ma, M., Oikonomou, A., Jain, L.C. (eds.) Serious Games and Edutainment Applications, pp. 9–23. Springer, London (2011)
De Gloria, A., Bellotti, F., Berta, R.: Serious games for education and training. Int. J. Serious Games. 1(1). http://dx.doi.org/10.17083/ijsg.v1i1.11
Djaouti, D., Alvarez, J., Jessel, J., Rampnoux, O.: Origins of Serious Games. In: Ma, M., Oikonomou, A., Jain, L.C. (eds.) Serious Games and Edutainment Applications, pp. 25–43. Springer, London (2011)
Essential facts about the computer and video game industry. 2016 Sales, Demographic and Usage Data. ESA (2016). http://www.essentialfacts.theesa.com/Essential-Facts-2016.pdf
Ethical Principles of Psychologisys and Code of Conduct. APA (2010). http://www.apa.org/ethics/code/
Experiment. In: Merriam-Webster.com. Merriam-Webster. http://www.merriam-webster.com/dictionary/experiment
Ferguson, C., Kilburn, J.: Much ado about nothing: the misestimation and overinterpretation of violent video game effects in eastern and western nations: a comment on Anderson, et al. Psychol. Bull. 136(2), 174–178 (2010)
Foster, J., Barkus, E., Yavorsky, C.: Understanding and Using Advanced Statistics: A Practical Guide for Students. SAGE Puclications, London, Thousand Oaks (2006)
Girard, C., Ecalle, J., Magnan, A.: Serious games as new educational tools: how effective are they? a meta-analysis of recent studies. J. Comput. Assist. Learn. 29(3), 207–219 (2013)
Goodwin, C.J.: Research in Psychology Methods and Design. Wiley, New York (2010)
Hess, M., Wiemeyer, J., Hamacher, K., Goesele, M.: Serious games for solving protein sequence alignments - combining citizen science and gaming. In: Göbel, S., Wiemeyer, J. (eds.) GameDays 2014. LNCS, vol. 8395, pp. 175–185. Springer, Heidelberg (2014)
Hypothesis. In: Merriam-Webster.com. Merriam-Webster. http://www.merriam-webster.com/dictionary/hypothesis
Kantowitz, B.H., Roediger, H.L., Elmes, D.G.: Experimental Psychlogy, vol. III. Wadsworth, Belmont (2009)
Kaufman, S.B.: Sex differences in mental rotation and spatial visualization ability: can they be accounted for by differences in working memory capacity? Intelligence 35, 211–223 (2007)
King, D.L., Delfabbro, P., Griffiths, M.D.: The psychological study of video game players: methodological challenges and practical advice. Int. J. Mental Health Addict. 7(4), 555–562 (2009)
Kornilova, T.V.: Experimental Psychology. Jurajt, Moscow (2014). (in Russian)
Kuhn, T.S.: The Structure of Scientific Revolutions. International Encyclopedia of United Science, vol. 2. The University of Chicago Press, Chicago (1962)
Lim, C., Jung, H.: A study on the military serious game. Adv. Sci. Technol. Lett. 39, 73–77 (2013). Proceedings. SERSC 2013
Marchand, A., Hennig-Thurau, T.: Value creation in the video game industry: industry economics, consumer benefits, and research opportunities. J. Interact. Market. 27, 141–157 (2013)
Marsden, J.: The essential checklist for making an awesome video game, According to Futurlab (2013). http://indiegames.com/2013/07/the_essential_checklist_for_ma_1.html
Martin, D.W.: Doing Psychology Experiments. Thompson Higher Education, Belmont (2008)
Martins, T., Araújo, M., Carvalho, V., Soares, F., Torrão, L.: PhysioVinci – a first approach on a physical rehabilitation game. In: Ma, M., Oliveira, M.F., Baalsrud Hauge, J. (eds.) SGDA 2014. LNCS, vol. 8778, pp. 1–9. Springer, Heidelberg (2014)
Martin-SanJosé, J.-F., Juan, M.-C., Segui, I., Garcia-Garcia, I.: The effects of computer-based games and collaboration in large groups vs collaboration in pair or traditional methods. Comput. Educ. 87, 42–54 (2015)
Mayer, I., Zhou, Q., Keijser, X., Abspoel, L.: Gaming the future of the ocean: the marine spatial planning challenge 2050. In: Ma, M., Oliveira, M.F., Baalsrud Hauge, J. (eds.) SGDA 2014. LNCS, vol. 8778, pp. 150–162. Springer, Heidelberg (2014)
Omelina, L., Jansen, B., Bonnechère, B., Van Sint Jan, S., Cornelis, J.: Serious games for physical rehabilitation: designing highly configurable and adaptable games. In: Proceeding 9th International Conference on Disability, Virtual Reality and Associated Technologies Laval, France, 10–12 September 2012 (ICDVRAT 2012), pp. 195–201. ICDVRAT (2012)
Operationalization. Wikipedia. https://en.wikipedia.org/wiki/Operationalization
Popper, K.: Normal Science and its Dangers. In: Lakatos, I., Musgrave, A. (eds.) Criticism and the Growth of Knowledge, pp. 51–58. Cambridge University Press, Cambridge (1970)
Popper, K.: The Logic of Scientific Discovery. Routledge, London (2002)
Raulin, M.L., Graziano, A.M.: Quasi-experiments and correlational studies. In: Colman, A.M. (ed.) Companion Encyclopedia of Psychology, vol. 2, pp. 1124–1141. Routledge, London (1994)
Reliability. In: Merriam-Webster.com. Merriam-Webster. http://www.merriam-webster.com/dictionary/reliability
Ricciardi, F., De Paolis, L.T.: A Comprehensive review of serious games in health professions. Int. J. Comput. Games Technol. (2014). http://dx.doi.org/10.1155/2014/787968
Senevirathne, S.G., Kodagoda, M., Kadle, V., Haake, S.J., Senior, T., Heller, B.W.: Application of serious games to sports, health and exercise. In: Proceedings of the 6th SLIIT Research Symposium, Sri Lanka, vol. 4, pp. 6–9 (2011)
Shubert, T., Strobach, T.: Video game experience optimized executive control skills—on false positives and false negatives: reply to boot and simons. Acta Psychol. 141, 278–280 (2012)
Tomé, R.M., Pereira, J.M., Oliveira, M.: Using serious games for cognitive disabilities. In: Ma, M., Oliveira, M.F., Baalsrud Hauge, J. (eds.) SGDA 2014. LNCS, vol. 8778, pp. 34–47. Springer, Heidelberg (2014)
Tong, T., Chignell, M., Tierney, M.C., Masella, C.: A Serious game for clinical assessment of cognitive status: validation study. JMIR Serious Games 4(1), e7 (2016). doi:10.2196/games.5006
Validity. In: Merriam-Webster.com. Merriam-Webster. http://www.merriam-webster.com/dictionary/validity
Van der Spek, E.D.: Experiments in serious game design: a cognitive approach. Utrecht University Repository (Dissertation). Utrecht University, Utrecht (2011)
Van Muijden, J., Band, G.P.H., Hommel, B.: Online games training aging brains: limited transfer to cognitive control functions. Front. Hum. Neurosci. 6, 221 (2012). doi:10.3389/fnhum.2012.00221
Wouters, P., van Nimwegen, C., van Oostendorp, H., van der Spek, E.D.: A meta-analysis of the cognitive and motivational effects of serious games. J. Educ. Psychol. 105, 249 (2013). doi:10.1037/a0031311
Wrzesien, M., Alcañiz, M., Botella, C., Burkhardt, J.-M., Lopez, J.B., Ortega, A.R.: A pilot evaluation of a therapeutic game applied to small animal phobia treatment. In: Ma, M., Oliveira, M.F., Baalsrud Hauge, J. (eds.) SGDA 2014. LNCS, vol. 8778, pp. 10–20. Springer, Heidelberg (2014)
Yoo, K., Ahn, J., Lee, W.: A design of the stress relief game based on autonomic nervous system. In: Park, J.H., Jeong, Y.-S., Park, S.O., Chen, H.-C. (eds.) EMC 2012. LNEE, vol. 181, pp. 371–376. Springer, Heidelberg (2012)
Zakari, H.M., Ma, M., Simmons, D.: A review of serious games for children with autism spectrum disorders (ASD). In: Ma, M., Oliveira, M.F., Baalsrud Hauge, J. (eds.) SGDA 2014. LNCS, vol. 8778, pp. 93–106. Springer, Heidelberg (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Further Reading
Further Reading
For more experiments on serious games, see:
-
Van der Spek, E.D.: Experiments in Serious Game Design: a Cognitive Approach. Utrecht University Repository (Dissertation). Utrecht University, Utrecht (2011)
For deeper knowledge about experimental and other research types in social sciences and psychology, as well as for common statistical procedures see:
-
Kantowitz, B.H., Roediger, H.L.III, Elmes, D.G.: Experimental Psychology. Wadsworth, Belmont (2009)
-
Martin D.W.: Doing Psychology Experiments. Thompson Higher Education, Belmont (2008)
-
Goodwin C.J. Research in Psychology: Methods and Design. Wiley, Danvers (2010)
To learn about online experiments’ possibility, see:
-
Reips, U.-D.: Standards for Internet-based Experimenting. In: Experimental Psychology. 49(9), 243–256 (2002)
-
Reips, U.-D., Krantz, J.H.: Conducting True Experiments on the Web. In: Gosling, S., Johnson, J. (eds.) Advanced Methods for Conducting Online Behavioral Research, pp. 193–216. American Psychological Association, Washington DC (2010)
For basic knowledge about latent variables (variables, which can only be discovered through statistical procedures, very common in psychology and social sciences) see:
-
Bollen, K.A.: Latent Variables in Psychology and the Social Sciences. In: Annu. Rev. Psychol. 53, 605–634 (2002)
For advanced knowledge in statistical methods (multivariate analysis of variance, multiple regression, etc.), see:
-
Foster J., Barkus, E., Yavorsky, C.: Understanding and Using Advanced Statistics: A Practical Guide for Students. SAGE Puclications, London, Thousand Oaks, New Delhi (2006)
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this chapter
Cite this chapter
Bogacheva, N.V. (2016). The Experimental Method as an Evaluation Tool in Serious Games Research and Development. In: Dörner, R., Göbel, S., Kickmeier-Rust, M., Masuch, M., Zweig, K. (eds) Entertainment Computing and Serious Games. Lecture Notes in Computer Science(), vol 9970. Springer, Cham. https://doi.org/10.1007/978-3-319-46152-6_12
Download citation
DOI: https://doi.org/10.1007/978-3-319-46152-6_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-46151-9
Online ISBN: 978-3-319-46152-6
eBook Packages: Computer ScienceComputer Science (R0)