Skip to main content
SpringerLink
Log in

Head-mounted mixed reality projection display for games production and entertainment

  • Original Article
  • Published:
Personal and Ubiquitous Computing Aims and scope Submit manuscript

Abstract

This research presents a mixed reality (MR) application that is designed to be usable during a motion capture shoot and supports actors with their task to perform. Through our application, we allow seeing and exploring a digital environment without occluding an actor’s field of vision. A prototype was built by combining a retroreflective screen covering surrounding walls and a headband consisting of a laser scanning projector with a smartphone. Built-in sensors of a smartphone provide navigation capabilities in the digital world. The presented system was demonstrated in an initially published paper. Here, we extend these research results with our advances and discuss the potential use of our prototype in gaming and entertainment applications. To explore this potential use case, we built a gaming application using our MR prototype and tested it with 45 participants. In these tests, we use head movements as rather unconventional game controls. According to the performed user tests and their feedback, our prototype shows a potential to be used for gaming applications as well. Therefore, our MR prototype could become of special interest because the prototype is lightweight, allows for freedom of movement and is a low-cost, stand-alone mobile system. Moreover, the prototype also allows for 3D vision by mounting additional hardware.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Akşit K, Kade D, Özcan O, Ürey H (2014) Head-worn mixed reality projection display application. In: Proceedings of ACE 2014, 11th advances in computer entertainment conference. ACM

  2. Akşit K, Eldes O, Viswanathan S, Freeman MO, Urey H (2012) Portable 3D laser projector using mixed polarization technique. J Disp Technol 8(10):582–589

    Article  Google Scholar 

  3. Bolas M, Krum DM (2010) Augmented reality applications and user interfaces using head-coupled near-axis personal projectors with novel retroreflective props and surfaces. In: Pervasive 2010 Ubiprojection workshop

  4. Cakmakci O, Rolland J (2006) Head-worn displays: a review. J Disp Technol 2(3):199–216

    Article  Google Scholar 

  5. DeMaster RD (1977) Low-profile raised retroreflective pavement marker. US Patent 4,035,059

  6. Firth N (2013) First wave of virtual reality games will let you live the dream. New Sci 218(2922):19–20

    Article  Google Scholar 

  7. Freeman M, Champion M, Madhavan S (2009) Scanned laser pico-projectors: seeing the big picture (with a small device). Opt Photonics News 20(5):28–34

    Article  Google Scholar 

  8. Freeman MO, Viswanathan SP, Lashmet D (2013) Mixed polarization imaging system for three-dimensional projection and corresponding methods. US Patent 20,130,038,837

  9. Goodman JW, Gustafson SC (1996) Introduction to fourier optics. Opt Eng 35(5):1513–1513

    Article  Google Scholar 

  10. Guillaumée M, Vahdati SP, Tremblay E, Mader A, Bernasconi G, Cadarso VJ, Grossenbacher J, Brugger J, Sprague R, Moser C (2014) Curved holographic combiner for color head worn display. J Disp Technol 10(6):444–449

    Article  Google Scholar 

  11. Hammond L (1924) Stereoscopic motion-eectuke device. US Patent 1,506,524

  12. Harrison C, Benko H, Wilson AD (2011) Omnitouch: wearable multitouch interaction everywhere. In: Proceedings of the 24th annual ACM symposium on user interface software and technology. ACM, pp 441–450

  13. Hua H, Gao C, Rolland JP (2002) Imaging properties of retro-reflective materials used in head-mounted projective displays (HMPDs). In: AeroSense 2002. International Society for Optics and Photonics, pp 194–201

  14. Illusions T (2014) Castar. http://technicalillusions.com/castar/

  15. Inami M, Kawakami N, Sekiguchi D, Yanagida Y, Maeda T, Tachi S (2000) Visuo-haptic display using head-mounted projector. In: Virtual reality, 2000. Proceedings. IEEE, pp 233–240. doi:10.1109/VR.2000.840503

  16. Jennett C, Cox AL, Cairns P, Dhoparee S, Epps A, Tijs T, Walton A (2008) Measuring and defining the experience of immersion in games. Int J Hum Comput Stud 66(9):641–661

    Article  Google Scholar 

  17. Kade D, Özcan O, Lindell R (2013) An immersive motion capture environment. In: Proceedings of the ICCGMAT 2013, international conference on computer games, multimedia and allied technology. World Academy of Science, Engineering and Technology, pp 500–506

  18. Kade D, Özcan O, Lindell R (2013) Towards Stanislavski-based principles for motion capture acting in animation and computer games. In: Proceedings of CONFIA 2013, international conference in illustration and animation. IPCA, pp 277–292

  19. Kanbara RTM, Yokoya N (2003) A wearable augmented reality system using positioning infrastructures and a pedometer. In: Proceedings of the seventh IEEE international symposium on wearable computers (ISWC03), vol 1530, pp 17–00

  20. Kim JS, Gracanin D, Quek F (2012) Sensor-fusion walking-in-place interaction technique using mobile devices. In: Virtual reality workshops (VR), 2012 IEEE. IEEE, pp 39–42

  21. Martins R, Shaoulov V, Ha Y, Rolland J (2007) A mobile head-worn projection display. Opt Express 15(22):14530–14538

    Article  Google Scholar 

  22. Mistry P, Maes P (2009) Sixthsense: a wearable gestural interface. In: ACM SIGGRAPH ASIA 2009 Sketches. ACM, p 11

  23. Mistry P, Maes P, Chang L (2009) WUW-wear Ur world: a wearable gestural interface. In: CHI’09 extended abstracts on Human factors in computing systems. ACM, pp 4111–4116

  24. Moeslund TB, Hilton A, Krüger V (2006) A survey of advances in vision-based human motion capture and analysis. Comput Vis Image Underst 104(2):90–126

    Article  Google Scholar 

  25. Olsson MI, Heinrich MJ, Kelly D, Lapetina J (2013) Wearable device with input and output structures. US Patent 20,130,044,042

  26. Pascu T, White M, Patoli Z (2013) Motion capture and activity tracking using Smartphone-driven body sensor networks. In: Innovative computing technology (INTECH), 2013 third international conference on. IEEE, pp 456–462

  27. Pictet L (1924) Device for projecting and viewing stereoscopic pictures. US Patent 1,503,766

  28. Rolland J, Thompson K (2011) See-through head worn displays for mobile augmented reality. In: Proceedings of the China national computer conference

  29. Rolland JP, Thompson KP, Urey H, Thomas M (2012) See-through head worn display (HWD) architectures. In: Handbook of visual display technology. Springer, Berlin, pp 2145–2170

  30. Scholl MS (1995) Ray trace through a corner-cube retroreflector with complex reflection coefficients. JOSA A 12(7):1589–1592

    Article  Google Scholar 

  31. Smits G, Kikinis D (2013) System and method for 3-D projection and enhancements for interactivity. https://www.google.com/patents/US20130300637. US Patent App. 13/877,652

  32. Snyder J (1975) Paraxial ray analysis of a cats-eye retroreflector. Appl Opt 14(8):1825–1828

    Article  Google Scholar 

  33. Sonoda T, Endo T, Kawakami N, Tachi S (2005) X’talvisor: full open type head-mounted projector. In: ACM SIGGRAPH 2005 emerging technologies. ACM, p 32

  34. Urey H, Chellappan KV, Erden E, Surman P (2011) State of the art in stereoscopic and autostereoscopic displays. Proc IEEE 99(4):540–555

    Article  Google Scholar 

  35. Vlasic D, Adelsberger R, Vannucci G, Barnwell J, Gross M, Matusik W, Popović J (2007) Practical motion capture in everyday surroundings. ACM Trans Graph 26(3). doi:10.1145/1276377.1276421

  36. Watch AF (1895) The anaglyph: a new method of producing the stereoscopic effect. J Frankl Inst 140(6):401–419

    Article  Google Scholar 

Download references

Acknowledgments

On behalf of Mälardalen University, we would like to thank KKS Stiftelsen for their support of the Industrial Research School, ITS-EASY. On behalf of Koç University, we would like to thank the Scientific and Technological Research Council of Turkey (TÜBITAK) with their support in Project No. 111E183.

Author information

Authors and Affiliations

  1. Mälardalen University, Högskoleplan 1, 721 23, Västerås, Sweden

    Daniel Kade

  2. Department of Electrical Engineering, Koç University, Rumeli Feneri Mh., 34450, Istanbul, Turkey

    Kaan Akşit & Hakan Ürey

  3. Arçelik Research Centre for Creative Industries, Koç University, Rumeli Feneri Mh., 34450, Istanbul, Turkey

    Oğuzhan Özcan

Authors
  1. Daniel Kade
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kaan Akşit
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hakan Ürey
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Oğuzhan Özcan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Daniel Kade.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kade, D., Akşit, K., Ürey, H. et al. Head-mounted mixed reality projection display for games production and entertainment. Pers Ubiquit Comput 19, 509–521 (2015). https://doi.org/10.1007/s00779-015-0847-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00779-015-0847-y

Keywords

Search

Navigation