Finding Objects Faster in Dense Environments Using a Projection Augmented Robotic Arm

  • Hind Gacem
  • Gilles Bailly
  • James Eagan
  • Eric Lecolinet
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9298)


Locating an object in an unfamiliar and dense physical environment, such as a control room, supermarket, or warehouse, can be challenging. In this paper, we present the Projection-Augmented Arm (PAA), a motorized robotic arm augmented with a pico-projector to help users to localize targets in such environments. The arm moves and displays a projected spotlight on the target. We present the results of a study that shows that the PAA helps users to more quickly locate target objects in a dense environment. We further study the influence of the visibility of the projected spotlight while moving versus that of the physical movement of the projection arm on user performance and search strategy, finding that (1) information about the orientation of the arm has a stronger impact on performance than moving spotlight projected on the search space; (2) the orientation of the arm is useful (24 % improvement) and especially when the target is behind the user (26 % improvement); and (3) users’ strategies relied mainly on the arm when it is visible.


Guidance techniques Augmented arm Steerable pico-projector 



This work is supported by the French ANR/Investissement d’Avenir “Cluster Connexion” and by the ÉQUIPEX DigiScope ANR-10-EQPX-0026.


  1. 1.
  2. 2.
    Butz, A., Schneider, M., Spassova, M.: SearchLight – a lightweight search function for pervasive environments. In: Ferscha, A., Mattern, F. (eds.) PERVASIVE 2004. LNCS, vol. 3001, pp. 351–356. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  3. 3.
    Cauchard, J.R., Fraser, M., Han, T., Subramanian, S.: Steerable projection: exploring alignment in interactive mobile displays. Pers. Ubiquitous Comput. 16, 27–37 (2012)CrossRefGoogle Scholar
  4. 4.
    Cosgun, A., Sisbot, E.A., Christensen, H.I.: Evaluation of rotational and directional vibration patterns on a tactile belt for guiding visually impaired people. In: 2014 IEEE Haptics Symposium (HAPTICS), pp. 367−370 (2014)Google Scholar
  5. 5.
    Dramas, F., Oriola, B., Katz, B.G., Thorpe, S.J., Jouffrais, C.: Designing an assistive device for the blind based on object localization and augmented auditory reality. In: Proceedings of the 10th International ACM SIGACCESS Conference on Computers and accessibility, pp. 263−264. ACM (2008)Google Scholar
  6. 6.
    Ehnes, J., Hirose, M.: Projected Reality - Enhancing Projected Augmentations by Dynamically Choosing the Best Among Several Projection Systems. In: IEEE Conference on Virtual Reality, pp. 283−284 (2006)Google Scholar
  7. 7.
    Ehnes, J., Hirota, K., Hirose, M.: Projected augmentation - augmented reality using rotatable video projectors. In: IEEE & ACM ISMAR 2004, pp. 26−35 (2004)Google Scholar
  8. 8.
    Van Erp, J.B.F., Van Veen, H.A.H.C., Jansen, C., Dobbins, T.: Waypoint navigation with a vibrotactile waist belt. ACM Trans. Appl. Percept. 2(2), ACM (2005)Google Scholar
  9. 9.
    Gröhn, M., Lokki, T., Takala, T.: Comparison of auditory, visual, and audiovisual navigation in a 3D space. ACM Trans. Appl. Percept. 2(4), 564–570 (2005)CrossRefzbMATHGoogle Scholar
  10. 10.
    Harada, S., Takagi, H., Asakawa, C.: On the audio representation of radial direction. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2779−2788. ACM (2011)Google Scholar
  11. 11.
    Harrison, C., Benko, H., Wilson, A.D.: OmniTouch: wearable multitouch interaction everywhere. In: UIST 2011, pp. 441–450. ACM (2011)Google Scholar
  12. 12.
    Henderson, S.J., Feiner, S.: Evaluating the benefits of augmented reality for task localization in maintenance of an armored personnel carrier turret. In: IEEE ISMAR 2009, pp. 135–144 (2009)Google Scholar
  13. 13.
    Henry, P., Krainin, M., Herbst, E., Ren, X., Fox, D.: RGB-D maping:using kinect-style depth cameras for dense 3D modeling of indoorenvironments. IEEE Int. J. Rob. Res. (IJRR) 31(5), 647–663 (2012)CrossRefGoogle Scholar
  14. 14.
    Ishii, K., Yamamoto, Y., Imai, M., Nakadai, K.: A navigation system using ultrasonic directional speaker with rotating base. In: Smith, M.J., Salvendy, G. (eds.) HCII 2007. LNCS, vol. 4558, pp. 526–535. Springer, Heidelberg (2007)Google Scholar
  15. 15.
    Jones, B., Sodhi, R., Murdock, M., Mehra, R., Benko, H., Wilson, A., Shapira, L.: RoomAlive: magical experiences enabled by scalable, adaptive projector-camera units. In: UIST 2014, pp. 637–644. ACM (2014)Google Scholar
  16. 16.
    Kratz, S., Rohs, M., Reitberger, F., Moldenhauer, J.: Attjector: an attention-following wearable projector. In: Kinect Workshop at Pervasive (2012)Google Scholar
  17. 17.
    Lee, J.C., Dietz, P.H., Maynes-Aminzade, D., Raskar, R., Hudson, S.E.: Automatic projector calibration with embedded light sensors. In: UIST 2004, pp. 123–126. ACM (2004)Google Scholar
  18. 18.
    Lehtinen, V., Oulasvirta, A., Salovaara, A., Nurmi, P.: Dynamic tactile guidance for visual search tasks. In: UIST 2012, pp. 445–452. ACM (2012)Google Scholar
  19. 19.
    Li, M., et al.: ProFi: design and evaluation of a product finder in a supermarket scenario. In: UbiComp 2013, pp. 977–984. Adjunct (2013)Google Scholar
  20. 20.
    Li, M., Arning, K., Sack, O., Park, J., Kim, M.-H., Ziefle, M., Kobbelt, L.: Evaluation of a mobile projector-based indoor navigation interface. Interact. Comput. 26(6), 595–613 (2013)CrossRefGoogle Scholar
  21. 21.
    Lindeman, R.W., Sibert, J.L., Mendez-Mendez, E., Patil, S., Phifer, D.: Effectiveness of directional vibrotactile cuing on a building-clearing task. In: CHI 2005, pp. 271–280. ACM (2005)Google Scholar
  22. 22.
    Ngo, M., Spence, C.: Auditory, tactile, and multisensory cues facilitate search for dynamic visual stimuli. Attention Percept. Psychophys. 72, 1654–1665 (2010)CrossRefGoogle Scholar
  23. 23.
    Ogata, K., Seya, Y., Watanabe, K., Ifukube, T.: Effects of visual cues on the complicated search task. pp. 478–485. ACM (2012)Google Scholar
  24. 24.
    Ota, S., Takegawa, Y., Terada, T., Tsukamoto, M.: A method for wearable projector selection that considers the viewability of projected images. Comput. Entertain. 8, 17:1–17:16 (2010)CrossRefGoogle Scholar
  25. 25.
    Pinhanez, C.: The everywhere displays projector: a device to create. In: UbiComp 2001, pp. 315–331. ACM (2001)Google Scholar
  26. 26.
    Pinhanez, C., Kjeldsen, R., Levas, A., Pingali, G., Podlaseck, M., Sukaviriya, N.: Applications of steerable projector-camera systems. In: ICCV Workshop on Projector-Camera Systems, IEEE (2003)Google Scholar
  27. 27.
    Pulkki, V.: Virtual sound source positioning using vector base amplitude panning. J. Audio Eng. Soc. 45(6), 456–466 (1997)Google Scholar
  28. 28.
    Sukthankar, R., Stockton, R.G., Mullin, M.D.: Smarter presentations: exploiting homography in camera-projector systems. In: ICCV 2001, (1) pp. 247–253 (2001)Google Scholar
  29. 29.
    Umlauf, E.J., Piringer, H., Reitmayr, G., Schmalstieg, D.: ARLib: the augmented library. In: The First IEEE International Workshop on Augmented Reality Toolkit, p. 2 (2002)Google Scholar
  30. 30.
    Willis, K.D.D., Poupyrev, I., Hudson, S.E., Mahler, M.: SideBySide: Ad hoc multi-user interaction with handheld projectors. In: UIST 2011, pp. 431–440 (2011)Google Scholar
  31. 31.
    Wilson, A., Benko, H., Izadi, S., Hilliges, O.: Steerable augmented reality with the beamatron. In: UIST 2012, pp. 413–422. ACM (2012)Google Scholar
  32. 32.
    Yamano, S., Hamajo, T., Takahashi, S., Higuchi, K.: EyeSound: single-modal mobile navigation using directionally annotated music. In: Augmented Human, p. 1. ACM (2012)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2015

Authors and Affiliations

  • Hind Gacem
    • 1
    • 2
  • Gilles Bailly
    • 1
    • 2
  • James Eagan
    • 1
    • 2
  • Eric Lecolinet
    • 1
    • 2
  1. 1.Télécom ParisTechParisFrance
  2. 2.CNRS LTCI UMR 5141ParisFrance

Personalised recommendations