GPU Accelerated Left/Right Hand-Segmentation in First Person Vision

  • Alejandro BetancourtEmail author
  • Lucio Marcenaro
  • Emilia Barakova
  • Matthias Rauterberg
  • Carlo Regazzoni
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9913)


Wearable cameras allow users to record their daily activities from a user-centered (First Person Vision) perspective. Due to their favourable location, they frequently capture the hands of the user, and may thus represent a promising user-machine interaction tool for different applications. Existent First Person Vision, methods understand the hands as a background/foreground segmentation problem that ignores two important issues: (i) Each pixel is sequentially classified creating a long processing queue, (ii) Hands are not a single “skin-like” moving element but a pair of interacting entities (left-right hand). This paper proposes a GPU-accelerated implementation of a left right-hand segmentation algorithm. The GPU implementation exploits the nature of the pixel-by-pixel classification strategy. The left-right identification is carried out by following a competitive likelihood test based the position and the angle of the segmented pixels.


Egovision Hand-segmentation GPU Hand-detection Wearable cameras 



This work was partially supported by the Erasmus Mundus joint Doctorate in Interactive and Cognitive Environments, which is funded by the EACEA, Agency of the European Commission under EMJD ICE.


  1. 1.
    Baraldi, L., Paci, F., Serra, G., Benini, L., Cucchiara, R.: Gesture Recognition using wearable vision sensors to enhance visitors’ museum experiences. IEEE Sens. J. 15(5), 1 (2015). CrossRefGoogle Scholar
  2. 2.
    Betancourt, A., Lopez, M., Regazzoni, C., Rauterberg, M.: A sequential classifier for hand detection in the framework of egocentric vision. In: Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 600–605. IEEE, Columbus, June 2014.
  3. 3.
    Betancourt, A., Morerio, P., Barakova, E.I., Marcenaro, L., Rauterberg, M., Regazzoni, C.S.: A dynamic approach and a new dataset for hand-detection in first person vision. In: Azzopardi, G., Petkov, N., Yamagiwa, S. (eds.) CAIP 2015. LNCS, vol. 9256, pp. 274–287. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-23192-1_23 CrossRefGoogle Scholar
  4. 4.
    Betancourt, A., Morerio, P., Marcenaro, L., Barakova, E., Rauterberg, M., Regazzoni, C.: Towards a unified framework for hand-based methods in first person vision. In: IEEE International Conference on Multimedia and Expo (Workshops). IEEE, Turin (2015)Google Scholar
  5. 5.
    Betancourt, A., Morerio, P., Marcenaro, L., Rauterberg, M., Regazzoni, C.: Filtering SVM frame-by-frame binary classification in a detection framework. In: International Conference on Image Processing. IEEE, Quebec (2015)Google Scholar
  6. 6.
    Betancourt, A., Morerio, P., Regazzoni, C., Rauterberg, M.: The evolution of first person vision methods: a survey. IEEE Trans. Circuits Syst. Video Technol. 25(5), 744–760 (2015). CrossRefGoogle Scholar
  7. 7.
    Betancourt, A., Díaz-Rodríguez, N., Barakova, E., Marcenaro, L., Rauterberg, M., Regazzoni, C.: Unsupervised understanding of location and illumination changes in egocentric videos (2016). arXiv preprint:
  8. 8.
    Betancourt, A., Morerio, P., Marcenaro, L., Barakova, E., Rauterberg, M., Regazzoni, C.: Left/Right Hand Segmentation in Egocentric Videos. ArXiv e-prints Under Revi (2016)Google Scholar
  9. 9.
    Buso, V., Benois-Pineau, J., Domenger, J.P.: Geometrical cues in visual saliency models for active object recognition in egocentric videos. In: Proceedings of the 1st International Workshop on Perception Inspired Video Processing - PIVP 2014, pp. 9–14. ACM Press, New York (2014).
  10. 10.
    Coudert, F., Butin, D., Le Métayer, D.: Body-worn cameras for police accountability: opportunities and risks. Comput. Law Secur. Rev. 31(6), 749–762 (2015). CrossRefGoogle Scholar
  11. 11.
    Duncan, R.: A survey of parallel computer architectures. Computer 23(2), 5–16 (1990)CrossRefGoogle Scholar
  12. 12.
    Fathi, A., Ren, X., Rehg, J.M.: Learning to recognize objects in egocentric activities. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 3281–3288. IEEE, Providence, June 2011.
  13. 13.
    Feng, S., Caire, R., Cortazar, B., Turan, M., Wong, A., Ozcan, A.: Immunochromatographic diagnostic test analysis using google glass. ACS Nano 8(3), 3069–3079 (2014). CrossRefGoogle Scholar
  14. 14.
    Harvey, M., Langheinrich, M., Ward, G.: Remembering through lifelogging: a survey of human memory augmentation. Pervasive Mobile Comput. 27, 14–26 (2016). CrossRefGoogle Scholar
  15. 15.
    Hastie, T., Tibshirani, R.J., Friedman, J.: The Elements of Statistical Learning, vol. 1, 10th edn. Springer, Heidelberg (2009). CrossRefzbMATHGoogle Scholar
  16. 16.
    Jones, M.J., Rehg, J.M.: Statistical color models with application to skin detection. Int. J. Comput. Vis. 46, 81–96 (2002). IEEE Computer Society, Fort Collins, COCrossRefzbMATHGoogle Scholar
  17. 17.
    Li, C., Kitani, K.: Model recommendation with virtual probes for egocentric hand detection. In: 2013 IEEE International Conference on Computer Vision, pp. 2624–2631. IEEE Computer Society, Sydney (2013).
  18. 18.
    Li, C., Kitani, K.: Pixel-level hand detection in ego-centric videos. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition, pp. 3570–3577. IEEE, June 2013.
  19. 19.
    MathSoft: Classification and regression trees. Guide to Statistics 1, 369–401, February 1999Google Scholar
  20. 20.
    Matsuo, K., Yamada, K., Ueno, S., Naito, S.: An attention-based activity recognition for egocentric video. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 565–570. IEEE, June 2014.
  21. 21.
    Morerio, P., Marcenaro, L., Regazzoni, C.: Hand detection in first person vision. In: Fusion, p. 6. University of Genoa, Istanbul (2013).
  22. 22.
    Singhai, S., Satsangi, C.: Hand segmentation for hand gesture recognition. In: Workshop on Interactive Multimedia on Mobile & Portable Devices, vol. 1, pp. 48–52. ACM Press, New York (2014).
  23. 23.
    Zhu, X., Jia, X., Wong, K.-Y.K.: Pixel-level hand detection with shape-aware structured forests. In: Cremers, D., Reid, I., Saito, H., Yang, M.-H. (eds.) ACCV 2014, Part IV. LNCS, vol. 9006, pp. 64–78. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-16817-3_5 Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Alejandro Betancourt
    • 1
    • 2
    Email author
  • Lucio Marcenaro
    • 1
  • Emilia Barakova
    • 2
  • Matthias Rauterberg
    • 2
  • Carlo Regazzoni
    • 1
  1. 1.Department of Engineering (DITEN)University of GenovaGenovaItaly
  2. 2.Department of Industrial DesignEindhoven University of TechnologyEindhovenNetherlands

Personalised recommendations