Multimedia Tools and Applications

, Volume 76, Issue 4, pp 5367–5397 | Cite as

A social interactive whiteboard system using finger-tracking for mobile devices

  • Morten Bested
  • Andreas Harby Weisberg
  • Frederico Araújo Durão


Mobile devices have become a commodity in nowadays allowing people to get connected and perform several tasks for work. The usefulness of mobile is mobility leading people to solve their problems remotely, facilitating an everyday life. Based on these premises we propose a social interactive whiteboard system using finger-tracking for mobile devices. The application is a drawing tool, allowing users to create a virtual whiteboard and draw different kinds of lines, erase lines, zoom etc. from mobile devices that are featured with built-in projectors. Beyond the end-user application, this study investigates the problematic of tracking a finger using the build-in camera and how images are properly projected as a whiteboard from a mobile device. We present a thorough analysis of the various techniques that can be used to track a finger and depict our proposal to build the mobile-based interactive whiteboard application. The experimental evaluation analysed the performance, reliability and usability of the application taking into account valuable feedback for endusers.


Finger-tracking Whiteboard Detection Mobile-based Drawing tool 


  1. 1.
    Alvarado C, Davis R (2007) Resolving ambiguities to create a natural computer-based sketching environment. In: ACM SIGGRAPH 2007 courses, SIGGRAPH ’07. ACM, New YorkGoogle Scholar
  2. 2.
  3. 3.
    Arrington M (2010) Google nexus one: the techcrunch review. TechCrunch (TechCrunch) http://www.techcrunchcom/2010/01/05/google-nexus-one-the-techcrunch-review/ Retrieved January 6
  4. 4.
    Beardsley P, van Baar J, Raskar R, Forlines C (2005) Interaction using a handheld projector. IEEE Comput Graph Appl 25(1):39–43CrossRefGoogle Scholar
  5. 5.
  6. 6.
    Blasko G, Coriand F, Feiner S (2005) Exploring interaction with a simulated wrist-worn projection display. In: Ninth IEEE international symposium on wearable computers, 2005. Proceedings, pp 2–9Google Scholar
  7. 7.
    Boring S, Altendorfer M, Broll G, Hilliges O, Butz A (2007) Shoot & copy: phonecam-based information transfer from public displays onto mobile phones. In: Proceedings of the 4th international conference on mobile technology, applications, and systems and the 1st international symposium on computer human interaction in mobile technology, Mobility ’07. ACM, New York, pp 24–31Google Scholar
  8. 8.
    Boring S, Baur D, Butz A, Gustafson S, Baudisch P (2010) Touch projector: mobile interaction through video. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI ’10. ACM, New York, pp 2287–2296Google Scholar
  9. 9.
    Cao X, Forlines C, Balakrishnan R (2007) Multi-user interaction using handheld projectors. In: Proceedings of the 20th annual ACM symposium on user interface software and technology, UIST ’07. ACM, New York, pp 43–52Google Scholar
  10. 10.
    Crowley J L, Berard F, Coutaz J (1995) Finger tracking as an input device for augmented realityGoogle Scholar
  11. 11.
    Dao VN, Hosoi K, Sugimoto M (2007) A semi-automatic realtime calibration technique for a handheld projector. In: Proceedings of the 2007 ACM symposium on virtual reality software and technology, VRST ’07. ACM, New York, pp 43–46Google Scholar
  12. 12.
    Digitimes (2014) Foxlink developing micro projectors for handsets, say sources.
  13. 13.
  14. 14.
    Hosoi K, Dao VN, Mori A, Sugimoto M (2007) Cogame: manipulation using a handheld projector. ACM, New YorkGoogle Scholar
  15. 15.
    Jones MJ, Rehg JM (1999) Statistical color models with application to skin detection. In: IEEE Computer society conference on computer vision and pattern recognition, vol 1, p 280Google Scholar
  16. 16.
    Keaton T, Dominguez M S, Sayed H A (2002) Snap&tell: a multi-modal wearable computer interface for browsing the environment. In: ISWC 2002: Proceedings. sixth international symposium on wearable computers, pp 75–82Google Scholar
  17. 17.
    Kjeldskov J, Skov M B, Stage J (2004) Instant data analysis: conducting usability evaluations in a day. In: NordiCHI ’04: Proceedings of the third Nordic conference on Human-computer interaction. ACM, New York, pp 233–240Google Scholar
  18. 18.
    Kovac J, Peer P, Solina F (2003) Human skin color clustering for face detection. In: EUROCON 2003. Computer as a tool. The IEEE Region 8, vol 2, pp 144–148Google Scholar
  19. 19.
    LG (2010) The lg expo smartphone with pico projector.
  20. 20.
    Malik S (2003) Real-time hand tracking and finger tracking for interactionGoogle Scholar
  21. 21.
  22. 22.
    Mysliwiec T (1994) Fingermouse: A freehand computer pointing interface. In: Proc. of Intl conf. on automatic face and gesture recognition, pp 372–377Google Scholar
  23. 23.
    Oka K, Sato Y, Koike H (2002) Real-time tracking of multiple fingertips and gesture recognition for augmented desk interface systems. In: FGR ’02: Proceedings of the Fifth IEEE international conference on automatic face and gesture recognition. IEEE Computer Society, USA, p 429Google Scholar
  24. 24.
    Raskar R, van Baar J, Beardsley P, Willwacher T, Rao S, Forlines C (2003) ilamps: geometrically aware and self-configuring projectors. ACM Trans Graph 22(3):809–818CrossRefGoogle Scholar
  25. 25.
    Sato Y, Kobayashi Y, Koike H (2000) Fast tracking of hands and fingertips in infrared images for augmented desk interface. In: FG ’00: Proceedings of the Fourth IEEE international conference on automatic face and gesture recognition 2000. IEEE Computer Society, Washington, p 462Google Scholar
  26. 26.
    Schöning J, Rohs M, Kratz S, Löchtefeld M, Krüger A (2009) Map torchlight: A mobile augmented reality camera projector unit. In: CHI ’09 Extended abstracts on human factors in computing systems, CHI EA ’09. ACM, New York, pp 3841–3846Google Scholar
  27. 27.
    Song H, Guimbretiere F, Grossman T, Fitzmaurice G (2010) Mouselight: Bimanual interactions on digital paper using a pen and a spatially-aware mobile projector. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI ’10. ACM, New York, pp 2451–2460Google Scholar
  28. 28.
    Vezhnevets V, Sazonov V, Andreeva A (2003) A survey on pixel-based skin color detection techniques. In: IN PROC. GRAPHICON-2003, pp 85–92Google Scholar
  29. 29.
    von Hardenberg C, Bérard F (2001) Bare-hand human-computer interaction. In: PUI ’01: Proceedings of the 2001 workshop on perceptive user interfaces. ACM, New York, pp 1–8Google Scholar
  30. 30.
    Wilson AD (2005) Playanywhere: a compact interactive tabletop projection-vision system. In: Proceedings of the 18th annual ACM symposium on user interface software and technology, UIST ’05. ACM, New York, pp 83–92Google Scholar
  31. 31.
    Wu A, Shah M, da Vitoria Lobo N (2000a) A virtual 3d blackboard: 3d finger tracking using a single camera. In: FG ’00: Proceedings of the fourth IEEE international conference on automatic face and gesture recognition, vol 2000. IEEE Computer Society, Washington, DC, p 536Google Scholar
  32. 32.
    Wu Y, Shan Y, Zhangy Z, Shafer S (2000b) Visual panel: from an ordinary paper to a wireless and mobile input device. Microsoft ResearchGoogle Scholar
  33. 33.
    Yan Y, Ricci E, Liu G, Sebe N (2015) Egocentric daily activity recognition via multitask clustering. IEEE Trans Image Process 24(10):2984–2995MathSciNetCrossRefGoogle Scholar
  34. 34.
    Yan Y, Ricci E, Subramanian R, Liu G, Lanz O, Sebe N (2016) A multi-task learning framework for head pose estimation under target motion. IEEE Trans Pattern Anal Mach Intel 38(6):1070–1083CrossRefGoogle Scholar
  35. 35.
    Yang D D, Jin L W, Yin J X, Zhen L X, Huang J C (2005) An effective robust fingertip detection method for finger writing character recognition system. In: Proceedings of 2005 international conference on machine learning and cybernetics. IEEE Computer Society, pp 4991–4996Google Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Morten Bested
    • 2
  • Andreas Harby Weisberg
    • 2
  • Frederico Araújo Durão
    • 1
  1. 1.Department of Computer ScienceFederal University of BahiaSalvadorBrazil
  2. 2.Department of Computer Science and EngineeringAalborg UniversityAalborgDenmark

Personalised recommendations