Skip to main content
Log in

Using multiple sensors for reliable markerless identification through supervised learning

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

In many interaction models involving an active surface, there is a need to identify the specific object that performs an action. This is the case, for instance, when interactive contents are selected through differently shaped physical objects, or when a two-way communication is sought as the result of a touch event. When the technological facility is based on image processing, fiducial markers become the weapon of choice in order to associate a tracked object to its identity. Such approach, however, requires a clear and unoccluded view of the marker itself, which is not always the case. We came across this kind of hurdle during the design of a very large multi-touch interactive table. In fact, the thickness of the glass and the printed surface, which were required for our system, produced both blurring and occlusion at a level such that markers were completely unreadable. To overcome these limitations we propose an identification approach based on SVM that exploits the correlation between the optical features of the blob, as seen by the camera, and the data coming from active sensors available on the physical object that interacts with the table. This way, the recognition has been cast into a classification problem that can be solved through a standard machine learning framework. The resulting approach seems to be general enough to be applied in most of the problems where disambiguation can be achieved through the comparison of partial data coming from multiple simultaneous sensor readings. Finally, an extensive experimental section assesses the reliability of the identification.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Ardito, C., Costabile M.F., Lanzilotti, R.: Gameplay on a multitouch screen to foster learning about historical sites. In: AVI ’10, Proceedings of the International Conference on Advanced Visual Interfaces, pp. 75–78. ACM (2010)

  2. Barbieri, G., Celentano, A.: Multimedia technology: a companion to art visitors. In: Koukopoulos, D., Styliaras, G. (eds.) Handbook of Research on Technologies and Cultural Heritage: Applications and Environments, chapter 19, pp. 393–410. IGI Global (2011)

  3. Ben-David, A., Mandel, J.: Classification accuracy: Machine learning vs. explicit knowledge acquisition. Mach. Learn. 18, 109–114 (1995)

    Google Scholar 

  4. Bergamasco, F., Albarelli, A., Torsello, A.: Pi-tag: a fast image-space marker design based on projective invariants. Mach. Vis. Appl. (2012)

  5. Bergweiler, S., Deru, M., Porta, D.: Integrating a multitouch kiosk system with mobile devices and multimodal interaction. In: ACM International Conference on Interactive Tabletops and Surfaces, ITS ’10, pp. 245–246. ACM, New York (2010)

  6. Burges, C.J.C.: A tutorial on support vector machines for pattern recognition. Data Min. Knowl. Discov. 2(2), 121–167 (1998)

    Article  Google Scholar 

  7. Celentano, A., Orsini, R., Pittarello, F.: Towards an environment for designing and evaluating multimedia art guides. In: AVI ’10, Proceedings of the Working Conference on Advanced Visual Interfaces, pp. 93–96. ACM (2010)

  8. Cheshire, S., Baker, M.: Consistent overhead byte stuffing. IEEE/ACM Trans. Netw. 7(2), 159–172 (1999)

    Article  Google Scholar 

  9. Ciocca, G., Olivo, P., Schettini, R.: Browsing museum image collections on a multi-touch table. Inf. Syst. 37(2), 169–182 (2012)

    Article  Google Scholar 

  10. Cortes, C., Vapnik, V.: Support-vector networks. In: Machine Learning, pp. 273–297 (1995)

  11. Dachselt, R., Buchholz, R.: Natural throw and tilt interaction between mobile phones and distant displays. In Proceedings of the 27th International Conference Extended Abstracts on Human Factors in Computing Systems, CHI EA ’09, pp. 3253–3258. ACM, New York (2009)

  12. Dietz, P., Leigh, D.: Diamondtouch: a multi-user touch technology. In: UIST ’01, Proceedings of the 14th Annual ACM Symposium on User Interface Software and Technology, pp. 219–226. ACM (2001)

  13. Dippon, A., Klinker, G.. Kinecttouch: accuracy test for a very low-cost 2.5D multitouch tracking system. In: Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces, ITS ’11, pp. 49–52. ACM, New York (2011)

  14. Dohse, T., Still, J.D., Parkhurst, D.J.: Enhancing multi-user interaction with multi-touch tabletop displays using hand tracking. In: First International Conference on Advances in Computer–Human, Interaction, pp. 297–302 (2008)

  15. Döring, T., Shirazi, A.S., Schmidt, A.: Exploring gesture-based interaction techniques in multi-display environments with mobile phones and a multi-touch table. In: Proceedings of the PPD ’10, Workshop on Coupled Display Visual Interfaces, in Conjunction with AVI 2010, pp. 47–54 (2010)

  16. Echtler, F., Nestler, S., Dippon, A., Klinker, G.: Supporting casual interactions between board games on public tabletop displays and mobile devices. Personal Ubiquitous Comput. 13, 609–617 (2009)

    Google Scholar 

  17. Han, J.Y.: Low-cost multi-touch sensing through frustrated total internal reflection. In: UIST ’05, Proceedings of the 18th Annual ACM Symposium on User Interface Software and Technology, pp. 115–118. ACM (2005)

  18. He, Z.: Accelerometer based gesture recognition using fusion features and SVM. J. Softw. 6(6), 1042–1049 (2011)

    Google Scholar 

  19. Hesselmann, T., Henze, N., Boll, S.: Flashlight: optical communication between mobile phones and interactive tabletops. In: ACM International Conference on Interactive Tabletops and Surfaces, ITS ’10, pp. 135–138. ACM, New York (2010)

  20. Hornecker, E.: “I don’t understand it either, but it is cool”—visitor interactions with a multi-touch table in a museum. In: TABLETOP 2008. 3rd IEEE International Workshop on Horizontal Interactive Human Computer Systems, pp. 113–120 (2008)

  21. Jacucci, G., Morrison, A., Richard, G.T., Kleimola, J., Peltonen, P., Parisi, L., Laitinen, T.: Worlds of information: designing for engagement at a public multi-touch display. In: CHI ’10, Proceedings of the 28th International Conference on Human Factors in Computing Systems, pp. 2267–2276. ACM (2010)

  22. Kato, H., Billinghurst, M.: Marker tracking and HMD calibration for a video-based augmented reality conferencing system. In: Proceedings of the 2nd IEEE and ACM International Workshop on Augmented Reality. IEEE Computer Society, Washington, DC (1999)

  23. Ketabdar, H., Jahanbekam, A., Yuksel, K.A., Hirsch, T., Abolhassani, A.H.: Magimusic: using embedded compass (magnetic) sensor for touch-less gesture based interaction with digital music instruments in mobile devices. In: Proceedings of the Fifth International Conference on Tangible, Embedded, and Embodied Interaction, TEI ’11, pp. 241–244. ACM, New York (2011)

  24. Knecht K., Konig, R.: Augmented urban model: Bridging the gap between virtual and physical models to support urban design. In: CONVR 2011: Proceedings of the 11th International Conference on Construction Applications of Virtual Reality, pp. 142–152 (2011)

  25. Marquardt, N., Kiemer, J., Greenberg, S.: What caused that touch?: expressive interaction with a surface through fiduciary-tagged gloves. In: ACM International Conference on Interactive Tabletops and Surfaces, ITS ’10, pp. 139–142. ACM, New York (2010)

  26. Marzullo. K.A.: Maintaining the time in a distributed system: an example of a loosely-coupled distributed service. PhD thesis, Stanford (1984)

  27. Multimedia Information & Interaction Laboratory, Università Ca’ Foscari Venezia. Interactive Multimedia Art Guide Project. http://www.dais.unive.it/auce/artguide

  28. Boorsch, S.: Six Centuries of Master Prints. Cincinnati Art Museum, Cincinnati (1993)

    Google Scholar 

  29. Schöning, J., Rohs, M., Krüger, A.: Using mobile phones to spontaneously authenticate and interact with multi-touch surfaces. In Proceedings of PPD 2008, Workshop on Designing Multi-touch Interaction Techniques for Coupled Public and Private, Displays (2008)

  30. Sehgal A.K., Das, S., Noto, K., Saier, M., Elkan, C.: Identifying relevant data for a biological database: Handcrafted rules versus machine learning. IEEE/ACM Trans. Comput. Biol. Bioinform. 8(3), 851–857 (2011)

    Google Scholar 

  31. Shirazi, A.S., Winkler, C., Schmidt, A.: Flashlight interaction: a study on mobile phone interaction techniques with large displays. In Proceedings of the 11th International Conference on Human–Computer Interaction with Mobile Devices and Services, MobileHCI ’09, pp. 93:1–93:2. ACM, New York (2009)

  32. Sun, L., Zhang, D., Li, B., Guo, B., Li, S.: Activity recognition on an accelerometer embedded mobile phone with varying positions and orientations. In: Ubiquitous Intelligence and Computing, pp. 548–562 (2010)

  33. Villanueva, P.G., Gallud, J.A., Tesoriero, R.: WallShare: a collaborative multi-pointer system for portable devices. In: Proceedings of PPD 2010, Workshop on Designing Multi-touch Interaction Techniques for Coupled Public and Private Displays, pp. 31–34. ACM (2010)

  34. Wagner, D., Reitmayr, G., Mulloni, A., Drummond, T., Schmalstieg, D.: Real time detection and tracking for augmented reality on mobile phones. In: IEEE Transactions on Visualization and Computer Graphics, vol. 99 (2010)

  35. Wu, J., Pan, G., Zhang, D., Qi, G., Li, S.: Gesture recognition with a 3-d accelerometer. In: Proceedings of the 6th International Conference on Ubiquitous Intelligence and Computing, UIC ’09, pp. 25–38. Springer, Berlin (2009)

Download references

Acknowledgments

The Venice Imago Project is directed by Giuseppe Barbieri, Department of Philosophy and Cultural Heritage of Università Ca’ Foscari Venezia, who is gratefully acknowledged for the provided support.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Andrea Albarelli.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Albarelli, A., Bergamasco, F., Celentano, A. et al. Using multiple sensors for reliable markerless identification through supervised learning. Machine Vision and Applications 24, 1539–1554 (2013). https://doi.org/10.1007/s00138-013-0492-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-013-0492-2

Keywords

Navigation