Exploring a Novel Inexpensive Tangible Interface for Non-visual Math and Science

  • R. Stanton
  • E. Pontelli
  • Z. Toups
  • M. Manshad
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10896)


Tangible interaction enables physical manipulation of digital data, making it ideal to support visually impaired students. Visually impaired students are frequently integrated in mainstream courses, collaborating with sighted peers and instructors. This paper describes a tangible block localization and tracking method, using small inexpensive sensor packages that detect color placed on an interaction surface—i.e., a standard flat-screen display. The system recursively subdivides the display surface into regions of distinct colors that the sensor package can distinguish. Once located, the sensor package can be tracked by moving the color pattern underneath to follow it, re-expanding the pattern as needed to capture the sensor package if it moves too fast. The novel tracking infrastructure supports novel approaches to teach a number of science and math concepts to visually impaired students.


Assistive technologies Tangible interfaces 


  1. 1.
    Avago Technologies. APDS-9960 Digital Proximity, Ambient Light, RGB and Gesture Sensor. Data Sheet AV02-4191EN (2013)Google Scholar
  2. 2.
    Cuypers, T., Francken, Y., Vanaken, C., Van Reeth, F., Bekaert, P.: Smartphone localization on interactive surfaces using the built-in camera. In: Proceedings of the ProCams, vol. 2, pp. 3–4. Citeseer (2009)Google Scholar
  3. 3.
    Ishii, H., Ullmer, B.: Tangible bits: towards seamless interfaces between people, bits and atoms. In: Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, CHI 1997, pp. 234–241. ACM, New York (1997)Google Scholar
  4. 4.
    Leigh, S., Schoessler, P., Heibeck, F., Maes, P., Ishii, H.: THAW: tangible interaction with see-through augmentation for smartphones on computer screens. In: Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction, TEI 2015, pp. 89–96. ACM, New York (2015)Google Scholar
  5. 5.
    Samet, H.: The quadtree and related hierarchical data structures. ACM Comput. Surv. 16(2), 187–260 (1984)MathSciNetCrossRefGoogle Scholar
  6. 6.
    SciencesPo MediaLab. I Want Hue: Colors for Data Scientists (2017). Accessed 20 Jan 2017

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Computer ScienceNew Mexico State UniversityLas CrucesUSA

Personalised recommendations