Trackable Interactive Multimodal Manipulatives: Towards a Tangible User Environment for the Blind

  • Muhanad S. Manshad
  • Enrico Pontelli
  • Shakir J. Manshad
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7383)


This paper presents the development of Trackable Interactive Multi-modal Manipulatives (TIMM). This system provides a multimodal tangible user environment (TUE), enabling people with visual impairments to create, modify and naturally interact with graphical representations on a multitouch surface. The system supports a novel notion of active position, proximity, stacking, and orientation tracking of manipulatives. The platform has been developed and it is undergoing formal evaluation.


Haptic Feedback Graphing Accessibility Blind and Visually Impaired Multitouch Multimodal TUI Diagrams Tangible User Environment (TUE) NASA TLX Subjective Workload Manipulatives Fiducials Markers 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
  2. 2.
  3. 3.
    Community Core Vision Information,
  4. 4.
  5. 5.
    Ishii, H., Ullmer, B.: Tangible bits: towards seamless interfaces between people, bits, and atoms. In: Proc. CHI 1997: Conference on Human Factors in Computing Systems, Atlanta, GA, USA (1997)Google Scholar
  6. 6.
    Kaltenbrunner, M.: reacTIVision and TUIO: a tangible tabletop toolkit. In: Proc. ITS 2009: Conference on Interactive Tabletops and Surfaces, Banff, Canada (2009)Google Scholar
  7. 7.
    Manshad, M.S., Pontelli, E., Manshad, S.J.: MICOO (multimodal interactive cubes for object orientation): a tangible user interface for the blind and visually impaired. In: Proc. ASSETS 2011: Conference on Accessibility and Computers, Dundee, Scotland (2011)Google Scholar
  8. 8.
    Manshad, M.S., Manshad, A.S.: Multimodal vision glove for touchscreens. In: Proc. ASSETS 2009: Conference on Accessibility and Computers, Nova Scotia, Canada (2009)Google Scholar
  9. 9.
    Mcgookin, D.K., Robertson, E., Brewster, S.A.: Clutching at straws: using tangible interaction to provide non-visual access to graphs. In: Proc. CHI 2010: Conference on Human Factors in Computing Systems, Atlanta, GA, USA (2010)Google Scholar
  10. 10.
  11. 11.
    Resnick, M., Martin, F., Berg, R., Borovoy, R., Colella, V., Kramer, K., Silverman, B.: Digital manipulatives: New toys to think with. In: CHI 1998: Proc. of the ACM Conference on Human Factors in Computing Systems, Los Angeles, CA, USA (1998)Google Scholar
  12. 12.
    Shaer, O., Hornecker, E.: Tangible User Interfaces: Past, Present, and Future Directions. In: Foundations and Trends in HCI vol. 3(1-2), pp. 1–137Google Scholar
  13. 13.
    Texas School for the Blind and Visually Impaired,
  14. 14.

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Muhanad S. Manshad
    • 1
  • Enrico Pontelli
    • 1
  • Shakir J. Manshad
    • 2
  1. 1.Computer Science DepartmentNew Mexico State University (NMSU)New MexicoUSA
  2. 2.Math Adoptive Technology Lab for Students with DisabilitiesNMSUNew MexicoUSA

Personalised recommendations