A Paradigm of a Pervasive Multimodal Multimedia Computing System for the Visually-Impaired Users

  • Ali Awde
  • Manolo Dulva Hina
  • Chakib Tadj
  • Amar Ramdane-Cherif
  • Yacine Bellik
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3947)


Incorporating multimodality in a computing system makes computing more accessible to a wide range of users, including those with impairments. This work presents a paradigm of a multimodal multimedia computing system to make informatics accessible to visually-impaired users. The system’s infrastructure determines the suitable applications to be used. The user’s context and user data type are considered in determining the types of applications, media and modalities that are appropriate to use. The system design is pervasive, fault-tolerant and capable of self-adaptation under varying conditions (e.g. missing or defective components). It uses machine learning so that the system would behave in a pre-defined manner given a pre-conceived scenario. Incremental learning is adapted for added machine knowledge acquisition. A simulation of system’s behaviour, using a test case scenario, is presented in this paper. This work is our original contribution to an ongoing research to make informatics more accessible to handicapped users.


User Profile Pervasive Computing User Context Text Editor Condition Scenario 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Djenidi, H., et al.: Generic Multimedia Multimodal Agents Paradigms and their Dynamic Reconfiguration at the Architectural Level. EURASIP Journal on Applied Signal Processing 2004(11) (September 2004)Google Scholar
  2. 2.
    McCullough, M.: Digital Ground: Architecture, Pervasive Computing, and Environmental Knowing. MIT Press, Cambridge (2004)Google Scholar
  3. 3.
    Mitchell, T.M.: Machine Learning. McGraw-Hill, USA (1997)MATHGoogle Scholar
  4. 4.
    Giraud-Carrier, C.: A Note on the Utility of Incremental Learning. AI Communications 13(4), 215–223 (2000)MATHGoogle Scholar
  5. 5.
    Ross, D.A.: Cyber Crumbs for Successful Aging with Vision Loss. IEEE Pervasive Computing 3(2), 30–35 (2004)CrossRefGoogle Scholar
  6. 6.
    Royal Natl Institute of the Blind, Final Report of the TIDE, UK (1995), website:
  7. 7.
    Meijer, P.: The vOICe: Vision Technology for the Totally Blind, (2005), website:
  8. 8.
    Archambault, D.: BrailleSurf: An HTML Browser for Visually Handicapped People. In: CSUN Conf., Los Angeles, USA (1999)Google Scholar
  9. 9.
    King, A., et al.: WebbIE, A Web Browser for Visually Impaired People. In: 2nd CWUAAT Workshop, Cambridge, UK (2004)Google Scholar
  10. 10.
    Moço, V., Archambault, D.: Automatic Conversions of Mathematical Braille: A Survey of Main Difficulties in Different Languages. In: ICCHP Conference, Paris, France (2004)Google Scholar
  11. 11.
    Ferreira, H., Freitas, D.: Enhancing the Accessibility of Mathematics for Blind People: the AudioMath Project. In: ICCHP Conference, Paris, France (2004)Google Scholar
  12. 12.
    Wooldridge, M.: An Introduction to Multi-agent Systems. Wiley, Chichester (2001)Google Scholar
  13. 13.
    Bourbakis, N.G., et al.: An Intelligent Assistant for Navigation of Visually Impaired People. In: 2nd IEEE Intl. Symposium on Bioinformatics and Bioengineering Conference (2001)Google Scholar
  14. 14.
    Edwards, A.: MATHS, Mathematical Access for TecHnology and Science, UK (1997),
  15. 15.
    Bellik, Y.: Interfaces multimodales: concepts, modèles et architectures, Ph.D. Thesis, Université d’Orsay, Paris (1995)Google Scholar
  16. 16.
    Antoniol, G., et al.: A Distributed Architecture for Dynamic Analyses on User-profile Data. In: 8th European Conference on Software Maintenance and Reengineering (2004)Google Scholar
  17. 17.
    Bougant, F., Delmond, F., Pageot-Millet, C.: The User Profile for the Virtual Home Environment. IEEE Communications Magazine 41(1), 93–98 (2003)CrossRefGoogle Scholar
  18. 18.
    Okamoto, M.: Design and Application of Learning Conversational Agents. Ph.D. Thesis, Department of Social Informatics, Kyoto University (2003)Google Scholar
  19. 19.
    Hina, M.D., et al.: A Ubiquitous Context-sensitive Multimodal Multimedia Computing and Its Machine-Learning Assisted Reconfiguration at the Architectural Level. In: Workshop on Multimedia Information Proc. and Retrieval, 7th IEEE Intl. Symp. on Multimedia (2005)Google Scholar
  20. 20.
  21. 21.
  22. 22.
    Herbordt, W., et al.: Noise-Robust Hands-Free Speech Recognition on PDA’s Using Microphone Array Technology. Autumn Meeting of the Acous, Society of Japan (2005)Google Scholar
  23. 23.
    Han, T., et al.: Structure Analysis for Dynamic Software Architecture. In: 6th Intl. Conf. on Software Eng., Artificial Int., Net. and Parallel/Dist. Comp. (May 2005)Google Scholar
  24. 24.
    Horn, P.: Autonomic Computing: IBM’s Perspective on the State of Information Technology, IBM Research (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Ali Awde
    • 1
  • Manolo Dulva Hina
    • 1
    • 2
  • Chakib Tadj
    • 1
  • Amar Ramdane-Cherif
    • 2
  • Yacine Bellik
    • 3
  1. 1.LATIS LaboratoryUniversité du Québec, École de technologie superieureMontréalCanada
  2. 2.PRISM Laboratory CRNSUniversité de Versailles-Saint-Quentin-en-YvelinesVersaillesFrance
  3. 3.LIMSI-CRNSUniversité de Paris-SudOrsayFrance

Personalised recommendations