Artificial Intelligence Review

, Volume 26, Issue 4, pp 325–363 | Cite as

Bandwidth determined transmoding through fuzzy logic in mobile intelligent multimedia presentation systems

Article

Abstract

Mobile intelligent multimedia presentation systems are subject to various resource constraints including mobile network characteristics, mobile device capabilities and user preferences. Those presentation systems which incorporate remote multimedia content accessed across HTTP (Hyper Text Transfer Protocol) or RTP (Real Time Transfer Protocol) protocols are particularly reliant on the capabilities of the connecting mobile network (i.e. minimum, average and maximum bandwidth) and in particular on the real-time constraints (i.e. currently available bandwidth, packet loss, bit error rate, latency) which prevail during actual content transmission. One approach to address this is to scale content, thus reducing its datarate requirement, although this technique is inherently limited by the lowest acceptable quality of that media element. Alternatively, content can be converted from one modality to another with a lower resource requirement. TeleMorph, a cross-modality adaptation control platform is detailed here. Initially a brief introduction to Intelligent Multimedia and to Mobile Intelligent Multimedia is given, and key systems discussed. The main premise of TeleMorph is that cross-modality adaptations in mobile presentation systems must be controlled in a manner which gives primary consideration to bandwidth fluctuations as well as the constraints listed above. The current prototype of TeleMorph, which uses a fuzzy inference system to control cross-modality adaptations between video and audio, is described. Particular focus is given to the fuzzy inputs, fuzzy control rules and fuzzy outputs which have been utilised in decision making. TeleTuras, a tourist information application which has been implemented as a testbed for TeleMorph, gives promising evaluation results based on multimedia and bandwidth specific test scenarios. TeleMorph is related to other approaches in the area of Mobile Intelligent Multimedia Presentation Systems. TeleMorph differs from other approaches in that it focuses specifically on the challenges posed by controlling bandwidth determined cross-modality adaptations in a mobile network environment. Future work on TeleMorph’s output presentation composition will incorporate images and text also, thus allowing for extended adaptation between video, audio, images and text, as well as multimodal combinations of these media elements.

Keywords

TeleMorph TeleTuras Mobile multimedia transmoding Cross-modality adaptation Mobile intelligent multimedia presentation systems Multimodal output Fuzzy logic Mobile network constraints Bandwidth awareness 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Almeida L, Amdal I, Beires N, Boualem M, Boves L, den Os E, Filoche P, Gomes R, Knudsen JE, Kvale K, Rugelbak J, Tallec C, Warakagoda N (2002) Implementing and evaluating a multimodal and multilingual tourist guide. International CLASS Workshop on Natural, Intelligent and Effective Interaction in Multimodal Dialogue Systems Copenhagen, Denmark, 28–29 JuneGoogle Scholar
  2. Babuska R (1993) Fuzzy toolbox for MATLAB. In: Proceedings of the 2nd IMACS International Symposium on Mathematical and Intelligent Models in System Simulation. University Libre de Bruxelles, BrusselsGoogle Scholar
  3. Bernsen NO (1994) Modality theory in support of multimodal interface design. In: Proceedings of the spring symposium on intelligent multi-media multi-modal systems. Stanford University, USA, pp 37–44Google Scholar
  4. Bers J, Miller S, Makhoul J (1998) Designing conversational interfaces with multimodal interaction. In: Proceedings of the DARPA broadcast news transcription and understanding workshop. Lansdowne Conference Resort, Lansdowne, Virginia, pp 88–92Google Scholar
  5. Boll S and Klas W (2001). ZYX-A multimedia document model for reuse and adaptation of multimedia content. IEEE Transa Knowledge Data Eng, 13(3): 361–382 CrossRefGoogle Scholar
  6. Boll S, Klas W, Wandel J (1999) A cross-media adaptation strategy for multimedia presentations. In: Proceedings of ACM Multimedia. ACM, Orlando, pp 37–46Google Scholar
  7. Chalmers D, Sloman M (1999a) QoS and context awareness for mobile computing. In: Proceedings of international symposium on handheld and ubiquitous computing (HUC99). Springer-verlag, Karlsruhe, pp 380–382Google Scholar
  8. Chalmers D and Sloman M (1999b). A survey of quality of service in mobile computing environments. IEEE Commun Surv 2(2): 2–10 CrossRefGoogle Scholar
  9. Clark JM and Paivio A (1991). Dual coding theory and education. Educ Psychol Rev, 3(3): 149–210 CrossRefGoogle Scholar
  10. Crumpet (2006) http://www.ist-crumpet.org Site visited 28 July, 2006
  11. Curran K and Annesley S (2005). Transcoding media for bandwidth constrained mobile devices. Int J Netw Manage 15(2): 75–88 CrossRefGoogle Scholar
  12. DeMoyer R, Mitchell EE (1999) Use of the MATLAB graphical user interface development environment for some control system applications. In: Proceedings of IEEE Computer Society Conference on Frontiers in Education. Stripes Publishing L.L.C, San Juan, pp 7–11Google Scholar
  13. Elting C, Zwickel J, Malaka R (2002) Device-dependant modality selection for user-interfaces-an empirical study. In: Proceedings of Sixth International Conference on Intelligent User Interfaces. ACM, San Francisco, pp 55–62Google Scholar
  14. EML (2006) http://www.eml.org/english/Research/Memory Site visited 28 July, 2006
  15. Fox A, Brewer EA (1996) Reducing WWW latency and bandwidth requirements by real-time distillation. In: Proceedings of the Fifth International World Wide Web Conference. Elsevier, Paris, France, pp 1445–1456Google Scholar
  16. Fox A, Gribble SD, Brewer EA, Amir E (1996) Adapting to network and client variability via on-demand dynamic distillation. In: Proceedings 7th International Conference on Architectural Support for Programming Languages and Operating Systems. ACM, Cambridge, pp 160–170Google Scholar
  17. Fox A, Gribble SD, Chawathe Y and Brewer EA (1998). Adapting to network and client variation using infrastructural proxies: lessons and perspectives. IEEE Pers Communi 5(4): 10–19 CrossRefGoogle Scholar
  18. Hildebrand A (2000) EMBASSI: electronic multimedia and service assistance. In: Proceedings IMC’2000. Rostock-Warnemunde, Germany, November, pp 50–59Google Scholar
  19. ISO/IEC 21000–7 2004, Information Technology—Multimedia Framework—Part 7: Digital Item Adaptation, ISO/IECGoogle Scholar
  20. Klas W, Christian G, Reinhard F (1999) Cardio-OP: gallery of cardiac surgery. In: Proceedings of the IEEE International Conference on Multimedia Computing and Systems, Florence, Italy, Volume 2, June 07–11, pp 1092–1095Google Scholar
  21. Kurkowski S, Camp T, Mushell N, Colagrosso M (2005) A visualization and analysis tool for NS-2 wireless simulations: iNSpect. In: Proceedings of the 13th IEEE International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems. IEEE Computing Society, Atlanta, pp 503–506Google Scholar
  22. Malaka R (2000) Artificial intelligence goes mobile—artificial intelligence in mobile systems 2000, Workshop in conjunction with ECAI 2000. Berlin, Germany, August 22, Workshop Notes, pp 5–6Google Scholar
  23. Malaka R (2001) Multi-modal interaction in private environments. International Seminar on Coordination and Fusion in MultiModal Interaction, Schloss Dagstuhl International Conference and Research Center for Computer Science. Wadern, Saarland, Germany 29 October–2 November.(http://www.dfki.de/~wahlster/Dagstuhl_Multi_Modality/Dagstuhl-2001.pdf Site visited 28 July, 2006)
  24. Malaka R, Zipf A (2000) DEEP MAP—challenging IT research in the framework of a tourist information system. In: Fesenmaier D, Klein S, Buhalis D (eds) Information and communication technologies in tourism 2000, Proceedings of ENTER 2000, 7th International Congress on Tourism and Communications Technologies in Tourism, Barcelona, Spain. Springer Computer Science, Wien, pp 15–27Google Scholar
  25. Malaka R, Porzel R, Zipf A (2000) Integration of smart components for building your personal mobile guide. In: Malaka R (ed) Artificial intelligence in mobile systems—AIMS2000. Workshop in conjunction with ECAI 2000, Berlin, Germany, August 22, Workshop Notes, pp 22–26Google Scholar
  26. Oviatt SL, Cohen PR, Wu L, Vergo J, Duncan E, Suhm B, Bers J, Holzman T, Winograd T, Landay J, Larson J, Ferro D (2000) Designing the user interface for multimodal speech and gesture applications: state-of-the-art systems and research directions. Hum comput Interact 15:263–322. (to be reprinted in Carroll J (ed) Human–computer interaction in the new millennium. Addison-Wesley Press, Boston, 2001)Google Scholar
  27. Pastra K (2006) Beyond multimedia integration: corpora and annotations for cross-media decision mechanisms. In: Proceedings of the 5th Language Resources and Evaluation Conference (LREC)Google Scholar
  28. Pedersen JS, Larsen SR (2003) A pilot study in modality shifting due to changing network conditions. M.Sc. Thesis, Center for Person Communication. Aalborg University, DenmarkGoogle Scholar
  29. Pereira F, Smith JR and Vetro A (2005). Introduction to the special section on MPEG-21. IEEE Trans Multimedia 7(3): 397–399 CrossRefGoogle Scholar
  30. Qiu Q, Zhang D, Ma J (2004) GPRS network simulation model in NS-2. 2004 Joint Conference of the 10th Asia-Pacific Conference on Communications and the 5th International Symposium on Multi-Dimensional Mobile Communications Proceedings (APCC/MDMC). Institute of Electrical and Electronics Engineers Inc., New York, NY 10016–5997, United States, Beijing, China, pp 700–704Google Scholar
  31. Smith JR (2001). Universal multimedia access Multimedia Systems and Applications. International Society for Optical Engineering, Boston, 21–32 Google Scholar
  32. Solon AJ, Mc Kevitt P, Curran KJ (2004a) A bandwidth determined mobile multimodal system. 1st International Conference on E-Business and Telecommunication Networks (ICETE). INSTICC Press, pp 24–31Google Scholar
  33. Solon AJ, Mc Kevitt P, Curran KJ (2004b) Design of a tourist driven bandwidth determined MultiModal Mobile Presentation System. IEEE International Conference on Mobility Aware Technologies and Applications (MATA ’04), Florianopolis, Brazil, October 20–22. Lecture Notes in Computer Science (LNCS) Vol. 3284/2004, ISBN 3–540–23423–3, Heidelberg, Germany: Springer-Verlag GmbH, February, pp. 331–339Google Scholar
  34. Solon AJ, Mc Kevitt P, Curran KJ (2004c) Mobile multimodal presentation. ACM Multimedia 2004—Proceedings of the 12th ACM International Conference on Multimedia. Association for Computing Machinery, New York, pp 440–443Google Scholar
  35. Solon AJ, Mc Kevitt P, Curran KJ (2004d) TeleMorph: bandwidth determined Mobile MultiModal Presentation. In: Information Technology and Tourism (IT&T), vol 7, no 1, (ISSN 1098–3058), Cognizant Publishers, USA, June, pp 33–47Google Scholar
  36. Sun H, Vetro A, Asai K (2003) Resource adaptation based on MPEG-21 usage environment descriptions. International Symposium on Circuits and Systems (ISCAS), IEEE, Bangkok, Thailand, pp 536–539Google Scholar
  37. Thang TC, Jung YJ, Ro YM (2003) Modality conversion in content adaptation for universal multimedia access. International Conference on Imaging Science, Systems and Technology (CISST). CSREA Press, Las Vegas, NV, USA, pp 434–440Google Scholar
  38. Thang TC, Jung YJ, Lee J and Ro YM (2005a). Effective adaptation of multimedia documents with modality conversion. Signal Process Image Communi 20(5): 413–434 CrossRefGoogle Scholar
  39. Thang TC, Jung YJ and Ro YM (2005b). Modality conversion for QoS management in universal multimedia access. IEE Proc Vis Image Signal Process 152(3): 374–384 CrossRefGoogle Scholar
  40. Uden L, Campion R (2000) Integrating modality theory in educational multimedia design. Proceedings of the 17th Annual Conference of the Australasian Soceity for Computers in Learning in Tertiary Education (ASCILITE 2000), Learning to Choose and Choosing to Learn. Coff’s Harbour, Australia, December 11–14Google Scholar
  41. Vetro A and Timmerer C (2005). Digital item adaptation: overview of standardization and research activities. IEEE Transa Multimedia 7(3): 418–426 CrossRefGoogle Scholar
  42. Vetro A, Timmerer C, Devillers S (2005) Digital item adaptation. In: Burnett IS, Pereira F, Van de Walle R, Koenen R (eds) The MPEG-21 Book. WileyGoogle Scholar
  43. Wahlster W (2003) SmartKom: symmetric multimodality in an adaptive and reusable dialogue shell. In: Krahl R, Günther D (eds) Proceedings of the Human Computer Interaction Status Conference 2003. DLR, Berlin, June, pp 47–62Google Scholar
  44. Wirag S (1999) Scheduling of adaptive multimedia documents. In: Proceedings of IEEE International Conference on Multimedia Computing and Systems (ICMCS). IEEE Computing Society, Florence, Italy, pp 307–311Google Scholar
  45. Zadeh LA (1965). Fuzzy sets. Inf Control, 8: 338–353 MATHCrossRefMathSciNetGoogle Scholar
  46. Zipf A, Malaka R (2001) Developing Location Based Services (LBS) for tourism—the service providers view. ENTER 2001, In: Proceedings of the 8th International Congress on Tourism and Communications Technologies in Tourism, Montreal, Canada, April, pp 24–27Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2007

Authors and Affiliations

  • Anthony J. Solon
    • 1
  • Kevin Curran
    • 1
  • Paul Mc Kevitt
    • 1
  1. 1.Intelligent Multimedia Research Group, School of Computing & Intelligent Systems, Faculty of Computing & EngineeringUniversity of UlsterMagee, DerryNorthern Ireland, UK

Personalised recommendations