Toward automatic video-based whiteboard reading

  • Markus Wienecke
  • Gernot A. Fink
  • Gerhard Sagerer
Article

Abstract.

The increasing popularity of whiteboards as a popular tool in meeting rooms has been accompanied by a growing interest in making use of the whiteboard as a user interface for human-computer interaction. Therefore, systems based on electronic whiteboards have been developed in order to serve as meeting assistants for, e.g., collaborative working. However, as special pens and erasers are required, natural interaction is restricted. In order to render this communication method more natural, it was proposed to retain ordinary whiteboard and pens and to visually observe the writing process using a video camera [22, 25]. In this paper a prototype system for automatic video-based whiteboard reading is presented. The system is designed for recognizing unconstrained handwritten text and is further characterized by an incremental processing strategy in order to facilitate recognizing portions of text as soon as they have been written on the board. We will present the methods employed for extracting text regions, preprocessing, feature extraction, and statistical modeling and recognition. Evaluation results on a writer-independent unconstrained handwriting recognition task demonstrate the feasibility of the proposed approach.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Black MJ, Jepson AD (1998) A probabilistic framework for matching temporal trajectories: condensation-based recognition of gestures and expressions. In: Burkhardt H, Neumann B (eds) European conference on computer vision, Freiburg, Germany, pp 909-924Google Scholar
  2. 2.
    Bunke H, von Siebenthal T, Yamasaki T, Schenkel M (1999) Online handwriting data acquisition using a video camera. In: Proceedings of the international conference on document analysis and recognition, Bangalore, India, pp 573-576Google Scholar
  3. 3.
    Casey RG, Lecolinet E (1996) A survey of methods and strategies in character segmentation. IEEE Trans Pattern Anal Mach Intell 18(7):690-706Google Scholar
  4. 4.
    Chen SF, Goodman J (1999) An empirical study of smoothing techniques for language modeling. Comput Speech Lang 13:359-394Google Scholar
  5. 5.
    Clark P, Mirmehdi M (2002) Recognising text in real scenes. Int J Doc Anal Recog 4:243-257Google Scholar
  6. 6.
    Dolfing JGA, Haeb-Umbach R (1997) Signal representations for Hidden Markov Model based on-line handwriting recognition. In: Proceedings of the international conference on acoustics, speech, and signal processing, Munich, 4:3385-3388Google Scholar
  7. 7.
    Elrod S, Bruce R, Gold R, Goldberg D, Halasz F, Janssen W, Lee D, McCall K, Pedersen E, Pier K, Tang J, Welch B (1992) Liveboard: a large interactive display supporting group meetings, presentations and remote collaboration. In: Proceedings of ACM CHI’92, May 1992, pp 599-607Google Scholar
  8. 8.
    Fink GA (1999) Developing HMM-based recognizers with ESMERALDA. In: Matoušek V, Mautner P, Ocelíková J, Sojka P (eds) Lecture notes in artificial intelligence, vol 1692. Springer, Berlin Heidelberg New York, pp 229-234Google Scholar
  9. 9.
    Fink GA, Sagerer G (2000) Zeitsynchrone Suche mit n-Gramm-Modellen höherer Ordnung. In: Konvens 2000/Sprachkommunikation, ITG-Fachbericht 161, pp 145-150. VDE, BerlinGoogle Scholar
  10. 10.
    Fink GA, Schillo C, Kummert F, Sagerer G (1998) Incremental speech recognition for multimodal interfaces. In: Proceedings of the annual conference of the IEEE Industrial Electronics Society, Aachen, Germany, 4:2012-2017Google Scholar
  11. 11.
    Fink GA, Wienecke M, Sagerer G (2001) Video-based on-line handwriting recognition. In: Proceedings of the international conference on document analysis and recognition, pp 226-230Google Scholar
  12. 12.
    Johannson S, Leech GN, Goodluck H (1978) Manual of information to accompany the Lancaster-Oslo/Bergen corpus of British English, for use with digital computers. Department of English, University of Oslo, Oslo, NorwayGoogle Scholar
  13. 13.
    Li H, Doermann D, Kia O (2000) Automatic text detection and tracking in digital video. IEEE Trans Image Process 9(1):147-156Google Scholar
  14. 14.
    Marti U-V, Bunke H (1998) Erkennung handgeschriebener Wortsequenzen. In: Levi P, Ahlers R-J, May F, Schanz M (eds) Mustererkennung 98, 20. DAGM-Symposium Stuttgart, Informatik aktuell. Springer, Berlin Heidelberg New York, pp 263-270Google Scholar
  15. 15.
    Marti U-V, Bunke H (1999) A full English sentence database for off-line handwriting recognition. In: Proceedings of the international conference on document analysis and recognition, Bangalore, India, pp 705-708Google Scholar
  16. 16.
    Marti U-V, Bunke H (2000) Handwritten sentence recognition. In: Proceedings of the international conference on pattern recognition, Barcelona, 3:467-470Google Scholar
  17. 17.
    Marti U-V, Bunke H (2002) The IAM-database: an English sentence database for offline handwriting recognition. Int J Doc Anal Recog 5:39-46Google Scholar
  18. 18.
    Mirmehdi M, Clark P, Lam J (2001) Extracting low resolution text with an actvie camera for OCR. In: Sanchez J, Pla F (eds) Proceedings of the 9th Spanish symposium on pattern recognition and image processing, pp 43-48Google Scholar
  19. 19.
    Munich ME, Perona P (2002) Visual input for pen-based computers. IEEE Trans Pattern Anal Mach Intell 24(3):313-328Google Scholar
  20. 20.
    Otsu N (1979) A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern 9:62-66Google Scholar
  21. 21.
    Plamondon R, Srihari SN(2000) On-line and off-line handwriting recognition: a comprehensive survey. IEEE Trans Pattern Anal Mach Intell 22(1):63-83Google Scholar
  22. 22.
    Saund E (1999) Bringing the marks on a whiteboard to electronic life. In: Proceedings of the 2nd international workshop on cooperative buildings (CoBuild’99), Pittsburgh. Springer, Berlin Heidelberg New York, pp 69-78Google Scholar
  23. 23.
    Schomaker L, Segers E (1999) Finding features used in the human reading of cursive handwriting. Int J Doc Anal Recog 2:13-18Google Scholar
  24. 24.
    Senior AW, Robinson AJ (1998) An off-line cursive handwriting recognition system. IEEE Trans Pattern Anal Mach Intell 20(3):309-321Google Scholar
  25. 25.
    Stafford-Fraser Q, Robinson P (1996) Brightboard: a video-augmented environment. In: Proceedings of the conference on human factors and computing systems, Vancouver, BC, Canada, pp 134-141Google Scholar
  26. 26.
    Steinherz T, Rivlin E, Intrator N (1999) Offline cursive script word recognition - a survey. Int J Doc Anal Recog 2(2):90-110Google Scholar
  27. 27.
    Wienecke M, Fink GA, Sagerer G (2001) A handwriting recognition system based on visual input. In: 2nd international workshop on computer vision systems, Vancouver, BC, Canada, pp 63-72Google Scholar
  28. 28.
    Wienecke M, Fink GA, Sagerer G (2002) Experiments in unconstrained offline handwritten text recognition. In: Proceedings of the 8th international workshop on frontiers in handwriting recognition, Ontario, Canada, August 2002Google Scholar
  29. 29.
    Zhang Z, Tan C (2001) Restoration of images scanned from thick bound documents. In: Proceedings of the international conference on image processing, Thessaloniki, Greece, October 2001, pp 1074-1077Google Scholar

Copyright information

© Springer-Verlag Berlin/Heidelberg 2005

Authors and Affiliations

  • Markus Wienecke
    • 1
  • Gernot A. Fink
    • 1
  • Gerhard Sagerer
    • 1
  1. 1.Bielefeld UniversityFaculty of TechnologyBielefeldGermany

Personalised recommendations