Content Based Image and Video Retrieval Using Embedded Text

  • Chinmaya Misra
  • Shamik Sural
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3852)


Extraction of text from image and video is an important step in building efficient indexing and retrieval systems for multimedia databases. We adopt a hybrid approach for such text extraction by exploiting a number of characteristics of text blocks in color images and video frames. Our system detects both caption text as well as scene text of different font, size, color and intensity. We have developed an application for on-line extraction and recognition of texts from videos. Such texts are used for retrieval of video clips based on any given keyword. The application is available on the web for the readers to repeat our experiments and also to try text extraction and retrieval from their own videos.


Video Frame Optical Character Recognition Text Region Video Retrieval Text Block 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Agnihotri, L., Dimitrova, N.: Text Detection in Video Segments. In: Proc. of Workshop on Content Based Access to Image and Video Libraries, June 1999, pp. 109–113 (1999)Google Scholar
  2. 2.
    Hasan, Y.M.Y., Karam, L.J.: Morphological Text Extraction from Images. IEEE Transactions on Image Processing 9 (2000)Google Scholar
  3. 3.
    Jain, A.K., Yu, B.: Automatic Text Location in Images and Video Frames. Pattern Recognition 31(12), 2055–2076 (1998)CrossRefGoogle Scholar
  4. 4.
    Jung, K., Han, J.H.: Hybrid Approach to Efficient Text Extraction in Complex Color Images. Pattern Recognition Letters 25, 679–699 (2004)CrossRefGoogle Scholar
  5. 5.
    Kim, H.-K.: Efficient Automatic Text Location Method and Content-Based Indexing and Structuring of Video Database. Journal of Visual Communication and Image Representation 7(4), 336–344 (1996)CrossRefGoogle Scholar
  6. 6.
    Li, H., Doerman, D., Kia, O.: Automatic Text Detection and Tracking in Digital Video. IEEE Transactions on Image Processing 9, 147–156 (2000)CrossRefGoogle Scholar
  7. 7.
    Lienhart, R., Wernicke, A.: Localizing and Segmenting Text in Images and Videos. IEEE Transactions on Circuits and Systems for Video Technology 12(4), 256–268 (2002)CrossRefGoogle Scholar
  8. 8.
    Malobabic, J., O’Connor, N., Murphy, N., Marlow, S.: Automatic Detection and Extraction of Artificial Text in Video. In: Adaptive information cluster, center for digital video processing, Dublin city university (2002)Google Scholar
  9. 9.
    Nurgroho, A.S., Kuroyanagi, S., Iwata, A.: An Algorithm for Locating Characters in Color Image using Stroke Analysis Neural Network. In: Proc. of the 9th International Conference on Neural Information Processing (ICONIP 2002), November 18-22 (2002)Google Scholar
  10. 10.
    Sato, T., Kanade, T., Hughes, E., Smith, M.: Video OCR: Indexing Digital News Libraries by Recognition of Superimposed Captions. Multimedia Systems 7, 385–394 (1999)CrossRefGoogle Scholar
  11. 11.
    Shim, J.C., Dorai, C., Bolle, R.: Automatic Text Extraction from Video for Content-Based Annotation and Retrieval. In: Proc. of the 14th International Conference on Pattern Recognition, Brisbane, Australia, August 1998, vol. 1, pp. 618–620 (1998)Google Scholar
  12. 12.
    Wong, E.K., Chen, M.: A New Robust Algorithm for Video Extraction. Pattern Recognition 36(6), 1397–1406 (2003)zbMATHCrossRefGoogle Scholar
  13. 13.
    Zhang, D., Tseng, B.L., Lin, C.Y., Chang, S.F.: Accurate Overlay Text Extraction For Digital Video Analysis. Columbia University Advent Group Technical Report (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Chinmaya Misra
    • 1
  • Shamik Sural
    • 1
  1. 1.School of Information TechnologyIndian Institute of TechnologyKharagpurIndia

Personalised recommendations