Multimedia Data Mining Framework for Raw Video Sequences

  • JungHwan Oh
  • JeongKyu Lee
  • Sanjaykumar Kote
  • Babitha Bandi
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2797)


We extend our previous work [1] of the general framework for video data mining to further address the issue such as how to mine video data, in other words, how to extract previously unknown knowledge and detect interesting patterns. In our previous work, we have developed how to segment the incoming raw video stream into meaningful pieces, and how to extract and represent some feature (i.e., motion) for characterizing the segmented pieces. We extend this work as follows. To extract motions, we use an accumulation of quantized pixel differences among all frames in a video segment. As a result, the accumulated motions of segment are represented as a two dimensional matrix. We can get very accurate amount of motion in a segment using this matrix. Further, we develop how to capture the location of motions occurring in a segment using the same matrix generated for the calculation of the amount. We study how to cluster those segmented pieces using the features (the amount and the location of motions) we extract by the matrix above. We investigate an algorithm to find whether a segment has normal or abnormal events by clustering and modeling normal events, which occur mostly. In addition to deciding normal or abnormal, the algorithm computes Degree of Abnormality of a segment, which represents to what extent a segment is distant to the existing segments in relation with normal events. Our experimental studies indicate that the proposed techniques are promising.


Multimedia Data Mining Video Segmentation Motion Extraction Video Data Clustering 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Oh, J., Bandi, B.: Multimedia data mining framework for raw video sequences. In: Proc. of ACM Third International Workshop on Multimedia. Data Mining (MDM/KDD2002), Edmonton, Alberta, Canada (July 2002)Google Scholar
  2. 2.
    Stolorz, P., Nakamura, H., Mesrobian, E., Muntz, R., Shek, E., Santos, J., Yi, J., Ng, K., Chien, S., Mechoso, C., Farrara, J.: Fast spatio-temporal data mining of large geophysical datasets. In: Proc. of Int’l Conf. on KDD, pp. 300–305 (1995)Google Scholar
  3. 3.
    Fayyad, U., Djorgovski, S., Weir, N.: Automating the analysis and cataloging of sky surveys. Advances in Knowledge Discovery with Data Mining, 471–493 (1996)Google Scholar
  4. 4.
    Li, Z.-N., Zaïane, O.R., Tauber, Z.: Illumination invariance and object model in content-based image and video retrieval. Journal of Visual Communication and Image Representation (1998)Google Scholar
  5. 5.
    Zaïane, O.R., Han, J., Li, Z.-N., Chee, S., Chiang, J.: Multimediaminer: A system prototype for multimedia data mining. In: Proc. of the 1998 ACM SIGMOD Conf., pp. 581–583 (1998)Google Scholar
  6. 6.
    Zaïane, O.R., Han, J., Li, Z.-N., Hou, J.: Mining multimedia data. In: Proc. of the CASCON 1998: Meeting of Minds, pp. 27–32 (1998)Google Scholar
  7. 7.
    Koperski, K., Adikary, J., Han, J.: Spatial data mining: Progress and challenges. In: Proc. of SIGMOD Workshop on Research Issues on Data Mining and Knowledge. Discovery (DMKD 1996), Montreal, Canada, pp. 27–32 (1996)Google Scholar
  8. 8.
    Koperski, K., Han, J.: Mining knowledge in geographical data. Communication of ACM (1998)Google Scholar
  9. 9.
    Wijesekera, D., Barbara, D.: Mining cinematic knowledge: Work in progress. In: Proc. of International Workshop on Multimedia Data Mining (MDM/KDD 2000), Boston, MA, August 2000, pp. 98–103 (2000)Google Scholar
  10. 10.
    Shearer, K., Dorai, C., Venkatesh, S.: Incorporating domain knowledge with video and voice data analysis in news broadcasts. In: Proc. of International Workshop on Multimedia Data Mining (MDM/KDD 2000), Boston, MA, August 2000, pp. 46–53 (2000)Google Scholar
  11. 11.
    Kulesh, V., Petrushin, V., Sethi, I.: The perseus project: Creating personalized multimedia news portal. In: Proc. of International Workshop on Multimedia Data Mining (MDM/KDD 2001), San Francisco, CA, August 2001, pp. 31–37 (2001)Google Scholar
  12. 12.
    Chen, Y., Gao, W., Wang, Z., Miao, J., Jiang, D.: Mining audio/visual database for speech driven face animation. In: Proc. of International Conference on Systems, Man and Cybernetics, pp. 2638–2643 (2001)Google Scholar
  13. 13.
    Singh, P.K., Majumdar, A.K.: Semantic content-based retrieval in a video database. In: Proc. of International Workshop on Multimedia Data Mining (MDM/KDD 2001), San Francisco, CA, August 2001, pp. 50–57 (2001)Google Scholar
  14. 14.
    Chen, S., Shyu, M., Zhang, C., Strickrott, J.: Multimedia data mining for traffic video sequences. In: Proc. of International Workshop on Multimedia Data Mining (MDM/KDD 2001), San Francisco, CA, August 2001, pp. 78–86 (2001)Google Scholar
  15. 15.
    Cucchiara, R., Piccardi, M., Mello, P.: Image analysis and rule-based reasoning for a traffic monitoring system. IEEE Transactions on Intelligent Transportation Systems 1(2), 119–130 (2000)CrossRefGoogle Scholar
  16. 16.
    Dailey, D., Cathey, F., Pumrin, S.: An algorithm to estimate mean traffic speed using uncalibrated cameras. IEEE Transactions on Intelligent Transportation Systems 1(2), 98–107 (2000)CrossRefGoogle Scholar
  17. 17.
    Krumm, J., Harris, S., Meyers, B., Brumitt, B., Hale, M., Shafer, S.: Multi-camera multi-person tacking for easyliving. In: Proc. of 3rd IEEE International Workshop on Visual Surveillance, pp. 3–10 (2000)Google Scholar
  18. 18.
    Shafer, S., Krumm, J., Meyers, B., Brumitt, B., Czerwinski, M., Robbins, D.: The new easyliving project at microsoft research. In: Proc. of DARPA/NIST Workshop on Smart Spaces, pp. 127–130 (1998)Google Scholar
  19. 19.
    Coen, M.: The future of human-computer interaction on how I learned to stop worrying and love my intelligent room. IEEE Intelligent Systems 14(2), 8–10 (1999)CrossRefGoogle Scholar
  20. 20.
    Pavlidis, I., Morellas, V., Tsiamyrtzis, P., Harp, S.: Urban surveillance systems: From the laboratory to the commercial world. Proceedings of The IEEE 89(10), 1478–1497 (2001)CrossRefGoogle Scholar
  21. 21.
    Kamijo, S., Matsushita, Y., Ikeuchi, K., Sakauchi, M.: Traffic monitoring and accident detection at intersections. In: IEEE Intenational Conference on Intelligent Tansportation Systems, Tokyo, Japan, pp. 703–708 (1999)Google Scholar
  22. 22.
    Huang, T., Koller, D., Malik, J., Ogasawara, G.: Automatic symbolic traffic scene analysis using belief networks. In: Proc. of AAAI, 12th National Conference on Artificial Intelligence (AAAI 1994), Seattle, WA, pp. 966–972 (1994)Google Scholar
  23. 23.
    Koller, D., Weber, J., Malik, J.: Robust multiple car tracking with occlusion reasoning. In: Proc. of European Conference on Computer Vision, Stockholm, Sweden, pp. 189–196 (1994)Google Scholar
  24. 24.
    Oh, J., Sankuratri, P.: Automatic distinction of camera and objects motions in video sequences. In: Proc. of IEEE International Conference on Multimedia and Expo (ICME 2002), Lausanne, Switzerland (August 2002) (to appear)Google Scholar
  25. 25.
    Foresti, G.L., Roli, F.: Learning and classification of suspicious events for advanced visual-based surveillance. In: Multimedia Video-based Surveillance Systems: Requirements, Issues and Solutions, pp. 84–93. Kluwer, Norwell (2000)Google Scholar
  26. 26.
    Sacchi, C., Regazzoni, C.S.: A distributed surveillance system for detection of abandoned objects in unmanned railway environments. IEEE Transaction on Veh. Technol. 49, 1355–1367 (2000)Google Scholar
  27. 27.
    Freer, J.A., Beggs, B.J., Fernandez-Canque, H.L., Chevrier, F., Goryashko, A.: Automatic intruder detection incorporating intelligent scene monitoring with video surveillance. In: Proc. Eur. Conf. Security and Detection, ECOS 1997, pp. 109–113 (1997)Google Scholar
  28. 28.
    Foresti, G., Pani, B.: Monitoring motorway infrastructures for detection of dangerous events. In: Proc. of IEEE Int. Conf. Image Analysis and Processing, pp. 1144–1147 (1999)Google Scholar
  29. 29.
    Zhao, L., Qi, W., Wang, Y., Yang, S., Zhang, H.: Video shot grouping using bestfirst model merging. In: Proc. of SPIE conf. on Storage and Retrieval for Media Databases 2001, San Jose, CA, January 2001, pp. 262–269 (2001)Google Scholar
  30. 30.
    Han, S., Kweon, I.: Shot detection combining bayesian and structural information. In: Proc. of SPIE conf. on Storage and Retrieval for Media Databases 2001, San Jose, CA, January 2001, pp. 509–516 (2001)Google Scholar
  31. 31.
    Oh, J., Hua, K.A.: An efficient and cost-effective technique for browsing and indexing large video databases. In: Proc. of 2000 ACM SIGMOD Intl. Conf. on Management of Data, Dallas, TX, May 2000, pp. 415–426 (2000)Google Scholar
  32. 32.
    Oh, J., Hua, K.A., Liang, N.: A content-based scene change detection and classification technique using background tracking. In: SPIE Conf. on Multimedia Computing and Networking 2000, San Jose, CA, January 2000, pp. 254–265 (2000)Google Scholar
  33. 33.
    Hua, K.A., Oh, J.: Detecting video shot boundaries up to 16 times faster. In: The 8th ACM International Multimedia Conference (ACM Multimedia 2000), LA, CA, October 2000, pp. 385–387 (2000)Google Scholar
  34. 34.
    Haritaoglu, I., Harwood, D., Davis, L.S.: W4 - who, where, when, what: A real-time system for detecting and tracking people. In: IEEE Third Intenational Conference on Face and Gesture Recognition, Nara, Japan, pp. 222–227 (1998)Google Scholar
  35. 35.
    Oh, J., Chowdary, T.: An efficient technique for measuring of various motions in video sequences. In: Proc. of The 2002 International Conference on Imaging Science, System, and technology (CISST 2002), Las Vegas, NV (June 2002)Google Scholar
  36. 36.
    Ngo, C.W., Pong, T.C., Zhang, H.J.: On clustering and retrieval of video shots. In: Proc. of ACM Multimedia 2001, Ottawa, Canada, October 2001, pp. 51–60 (2001)Google Scholar
  37. 37.
    Rousseeuw, P.J., Leroy, A.M.: Robust Regression and Outlier Detection. John Wiley and Sons, Chichester (1987)zbMATHCrossRefGoogle Scholar
  38. 38.
    Jain, A.K.: Algorithm for Clustering Data. Prentice-Hall, Englewood Cliffs (1988)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2003

Authors and Affiliations

  • JungHwan Oh
    • 1
  • JeongKyu Lee
    • 1
  • Sanjaykumar Kote
    • 1
  • Babitha Bandi
    • 1
  1. 1.Department of Computer Science and EngineeringUniversity of Texas at ArlingtonArlingtonUSA

Personalised recommendations