Abstract
The analysis and mining of traffic video sequences to discover important but previously unknown knowledge such as vehicle identification, traffic flow, queue detection, incident detection, and the spatio-temporal relations of the vehicles at intersections, provide an economic approach for daily traffic monitoring operations. To meet such demands, a multimedia data mining framework is proposed in this paper. The proposed multimedia data mining framework analyzes the traffic video sequences using background subtraction, image/video segmentation, vehicle tracking, and modeling with the multimedia augmented transition network (MATN) model and multimedia input strings, in the domain of traffic monitoring over traffic intersections. The spatio-temporal relationships of the vehicle objects in each frame are discovered and accurately captured and modeled. Such an additional level of sophistication enabled by the proposed multimedia data mining framework in terms of spatio-temporal tracking generates a capability for automation. This capability alone can significantly influence and enhance current data processing and implementation strategies for several problems vis-à-vis traffic operations. Three real-life traffic video sequences obtained from different sources and with different weather conditions are used to illustrate the effectiveness and robustness of the proposed multimedia data mining framework by demonstrating how the proposed framework can be applied to traffic applications to answer the spatio-temporal queries.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Caltrans. Caltrans Live Traffic Cameras, http://video.dot.ca.gov/.
Chen, S.-C. and Kashyap, R.L. (2001). A Spatio-Temporal Semantic Model for Multimedia Database Systems and Multimedia Information Systems. IEEE Trans. on Knowledge and Data Engineering, 13(4), 607–622.
Chen, S.-C., Shyu, M.-L., and Zhang, C.C. (2001). An Intelligent Framework for Spatio-Temporal Vehicle Tracking. In The 4th International IEEE Conference on Intelligent Transportation Systems, Oakland, CA, USA (pp. 213–218).
Chen, S.-C., Shyu, M.-L., Zhang, C.C., and Kashyap, R.L. (2000a). Object Tracking and Augmented Transition Network for Video Indexing and Modeling. In The 12th IEEE International Conference on Tools with Artificial Intelligence (ICTAI 2000), Vancouver, British Columbia, Canada (pp. 428–435).
Chen, S.-C., Sista, S., Shyu, M.-L., and Kashyap, R.L. (1999). Augmented Transition Networks as Video Browsing Models for Multimedia Databases and Multimedia Information Systems. In The 11th IEEE International Conference on Tools with Artificial Intelligence (ICTAI' 99), Chicago, IL, U.S.A. (pp. 175–182).
Chen, S.-C., Sista, S., Shyu, M.-L., and Kashyap, R.L. (2000b). An Indexing and Searching Structure for Multimedia Database Systems. In IS&T/SPIE conference on Storage and Retrieval for Media Databases 2000, San Jose, CA, U.S.A. (pp. 262–270).
Courtney, J.D. (1997). Automatic Video Indexing via Object Motion Analysis. Pattern Recognition, 30(4), 607–625.
Cucchiara, R., Piccardi, M., and Mello, P. (2000). Image Analysis and Rule-based Reasoning for a Traffic Monitoring System. IEEE Trans. on Intelligent Transportation Systems, 1(2), 119–130.
Dailey, D.J., Cathey, F., and Pumrin, S. (2000). An Algorithm to Estimate Mean Traffic Speed Using Uncalibrated Cameras. IEEE Transactions on Intelligent Transportations Systems, 1(2), 98–107.
Fan, L. and Sung, K.K. (2000). Model-Based Varying Pose Face Detection and Facial Feature Registration in Video Images. In The 8th ACM International Conference on Multimedia, Los Angeles, CA (pp. 295–302).
Ferman, A.M., Guensel, B., and Tekalp, A.M. (1997). Object-Based Indexing of MPEG-4 Compressed Video. In Proceedings of SPIE: Visual Communications and Image Processing, San Jose, CA (pp. 953–963).
Friedman, N. and Russell, S. (1997). Image Segmentation in Video Sequences: A Probabilistic Approach. In Proc. Thirteenth Conf. on Uncertainty in Artificial Intelligence (UAI' 97), Providence; RI.
Gonzalez, R.C. and Woods, R.E. (1993). Digital Image Processing. Reading, MA: Addison-Wesley.
Grimson, W.E.L., Stauffer, C., Romano, R., and Lee, L. (1998). Using Adaptive Tracking to Classify and Monitor Activities in a Site. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition Preceding (pp. 22–31).
Haritaoglu, I., Harwood, D., and Davis, L. (1998). W 4—Who, Where, When, What: A Real-Time System for Detecting and Tracking People. In IEEE Third International Conference on Face and Gesture Recognition, Nara, Japan (pp. 222–227).
Haritaoglu, I., Harwood, D., and Davis, L. (2000). A Fast Background Scene Modeling and Maintenance for Outdoor Surveillance. In The 15th IEEE International Conference on Pattern Recognition: Applications, Robotics Systems and Architectures, Barcelona, Spain (pp. 179–183).
Huang, T., Koller, D., Malik, J., and Ogasawara, G. (1994). Automatic Symbolic Traffic Scene Analysis Using Belief Networks. In Proceedings of the AAAI, 12th National Conference on Artificial Intelligence (AAAI' 94), Seattle, WA (pp. 966–972).
IRA. http://i21www.ira.uka.de/image sequences/.
Kamijo, S., Matsushita, Y., Ikeuchi, K., and Sakauchi, M. (1999). Automatic Symbolic Traffic Scene Analysis Using Belief Networks. In IEEE International Conference on Intelligent Transportation Systems, Tokyo Japan (pp. 703–708).
Montgomery Co. Department of Public Works Transportation. ATMS Video Monitoring System Live Traffic Camera Pictures, http://www.dpwt.com/jpgcap/camintro.html.
Sista, S. and Kashyap, R.L. (2000). UnsupervisedVideo Segmentation and Object Tracking. Computers in Industry, 42(2/3), 127–146.
Stauffer, C. and Grimson, W.E.L. (1999). Adaptive Background Mixture Models for Real-Time Tracking. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (pp. 246–252).
Toyama, K., Krumm, J., Brumitt, B., and Meyers, B. (1999). Wallflower: Principles and Practice of Background Maintenance. In 7th International Conference on Computer Vision (ICCV' 99), Island of Crete (pp. 255–261).
Yeo, B.-L. and Yeung, M.M. (1997). Retrieving and Visualizing Video. Communications of the ACM, 40(12), 43–52.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Chen, SC., Shyu, ML., Zhang, C. et al. A Multimedia Data Mining Framework: Mining Information from Traffic Video Sequences. Journal of Intelligent Information Systems 19, 61–77 (2002). https://doi.org/10.1023/A:1015564420544
Issue Date:
DOI: https://doi.org/10.1023/A:1015564420544