Automatic Learning of Background Semantics in Generic Surveilled Scenes

  • Carles Fernández
  • Jordi Gonzàlez
  • Xavier Roca
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6312)


Advanced surveillance systems for behavior recognition in outdoor traffic scenes depend strongly on the particular configuration of the scenario. Scene-independent trajectory analysis techniques statistically infer semantics in locations where motion occurs, and such inferences are typically limited to abnormality. Thus, it is interesting to design contributions that automatically categorize more specific semantic regions. State-of-the-art approaches for unsupervised scene labeling exploit trajectory data to segment areas like sources, sinks, or waiting zones. Our method, in addition, incorporates scene-independent knowledge to assign more meaningful labels like crosswalks, sidewalks, or parking spaces. First, a spatiotemporal scene model is obtained from trajectory analysis. Subsequently, a so-called GI-MRF inference process reinforces spatial coherence, and incorporates taxonomy-guided smoothness constraints. Our method achieves automatic and effective labeling of conceptual regions in urban scenarios, and is robust to tracking errors. Experimental validation on 5 surveillance databases has been conducted to assess the generality and accuracy of the segmentations. The resulting scene models are used for model-based behavior analysis.


Ground Truth Segmentation Accuracy Parking Space Smoothness Constraint Ground Truth Image 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Robertson, N., Reid, I.: A general method for human activity recognition in video. CVIU 104, 232–248 (2006)Google Scholar
  2. 2.
    Ballan, L., Bertini, M., Serra, G., Del Bimbo, A.: Video annotation and retrieval using ontologies and rule learning. IEEE Multimedia (2010)Google Scholar
  3. 3.
    Makris, D., Ellis, T.: Learning semantic scene models from observing activity in visual surveillance. IEEE TSCM, Part B 35, 397–408 (2005)Google Scholar
  4. 4.
    Hu, W., Xiao, X., Fu, Z., Xie, D.: A system for learning statistical motion patterns. PAMI 28, 1450–1464 (2006)Google Scholar
  5. 5.
    Piciarelli, C., Foresti, G.L.: On-line trajectory clustering for anomalous events detection. PRL 27, 1835–1842 (2006)Google Scholar
  6. 6.
    Basharat, A., Gritai, A., Shah, M.: Learning object motion patterns for anomaly detection and improved object detection. In: CVPR, Anchorage, USA (2008)Google Scholar
  7. 7.
    Baiget, P., Sommerlade, E., Reid, I., Gonzàlez, J.: Finding prototypes to estimate trajectory development in outdoor scenarios. In: 1st THEMIS, Leeds, UK (2008)Google Scholar
  8. 8.
    Wang, X., Tieu, K., Grimson, E.: Learning semantic scene models by trajectory analysis. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3953, pp. 110–123. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  9. 9.
    Li, J., Gong, S., Xiang, T.: Scene segmentation for behaviour correlation. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part IV. LNCS, vol. 5305, pp. 383–395. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  10. 10.
    Nagel, H.H., Gerber, R.: Representation of occurrences for road vehicle traffic. AI-Magazine 172, 351–391 (2008)Google Scholar
  11. 11.
    Gonzàlez, J., Rowe, D., Varona, J., Xavier Roca, F.: Understanding dynamic scenes based on human sequence evaluation. IVC 27, 1433–1444 (2009)Google Scholar
  12. 12.
    Albanese, M., Chellappa, R., Moscato, V., Picariello, A., Subrahmanian, V.S., Turaga, P., Udrea, O.: A constrained probabilistic petri net framework for human activity detection in video. IEEE TOM 10, 982–996 (2008)Google Scholar
  13. 13.
    Fusier, F., Valentin, V., Bremond, F., Thonnat, M., Borg, M., Thirde, D., Ferryman, J.: Video understanding for complex activity recognition. MVA 18, 167–188 (2007)zbMATHCrossRefGoogle Scholar
  14. 14.
    Kumar, M., Torr, P., Zisserman, A.: Obj. Cut. In: CVPR (2005)Google Scholar
  15. 15.
    Kumar, S., Hebert, M.: Discriminative fields for modeling spatial dependencies in natural images. In: Advances in Neural Information Processing Systems, vol. 16 (2004)Google Scholar
  16. 16.
    Winn, J., Shotton, J.: The layout consistent random field for recognizing and segmenting partially occluded objects. In: CVPR, pp. 37–44 (2006)Google Scholar
  17. 17.
    Shotton, J., Johnson, M., Cipolla, R., Center, T., Kawasaki, J.: Semantic texton forests for image categorization and segmentation. In: CVPR (2008)Google Scholar
  18. 18.
    Li, S.: Markov random field modeling in image analysis. Springer, Heidelberg (2001)zbMATHGoogle Scholar
  19. 19.
    Hartley, R., Zisserman, A.: Multiple view geometry in computer vision. Cambridge Univ. Press, Cambridge (2003)Google Scholar
  20. 20.
    Felzenszwalb, P., Huttenlocher, D.: Efficient belief propagation for early vision. IJCV 70, 41–54 (2006)CrossRefGoogle Scholar
  21. 21.
    Croft, W., Cruse, D.: Cognitive linguistics. Cambridge Univ. Press, Cambridge (2004)Google Scholar
  22. 22.
    Rowe, D., Gonzàlez, J., Pedersoli, M., Villanueva, J.: On tracking inside groups. Machine Vision and Applications 21, 113–127 (2010)CrossRefGoogle Scholar
  23. 23.
    Bose, B., Grimson, E.: Improving object classification in far-field video. In: CVPR (2004)Google Scholar
  24. 24.
    Black, J., Makris, D., Ellis, T.: Hierarchical database for a multi-camera surveillance system. Pattern Analysis and Applications 7, 430–446 (2004)CrossRefMathSciNetGoogle Scholar
  25. 25.
    Landis, J., Koch, G.: The measurement of observer agreement for categorical data. Biometrics 33, 159–174 (1977)zbMATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Carles Fernández
    • 1
  • Jordi Gonzàlez
    • 1
  • Xavier Roca
    • 1
  1. 1.Dept. Ciències de la Computació & Computer Vision Center, Edifici O, Campus UABBarcelonaSpain

Personalised recommendations