Automatic Activity Profile Generation from Detected Functional Regions for Video Scene Analysis

  • Eran Swears
  • Matthew Turek
  • Roderic Collins
  • A. G. Amitha Perera
  • Anthony Hoogs
Part of the Studies in Computational Intelligence book series (SCI, volume 409)


The potential applications of video surveillance to the Business Intelligence domain continue to grow. For example, automatic computer vision algorithms can provide a fast, efficient process to screen hundreds of hours of video for activity patterns that potentially impact the business. Two such algorithms and their variants are discussed in this chapter. These algorithms analyze surveillance video in order to automatically recognize various functional elements, such as: walkways, roadways, parking-spots, and doorways, through their interactions with pedestrian and vehicle detections. The recognized functional element regions provide a means of capturing statistics related to particular businesses. For example, the owner may be interested in the number of people that enter or exit their business versus the number of people that walk past. Results are shown on functional element recognition and business related activity profiles that demonstrate the effectiveness of these algorithms. Experiments are performed using webcam video of a downtown main street in Ocean City NJ, and surveillance video from the CAVIAR shopping center dataset.


Functional Category Functional Element Functional Region Detect Region Grid Cell Size 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Swears, E., Hoogs, A.: Functional Scene Element Recognition for Video Scene Analysis. In: 2009 IEEE Workshop on Motion and Video Computing (2009)Google Scholar
  2. 2.
    Turek, M.W., Hoogs, A., Collins, R.: Unsupervised Learning of Functional Categories in Video Scenes. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6312, pp. 664–677. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  3. 3.
    CAVIAR project IST 2001 37540. In: CAVIAR: Context Aware Vision Using Image-Based Active Recognition. EC Information Society Technology’s, (cited March 29, 2011)
  4. 4.
    Swears, E., Hoogs, A., Perera, A.G.A.: Learning Motion Patterns in Surveillance Video using HMM Clustering. In: 2008 IEEE Workshop on Motion and Video Computing (2008)Google Scholar
  5. 5.
    Grimson, W., Stauffer, C., Romano, R., Lee, L.: Using adaptive tracking to classify and monitor activities in a site. In: 1998 IEEE Conference on Computer Vision and Pattern Recognition (1998)Google Scholar
  6. 6.
    Stauffer, C., Grimson, W.: Learning Patterns of Activity Using Real-Time Tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence 22, 747–757 (2000)CrossRefGoogle Scholar
  7. 7.
    Makris, D., Ellis, T.: Learning semantic scene models from observing activity in visual surveillance. 2005 IEEE Transactions on Systems, Man, and Cybernetics - Part B: Cybernetics, 397–408 (2005)Google Scholar
  8. 8.
    Makris, D., Ellis, T.J.: Path Detection in Video Surveillance. In: Image and Vision Computing, vol. 20, pp. 895–903 (2002)Google Scholar
  9. 9.
    Junejo, I., Javed, O., Shah, M.: Multi Feature Path Modeling for Video Surveillance. In: International Conference on Pattern Recognition, pp. 716–719 (2004)Google Scholar
  10. 10.
    Stauffer, C.: Estimating tracking sources and sinks. In: 2003 Computer Vision and Pattern Recognition Workshop (2003)Google Scholar
  11. 11.
    Stark, L., Bowyer, K.: Achieving Generalized Object Recognition through Reasoning about Association of Function to Structure. IEEE Pattern Analysis and Machine Intelligence (1991)Google Scholar
  12. 12.
    Gupta, A., Davis, L.: Objects in Action: An Approach for Combining Action Understanding and Object Perception. In: Computer Vision and Pattern Recognition (2007)Google Scholar
  13. 13.
    Peursum, P., West, G., Venkatesh, S.: Combining Image Regions and Human Activity for Indirect Object Recognition in Indoor Wide-Angle Video. In: 2005 International Conference on Computer Vision (2005)Google Scholar
  14. 14.
    Perera, A.G.A., Srinivas, C., Hoogs, A., Brooksby, G., Hu, W.: Multi-object tracking through simultaneous long occlusions and split-merge conditions. In: Computer Vision and Pattern Recognition (2006)Google Scholar
  15. 15.
    Comaniciu, D., Meer, P.: Mean Shift: a robust approach toward feature space analysis. 2002 IEEE Transactions on Pattern Analysis and Machine Intelligence 24, 603–619 (2002)CrossRefGoogle Scholar
  16. 16.
    Li, J., Gong, S., Xiang, T.: Scene Segmentation for Behaviour Correlation. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part IV. LNCS, vol. 5305, pp. 383–395. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  17. 17.
    Wang, X., Tieu, K., Grimson, W.E.L.: Learning Semantic Scene Models by Trajectory Analysis. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3953, pp. 110–123. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  18. 18.
    Ali, S., Shah, M.: Floor Fields for Tracking in High Density Crowd Scenes. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 1–14. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  19. 19.
    Yang, Y., Liu, J., Shah, M.: Video Scene Understanding Using Multi-scale Analysis. In: 2009 International Conference on Computer Vision (2009)Google Scholar
  20. 20.
    Loy, C., Xiang, T., Gong, S.: Time Delayed Correlation Analysis for Multi-Camera Activity Understanding. International Journal of Computer Vision (2010)Google Scholar

Copyright information

© Springer Berlin Heidelberg 2012

Authors and Affiliations

  • Eran Swears
    • 1
  • Matthew Turek
    • 1
  • Roderic Collins
    • 1
  • A. G. Amitha Perera
    • 1
  • Anthony Hoogs
    • 1
  1. 1.KitwareClifton ParkUSA

Personalised recommendations