Journal of Real-Time Image Processing

, Volume 5, Issue 4, pp 215–230 | Cite as

SEMARTCam scheduler: semantics driven real-time data collection from indoor camera networks to maximize event detection

  • Ronen VaisenbergEmail author
  • Sharad Mehrotra
  • Deva Ramanan
Special Issue


A distributed camera network allows for many compelling applications such as large-scale tracking or event detection. In most practical systems, resources are constrained. Although one would like to probe every camera at every time instant and store every frame, this is simply not feasible. Constraints arise from network bandwidth restrictions, I/O and disk usage from writing images, and CPU usage needed to extract features from the images. Assume that, due to resource constraints, only a subset of sensors can be probed at any given time unit. This paper examines the problem of selecting the “best” subset of sensors to probe under some user-specified objective—e.g., detecting as much motion as possible. With this objective, we would like to probe a camera when we expect motion, but would not like to waste resources on a non-active camera. The main idea behind our approach is the use of sensor semantics to guide the scheduling of resources. We learn a dynamic probabilistic model of motion correlations between cameras, and use the model to guide resource allocation for our sensor network. Although previous work has leveraged probabilistic models for sensor-scheduling, our work is distinct in its focus on real-time building-monitoring using a camera network. We validate our approach on a sensor network of a dozen cameras spread throughout a university building, recording measurements of unscripted human activity over a two week period. We automatically learnt a semantic model of typical behaviors, and show that one can significantly improve efficiency of resource allocation by exploiting this model.


Video surveillance Algorithms Sensor network Real-time 



Support of this research by the National Science Foundation under Award Numbers 0331707 and 0331690 and the Department of Homeland Security under award number EMW-2007-FP-02535 is gratefully acknowledged.


  1. 1.
    Anandathirtha, P., Ramakrishnan, K., Raja, S., Kankanhalli, M.: Experiential sampling for object detection in video. In: Multimedia Content Analysis: Theory and Applications, p. 175 (2009)Google Scholar
  2. 2.
    Brown, R.G., Hwang, P.Y.C.: Introduction to Random Signals and Applied Kalman Filtering. Wiley, New York (1992)Google Scholar
  3. 3.
    Chen, C., Roussopoulos, N.: Adaptive selectivity estimation using query feedback. In: Proceedings of the 1994 ACM SIGMOD International Conference on Management of Data, pp. 161–172 (1994)Google Scholar
  4. 4.
    Daeipour, E., Bar-Shalom, Y.: An interacting multiple model approach for target tracking withglint noise. IEEE Trans. Aerosp. Electr. Syst. 31(2), 706–715 (1995)CrossRefGoogle Scholar
  5. 5.
    Gupta, V., Chung, T., Hassibi, B., Murray, R.: On a stochastic sensor selection algorithm with applications in sensor scheduling and sensor coverage. Automatica 42(2), 251–260 (2006)zbMATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Han, Q., Mehrotra, S., Venkatasubramanian, N.: Energy efficient data collection in distributed sensor environments. In: Proceedings of 24th International Conference on Distributed Computing Systems, vol. 39, pp. 590–597 (2004)Google Scholar
  7. 7.
    Ihler, A., Hutchins, J., Smyth, P.: Adaptive event detection with time-varying poisson processes. In: KDD ’06: Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 207–216. ACM Press, New York (2006). doi: 10.1145/1150402.1150428
  8. 8.
    Isler, V., Bajcsy, R.: The sensor selection problem for bounded uncertainty sensing models. In: IPSN 2005 Fourth International Symposium on Information Processing in Sensor Networks, 2005, pp. 151–158 (2005)Google Scholar
  9. 9.
    Kao, B., Garcia-Molina, H.: Deadline assignment in a distributed soft real-time system. IEEE Trans. Parallel Distrib. Syst. 8(12), 1268–1274 (1997)CrossRefGoogle Scholar
  10. 10.
    Krishnamurthy, V.: Algorithms for optimal scheduling and management of hidden Markovmodel sensors. IEEE Trans. Signal Process. [see also IEEE Trans. Acoust. Speech Signal Process.] 50(6), 1382–1397 (2002)MathSciNetGoogle Scholar
  11. 11.
    Lazaridis, I., Mehrotra, S.: Capturing sensor-generated time series with quality guarantees. In: International Conference on Data Engineering (ICDE) (2003)Google Scholar
  12. 12.
    Makris, D., Ellis, T., Black, J.: Bridging the gaps between cameras. In: CVPR 2004 Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2 (2004)Google Scholar
  13. 13.
    Murphy, K.P., Weiss, Y.: The factored frontier algorithm for approximate inference in dbns. In: UAI ’01: Proceedings of the 17th Conference in Uncertainty in Artificial Intelligence, pp. 378–385. Morgan Kaufmann Publishers Inc., San Francisco (2001)Google Scholar
  14. 14.
    Nieh, J., Lam, M.: The design, implementation and evaluation of SMART: a scheduler for multimedia applications. In: Proceedings of the Sixteenth ACM Symposium on Operating Systems Principles, pp. 184–197. ACM, New York (1997)Google Scholar
  15. 15.
    Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann Publishers Inc., San Francisco (1988)Google Scholar
  16. 16.
    Rabiner, L.R.: A tutorial on hidden markov models and selected applications in speech recognition. Proc. IEEE 77(2), 257–286 (1989)CrossRefGoogle Scholar
  17. 17.
    Schulz, D., Fox, D., Hightower, J.: People tracking with anonymous and id-sensors using rao-blackwellised particle filters. In: Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI) (2003)Google Scholar
  18. 18.
    Scott, S.: Detecting network intrusion using a Markov modulated nonhomogeneous Poisson process. J. Am. Stat. Assoc. (2010)Google Scholar
  19. 19.
    Song, B., Roy-Chowdhury, A.: Stochastic adaptive tracking in a camera network. In: IEEE 11th International Conference on Computer Vision, 2007 ICCV, pp. 1–8 (2007)Google Scholar
  20. 20.
    Vaisenberg, R., Mehrotra, S., Ramanan, D.: Exploiting semantics for scheduling real-time data collection from sensors to maximize event detection. In: Proceedings of SPIE, vol. 7253, p. 72530B (2009)Google Scholar
  21. 21.
    Williams, J., Fisher, J. III, Willsky, A.: An approximate dynamic programming approach to a communication constrained sensor management problem. In: Proceedings of Eighth International Conference of Information Fusion (2005)Google Scholar
  22. 22.
    Yu, X., Niyogi, K., Mehrotra, S., Venkatasubramanian, N.: Adaptive target tracking in sensor networks. In: Communication Networks and Distributed Systems Modeling and Simulation Conference (CNDS04) (2004)Google Scholar

Copyright information

© Springer-Verlag 2010

Authors and Affiliations

  • Ronen Vaisenberg
    • 1
    Email author
  • Sharad Mehrotra
    • 1
  • Deva Ramanan
    • 1
  1. 1.School of Information and Computer SciencesUniversity of CaliforniaIrvineUSA

Personalised recommendations