Skip to main content
SpringerLink
Log in

Algorithms for Visual Tracking of Visitors Under Variable-Lighting Conditions for a Responsive Audio Art Installation

  • Chapter
  • First Online:
Controls and Art

Abstract

For a responsive audio art installation in a skylit atrium, we developed a single-camera statistical segmentation and tracking algorithm. The algorithm combines statistical background image estimation, per-pixel Bayesian classification, and an approximate solution to the multi-target tracking problem using a bank of Kalman filters and Gale-Shapley matching. A heuristic confidence model enables selective filtering of tracks based on dynamic data. Experiments suggest that our algorithm improves recall and \(F_{2}\)-score over existing methods in OpenCV 2.1. We also find that feedback between the tracking and the segmentation systems improves recall and \(F_{2}\)-score. The system operated effectively for 5–8 h per day for 4 months. Source code and sample data is open source and available in OpenCV.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Adler S (2011) Q &A and Q at Jewish Contemporary Museum. SFChronicle, 26 April 2011. http://www.sfgate.com/entertainment/article/Q-A-and-Q-at-Jewish-Contemporary-Museum-2373817.php

  2. Alonso IP, Llorca DF, Sotelo MÁ, Bergasa LM, de Toro PR, Nuevo J, Ocaña M, Garrido MAG (2007) Combination of feature extraction methods for svm pedestrian detection. IEEE Trans Intell Transp Syst 8(2):292–307

    Google Scholar 

  3. Are we there yet? at the Contemporary Jewish Museum. DesignBoom Magazine, 28 June 2011. http://www.designboom.com/art/are-we-there-yet-at-the-contemporary-jewish-museum/

  4. ‘Are we there yet?’: Gil Gershoni, Ken Goldberg At San Francisco’s Contemporary Jewish Museum. Huffington Post, 10 August 2011. http://www.huffingtonpost.com/2011/06/10/are-we-there-yet-gil-gers_n_872899.html#s288913

  5. Argyros AA, Lourakis MIA (2006) Binocular hand tracking and reconstruction based on 2D shape matching. In: IEEE 8th international conference on pattern recognition, vol 1. pp 207–210

    Google Scholar 

  6. Baillieul J, Ozcimder K (2012) The control theory of motion-based communication: problems in teaching robots to dance. In: IEEE American control conference (ACC’ 2012), pp 4319–4326

    Google Scholar 

  7. Blackman SS (1986) Multiple-target tracking with radar applications. Artech House Inc, Dedham, 463 p 1

    Google Scholar 

  8. Bradski GR, Pisarevsky V (2000) Intel’s computer vision library: applications in calibration, stereo segmentation, tracking, gesture, face and object recognition. In IEEE computer vision and pattern recognition 2000, vol 2, pp 796–797

    Google Scholar 

  9. Bradski G, Kaehler A (2008) Learning OpenCV: computer vision with the OpenCV library. O’Reilly Media Inc, Sebastopol, USA

    Google Scholar 

  10. Chen T, Haussecker H, Bovyrin A, Belenov R, Rodyushkin K, Eruhimov V (2005) Computer vision workload analysis: case study of video surveillance systems. Intel Technol J 9(2):109–118

    Google Scholar 

  11. Chen Y, Yang J (2013) Robust principal component analysis for recognition. In: Sun C, Fang F, Zhou ZH, Yang W, Liu ZY (eds) IScIDE 2013, vol 8261. LNCS Springer, Heidelberg, pp 223–229

    Google Scholar 

  12. Coifman B, Beymer D, McLauchlan P, Malik J (1998) A real-time computer vision system for vehicle tracking and traffic surveillance. Transp Res Part C: Emerg Technol 6(4):271–288

    Article  Google Scholar 

  13. Comaniciu D, Meer P (2002) Mean shift: a robust approach toward feature space analysis. IEEE Trans Pattern Anal Mach Intell 24(5):603–619

    Article  Google Scholar 

  14. Curiel J (2011) ‘Are we there yet? 5,000 years of answering questions with questions’: art review. SFWeekly, 13 April 2011. http://bit.ly/awty-sf-weekly

  15. Egerstedt M, Murphey T, Ludwig J (2007) Motion programs for puppet choreography and control. In: Bemporad A, Bicchi A, Buttazzo G (eds) HSCC 2007, vol 4416. LNCS Springer, Heidelberg, pp 190–202

    Google Scholar 

  16. Elgammal A, Duraiswami R, Harwood D, Davis LS (2002) Background and foreground modeling using nonparametric kernel density estimation for visual surveillance. Proc IEEE 90(7):1151–1163

    Google Scholar 

  17. Emily S (2011) The art of the question: robotic algorithms put visitors in an auditory sea of queries. JWeekly, 7 April 2011. http://www.jweekly.com/article/full/61372/the-art-of-the-question-robotic-algorithms-put-visitors-in-an-auditory-sea-/

  18. Friedman J, Hastie T, Tibshirani R (2000) Additive logistic regression: a statistical view of boosting (with discussion and a rejoinder by the authors). Ann Stat 28(2):337–407

    Google Scholar 

  19. Gale D, Shapley LS (1962) College admissions and the stability of marriage. Am Math Mon 69(1):9–15

    Google Scholar 

  20. Horprasert T, Harwood D, Davis LS (1999) A statistical approach for real-time robust background subtraction and shadow detection. In IEEE ICCV, vol 99, pp 1–19

    Google Scholar 

  21. Huepe C, Cádiz RF, Colasso M (2012) Generating music from flocking dynamics. In: IEEE American control conference (ACC’ 2012), pp 4339–4344

    Google Scholar 

  22. Jeon J, Lavrenko V, Manmatha R (2003) Automatic image annotation and retrieval using cross-media relevance models. In: Proceedings of the 26th annual international ACM SIGIR conference on research and development in informaion retrieval, pp 119–126

    Google Scholar 

  23. KaewTraKulPong P, Bowden R (2002) An improved adaptive background mixture model for real-time tracking with shadow detection. In: Video-based surveillance systems. Springer, pp 135–144

    Google Scholar 

  24. Kalman RE et al (1960) A new approach to linear filtering and prediction problems J Basic Eng 82(1):35–45

    Google Scholar 

  25. LaViers A, Egerstedt M (2011) The ballet automaton: a formal model for human motion. In: IEEE American control conference (ACC’ 2011), pp 3837–3842

    Google Scholar 

  26. LaViers A, Egerstedt M (2012) Style based robotic motion. In: IEEE American control conference (ACC’ 2012), pp 4327–4332

    Google Scholar 

  27. Leonard NE, Young G, Hochgraf K, Swain D, Trippe A, Chen W, Marshall S (2012) The dance studio: analysis of human flocking. In: IEEE American control conference (ACC’ 2012), pp 4333–4338

    Google Scholar 

  28. Li L, Huang W, Gu IYH, Tian Q (2003) Foreground object detection from videos containing complex background. In: Proceedings of the 11th ACM international conference on multimedia, pp 2–10

    Google Scholar 

  29. Li L, Huang W, Gu IYH, Tian Q (2004) Statistical modeling of complex backgrounds for foreground object detection. IEEE Trans Image Process 13(11):1459–1472

    Google Scholar 

  30. Martin DR, Fowlkes CC, Malik J (2004) Learning to detect natural image boundaries using local brightness, color, and texture cues. IEEE Trans Pattern Anal Mach Intell 26(5):530–549

    Google Scholar 

  31. Masoud O, Papanikolopoulos NP (2001) A novel method for tracking and counting pedestrians in real-time using a single camera. IEEE Trans Veh Technol 50(5):1267–1278

    Google Scholar 

  32. Meyer F, Beucher S (1990) Morphological segmentation. J Vis Commun Image Represent 1(1):21–46

    Article  Google Scholar 

  33. Najar M, Vidal J (2003) Kalman tracking for mobile location in nlos situations. In: 14th IEEE proceedings on personal, indoor and mobile radio communications, 2003 (PIMRC 2003), vol 3. IEEE, pp 2203–2207

    Google Scholar 

  34. Nataraj N (2011) ‘Are we there yet?’ at the Jewish Museum. SFChronicle, 31 March 2011. http://www.sfgate.com/art/article/Are-We-There-Yet-at-the-Jewish-Museum-2377224.php

  35. Nummiaro K, Koller-Meier E, Van Gool L (2003) An adaptive color-based particle filter. Image Vis Comput 21(1):99–110

    Article  Google Scholar 

  36. Okuma K, Taleghani A, Freitas ND, Little JJ, Lowe DG (2004) A boosted particle filter: multitarget detection and tracking. In: Proceedings of the computer vision-ECCV 2004. Springer, Heidelberg, pp 28–39

    Google Scholar 

  37. Olson DL, Delen D (2008) Advanced data mining techniques. Springer, Heidelberg

    Google Scholar 

  38. Orma S (2011) ‘Are we there yet?’ leaves many unanswered questions—but that’s the point. SFWeekly, 31 March 2011. http://blogs.sfweekly.com/exhibitionist/2011/03/are_we_there_yet_leaves_visito.php

  39. Pescovitz D (2011) Are we there yet?: new art installation at SF Contemporary Jewish Museum. BoingBoing, 22 March 2011. http://boingboing.net/2011/03/22/are-we-there-yet-new.html

  40. Power PW, Schoonees JA (2002) Understanding background mixture models for foreground segmentation. In: Proceedings image and vision computing, New Zealand, vol 2002

    Google Scholar 

  41. Rosenkrantz G (2011) The art of questioning in the 21st century. The Covenant Foundation, 16 June 2011. http://www.covenantfn.org/news/79/251/The-Art-Of-Questioning-In-The-21St-Century.

  42. Samuel M (2011) Have you ever questioned art? This art questions you. CNN, 19 April 2011. http://www.cnn.com/2011/LIVING/04/19/questions.as.art/index.html

  43. Samuel M, Samuel S (2011) Are you listening? The sound installation that gets inside your head. KALW News, 30 June 2011. http://www.kalwnews.org/audio/2011/04/13/are-you-listening-the-sound-installation-gets-inside-your-head_934965.html

  44. Schoellig AP, Wiltsche C, D’Andrea R (2012) Feed-forward parameter identification for precise periodic quadrocopter motions. In: IEEE American control conference (ACC’ 2012), pp 4313–4318

    Google Scholar 

  45. Sinopoli B, Schenato L, Franceschetti M, Poolla K, Jordan MI, Sastry S (2004) Kalman filtering with intermittent observations. IEEE Trans Autom Control 49(9):1453–1464

    Google Scholar 

  46. Stauffer C, Grimson WEL (2000) Learning patterns of activity using real-time tracking. IEEE Trans Pattern Anal Mach Intell 22(8):747–757

    Google Scholar 

  47. Vincent L (1994) Morphological area openings and closings for grey-scale images. NATO ASI Series F Comput Syst Sci 126:197–208

    Google Scholar 

  48. Vincent Luc (1993) Morphological grayscale reconstruction in image analysis: applications and efficient algorithms. IEEE Trans Image Process 2(2):176–201

    Article  Google Scholar 

  49. Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition (CVPR’ 2001), vol 1

    Google Scholar 

  50. Viola P, Jones MJ, Snow D (2005) Detecting pedestrians using patterns of motion and appearance. Int J Comput Vis 63(2):153–161

    Google Scholar 

  51. White R (2011) M.V. artist would like to ask you a few questions. Mill Valley Herald, 13 April 2011. http://www.marinscope.com/mill_valley_herald/news/article_f3a7b988-7f35-5ef0-8617-cbe95ec14867.html

  52. Winant C (2011) Are we Jewish yet? KQED, 12 April 2011. http://www.kqed.org/arts/visualarts/article.jsp?essid=50353

  53. Yang C, Duraiswami R, Davis L (2005) Fast multiple object tracking via a hierarchical particle filter. In: IEEE computer society: 10th IEEE international conference on computer vision (ICCV’ 2005), vol 1, pp 212–219

    Google Scholar 

  54. Yilmaz A, Javed O, Shah M (2006) Object tracking: a survey. ACM Comput Surv (CSUR) 38(4):13

    Article  Google Scholar 

  55. Zivkovic Z, van der Heijden F (2006) Efficient adaptive density estimation per image pixel for the task of background subtraction. Pattern Recogn lett 27(7):773–780

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of EECS and IEOR, University of California at Berkeley, Berkeley, CA, USA

    Andrew B. Godbehere & Ken Goldberg

Authors
  1. Andrew B. Godbehere
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ken Goldberg
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andrew B. Godbehere .

Editor information

Editors and Affiliations

  1. Systems and Information Engineering Department, University of Virginia, Charlottesville, Virginia, USA

    Amy LaViers

  2. School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, Georgia, USA

    Magnus Egerstedt

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Godbehere, A.B., Goldberg, K. (2014). Algorithms for Visual Tracking of Visitors Under Variable-Lighting Conditions for a Responsive Audio Art Installation. In: LaViers, A., Egerstedt, M. (eds) Controls and Art. Springer, Cham. https://doi.org/10.1007/978-3-319-03904-6_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-03904-6_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-03903-9

  • Online ISBN: 978-3-319-03904-6

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation