Abstract
For a responsive audio art installation in a skylit atrium, we developed a single-camera statistical segmentation and tracking algorithm. The algorithm combines statistical background image estimation, per-pixel Bayesian classification, and an approximate solution to the multi-target tracking problem using a bank of Kalman filters and Gale-Shapley matching. A heuristic confidence model enables selective filtering of tracks based on dynamic data. Experiments suggest that our algorithm improves recall and \(F_{2}\)-score over existing methods in OpenCV 2.1. We also find that feedback between the tracking and the segmentation systems improves recall and \(F_{2}\)-score. The system operated effectively for 5–8 h per day for 4 months. Source code and sample data is open source and available in OpenCV.
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Adler S (2011) Q &A and Q at Jewish Contemporary Museum. SFChronicle, 26 April 2011. http://www.sfgate.com/entertainment/article/Q-A-and-Q-at-Jewish-Contemporary-Museum-2373817.php
Alonso IP, Llorca DF, Sotelo MÁ, Bergasa LM, de Toro PR, Nuevo J, Ocaña M, Garrido MAG (2007) Combination of feature extraction methods for svm pedestrian detection. IEEE Trans Intell Transp Syst 8(2):292–307
Are we there yet? at the Contemporary Jewish Museum. DesignBoom Magazine, 28 June 2011. http://www.designboom.com/art/are-we-there-yet-at-the-contemporary-jewish-museum/
‘Are we there yet?’: Gil Gershoni, Ken Goldberg At San Francisco’s Contemporary Jewish Museum. Huffington Post, 10 August 2011. http://www.huffingtonpost.com/2011/06/10/are-we-there-yet-gil-gers_n_872899.html#s288913
Argyros AA, Lourakis MIA (2006) Binocular hand tracking and reconstruction based on 2D shape matching. In: IEEE 8th international conference on pattern recognition, vol 1. pp 207–210
Baillieul J, Ozcimder K (2012) The control theory of motion-based communication: problems in teaching robots to dance. In: IEEE American control conference (ACC’ 2012), pp 4319–4326
Blackman SS (1986) Multiple-target tracking with radar applications. Artech House Inc, Dedham, 463 p 1
Bradski GR, Pisarevsky V (2000) Intel’s computer vision library: applications in calibration, stereo segmentation, tracking, gesture, face and object recognition. In IEEE computer vision and pattern recognition 2000, vol 2, pp 796–797
Bradski G, Kaehler A (2008) Learning OpenCV: computer vision with the OpenCV library. O’Reilly Media Inc, Sebastopol, USA
Chen T, Haussecker H, Bovyrin A, Belenov R, Rodyushkin K, Eruhimov V (2005) Computer vision workload analysis: case study of video surveillance systems. Intel Technol J 9(2):109–118
Chen Y, Yang J (2013) Robust principal component analysis for recognition. In: Sun C, Fang F, Zhou ZH, Yang W, Liu ZY (eds) IScIDE 2013, vol 8261. LNCS Springer, Heidelberg, pp 223–229
Coifman B, Beymer D, McLauchlan P, Malik J (1998) A real-time computer vision system for vehicle tracking and traffic surveillance. Transp Res Part C: Emerg Technol 6(4):271–288
Comaniciu D, Meer P (2002) Mean shift: a robust approach toward feature space analysis. IEEE Trans Pattern Anal Mach Intell 24(5):603–619
Curiel J (2011) ‘Are we there yet? 5,000 years of answering questions with questions’: art review. SFWeekly, 13 April 2011. http://bit.ly/awty-sf-weekly
Egerstedt M, Murphey T, Ludwig J (2007) Motion programs for puppet choreography and control. In: Bemporad A, Bicchi A, Buttazzo G (eds) HSCC 2007, vol 4416. LNCS Springer, Heidelberg, pp 190–202
Elgammal A, Duraiswami R, Harwood D, Davis LS (2002) Background and foreground modeling using nonparametric kernel density estimation for visual surveillance. Proc IEEE 90(7):1151–1163
Emily S (2011) The art of the question: robotic algorithms put visitors in an auditory sea of queries. JWeekly, 7 April 2011. http://www.jweekly.com/article/full/61372/the-art-of-the-question-robotic-algorithms-put-visitors-in-an-auditory-sea-/
Friedman J, Hastie T, Tibshirani R (2000) Additive logistic regression: a statistical view of boosting (with discussion and a rejoinder by the authors). Ann Stat 28(2):337–407
Gale D, Shapley LS (1962) College admissions and the stability of marriage. Am Math Mon 69(1):9–15
Horprasert T, Harwood D, Davis LS (1999) A statistical approach for real-time robust background subtraction and shadow detection. In IEEE ICCV, vol 99, pp 1–19
Huepe C, Cádiz RF, Colasso M (2012) Generating music from flocking dynamics. In: IEEE American control conference (ACC’ 2012), pp 4339–4344
Jeon J, Lavrenko V, Manmatha R (2003) Automatic image annotation and retrieval using cross-media relevance models. In: Proceedings of the 26th annual international ACM SIGIR conference on research and development in informaion retrieval, pp 119–126
KaewTraKulPong P, Bowden R (2002) An improved adaptive background mixture model for real-time tracking with shadow detection. In: Video-based surveillance systems. Springer, pp 135–144
Kalman RE et al (1960) A new approach to linear filtering and prediction problems J Basic Eng 82(1):35–45
LaViers A, Egerstedt M (2011) The ballet automaton: a formal model for human motion. In: IEEE American control conference (ACC’ 2011), pp 3837–3842
LaViers A, Egerstedt M (2012) Style based robotic motion. In: IEEE American control conference (ACC’ 2012), pp 4327–4332
Leonard NE, Young G, Hochgraf K, Swain D, Trippe A, Chen W, Marshall S (2012) The dance studio: analysis of human flocking. In: IEEE American control conference (ACC’ 2012), pp 4333–4338
Li L, Huang W, Gu IYH, Tian Q (2003) Foreground object detection from videos containing complex background. In: Proceedings of the 11th ACM international conference on multimedia, pp 2–10
Li L, Huang W, Gu IYH, Tian Q (2004) Statistical modeling of complex backgrounds for foreground object detection. IEEE Trans Image Process 13(11):1459–1472
Martin DR, Fowlkes CC, Malik J (2004) Learning to detect natural image boundaries using local brightness, color, and texture cues. IEEE Trans Pattern Anal Mach Intell 26(5):530–549
Masoud O, Papanikolopoulos NP (2001) A novel method for tracking and counting pedestrians in real-time using a single camera. IEEE Trans Veh Technol 50(5):1267–1278
Meyer F, Beucher S (1990) Morphological segmentation. J Vis Commun Image Represent 1(1):21–46
Najar M, Vidal J (2003) Kalman tracking for mobile location in nlos situations. In: 14th IEEE proceedings on personal, indoor and mobile radio communications, 2003 (PIMRC 2003), vol 3. IEEE, pp 2203–2207
Nataraj N (2011) ‘Are we there yet?’ at the Jewish Museum. SFChronicle, 31 March 2011. http://www.sfgate.com/art/article/Are-We-There-Yet-at-the-Jewish-Museum-2377224.php
Nummiaro K, Koller-Meier E, Van Gool L (2003) An adaptive color-based particle filter. Image Vis Comput 21(1):99–110
Okuma K, Taleghani A, Freitas ND, Little JJ, Lowe DG (2004) A boosted particle filter: multitarget detection and tracking. In: Proceedings of the computer vision-ECCV 2004. Springer, Heidelberg, pp 28–39
Olson DL, Delen D (2008) Advanced data mining techniques. Springer, Heidelberg
Orma S (2011) ‘Are we there yet?’ leaves many unanswered questions—but that’s the point. SFWeekly, 31 March 2011. http://blogs.sfweekly.com/exhibitionist/2011/03/are_we_there_yet_leaves_visito.php
Pescovitz D (2011) Are we there yet?: new art installation at SF Contemporary Jewish Museum. BoingBoing, 22 March 2011. http://boingboing.net/2011/03/22/are-we-there-yet-new.html
Power PW, Schoonees JA (2002) Understanding background mixture models for foreground segmentation. In: Proceedings image and vision computing, New Zealand, vol 2002
Rosenkrantz G (2011) The art of questioning in the 21st century. The Covenant Foundation, 16 June 2011. http://www.covenantfn.org/news/79/251/The-Art-Of-Questioning-In-The-21St-Century.
Samuel M (2011) Have you ever questioned art? This art questions you. CNN, 19 April 2011. http://www.cnn.com/2011/LIVING/04/19/questions.as.art/index.html
Samuel M, Samuel S (2011) Are you listening? The sound installation that gets inside your head. KALW News, 30 June 2011. http://www.kalwnews.org/audio/2011/04/13/are-you-listening-the-sound-installation-gets-inside-your-head_934965.html
Schoellig AP, Wiltsche C, D’Andrea R (2012) Feed-forward parameter identification for precise periodic quadrocopter motions. In: IEEE American control conference (ACC’ 2012), pp 4313–4318
Sinopoli B, Schenato L, Franceschetti M, Poolla K, Jordan MI, Sastry S (2004) Kalman filtering with intermittent observations. IEEE Trans Autom Control 49(9):1453–1464
Stauffer C, Grimson WEL (2000) Learning patterns of activity using real-time tracking. IEEE Trans Pattern Anal Mach Intell 22(8):747–757
Vincent L (1994) Morphological area openings and closings for grey-scale images. NATO ASI Series F Comput Syst Sci 126:197–208
Vincent Luc (1993) Morphological grayscale reconstruction in image analysis: applications and efficient algorithms. IEEE Trans Image Process 2(2):176–201
Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition (CVPR’ 2001), vol 1
Viola P, Jones MJ, Snow D (2005) Detecting pedestrians using patterns of motion and appearance. Int J Comput Vis 63(2):153–161
White R (2011) M.V. artist would like to ask you a few questions. Mill Valley Herald, 13 April 2011. http://www.marinscope.com/mill_valley_herald/news/article_f3a7b988-7f35-5ef0-8617-cbe95ec14867.html
Winant C (2011) Are we Jewish yet? KQED, 12 April 2011. http://www.kqed.org/arts/visualarts/article.jsp?essid=50353
Yang C, Duraiswami R, Davis L (2005) Fast multiple object tracking via a hierarchical particle filter. In: IEEE computer society: 10th IEEE international conference on computer vision (ICCV’ 2005), vol 1, pp 212–219
Yilmaz A, Javed O, Shah M (2006) Object tracking: a survey. ACM Comput Surv (CSUR) 38(4):13
Zivkovic Z, van der Heijden F (2006) Efficient adaptive density estimation per image pixel for the task of background subtraction. Pattern Recogn lett 27(7):773–780
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Godbehere, A.B., Goldberg, K. (2014). Algorithms for Visual Tracking of Visitors Under Variable-Lighting Conditions for a Responsive Audio Art Installation. In: LaViers, A., Egerstedt, M. (eds) Controls and Art. Springer, Cham. https://doi.org/10.1007/978-3-319-03904-6_8
Download citation
DOI: https://doi.org/10.1007/978-3-319-03904-6_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-03903-9
Online ISBN: 978-3-319-03904-6
eBook Packages: EngineeringEngineering (R0)