Advertisement

Particle Flow SMC-PHD Filter for Audio-Visual Multi-speaker Tracking

  • Yang LiuEmail author
  • Wenwu Wang
  • Jonathon Chambers
  • Volkan Kilic
  • Adrian Hilton
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10169)

Abstract

Sequential Monte Carlo probability hypothesis density (SMC-PHD) filtering has been recently exploited for audio-visual (AV) based tracking of multiple speakers, where audio data are used to inform the particle distribution and propagation in the visual SMC-PHD filter. However, the performance of the AV-SMC-PHD filter can be affected by the mismatch between the proposal and the posterior distribution. In this paper, we present a new method to improve the particle distribution where audio information (i.e. DOA angles derived from microphone array measurements) is used to detect new born particles and visual information (i.e. histograms) is used to modify the particles with particle flow (PF). Using particle flow has the benefit of migrating particles smoothly from the prior to the posterior distribution. We compare the proposed algorithm with the baseline AV-SMC-PHD algorithm using experiments on the AV16.3 dataset with multi-speaker sequences.

Keywords

Audio-visual tracking PHD filter SMC implementation Multi-speaker tracking 

References

  1. 1.
    Baser, E., Efe, M.: A novel auxiliary particle PHD filter. In: 15th International Conference on Information Fusion, pp. 165–172. IEEE (2012)Google Scholar
  2. 2.
    Bernardin, K., Gehrig, T., Stiefelhagen, R.: Multi-level particle filter fusion of features and cues for audio-visual person tracking. In: Stiefelhagen, R., Bowers, R., Fiscus, J. (eds.) CLEAR/RT -2007. LNCS, vol. 4625, pp. 70–81. Springer, Heidelberg (2008). doi: 10.1007/978-3-540-68585-2_5 CrossRefGoogle Scholar
  3. 3.
    Cevher, V., Sankaranarayanan, A.C., McClellan, J.H., Chellappa, R.: Target tracking using a joint acoustic video system. IEEE Trans. Multimedia 9(4), 715–727 (2007)CrossRefGoogle Scholar
  4. 4.
    Cui, P., Sun, L.F., Wang, F., Yang, S.Q.: Contextual mixture tracking. IEEE Trans. Multimedia 11(2), 333–341 (2009)CrossRefGoogle Scholar
  5. 5.
    Daum, F., Huang, J.: Particle flow for nonlinear filters with log-homotopy. In: Proceedings of SPIE, pp. 696918-1–696918-12 (2008)Google Scholar
  6. 6.
    Daum, F., Huang, J.: Nonlinear filters with log-homotopy. In: Drummond, O.E., Teichgraeber, R.D. (eds.) Optical Engineering + Applications. pp. 669918–669918-15. International Society for Optics and Photonics (2007)Google Scholar
  7. 7.
    Daum, F., Huang, J.: Nonlinear filters with particle flow. In: SPIE Optical Engineering + Applications, pp. 74450R-1–74450R-9. International Society for Optics and Photonics (2009)Google Scholar
  8. 8.
    Daum, F., Huang, J.: Particle flow for nonlinear filters. In: 2011 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 5920–5923 (2011)Google Scholar
  9. 9.
    Daum, F., Huang, J.: Renormalization group flow and other ideas inspired by physics for nonlinear filters, Bayesian decisions, and transport. In: SPIE Defense + Security, p. 90910I. International Society for Optics and Photonics (2014)Google Scholar
  10. 10.
    Daum, F., Huang, J.: Renormalization group flow in k-space for nonlinear filters, Bayesian decisions and transport. In: 18th International Conference on Information Fusion, pp. 1617–1624. IEEE (2015)Google Scholar
  11. 11.
    Daum, F., Huang, J., Noushin, A.: Exact particle flow for nonlinear filters. In: SPIE Defense, Security, and Sensing, p. 769704 (2010)Google Scholar
  12. 12.
    Daum, F., Huang, J., Noushin, A.: Small curvature particle flow for nonlinear filters. Signal Processing, Sensor Fusion, and Target Recognition XIX 7697(1), 769704 (2010)CrossRefGoogle Scholar
  13. 13.
    Daum, F., Huang, J., Noushin, A.: Coulomb’s law particle flow for nonlinear filters. In: Drummond, O.E. (ed.) SPIE Optical Engineering + Applications, pp. 1–15. International Society for Optics and Photonics (2011)Google Scholar
  14. 14.
    Fortmann, T., Bar-Shalom, Y., Scheffe, M.: Sonar tracking of multiple targets using joint probabilistic data association. IEEE J. Oceanic Eng. 8(3), 173–184 (1983)CrossRefGoogle Scholar
  15. 15.
    Gatica-Perez, D., Lathoud, G., Odobez, J.M., McCowan, I.: Audiovisual probabilistic tracking of multiple speakers in meetings. IEEE Trans. Audio Speech Lang. Process. 15(2), 601–616 (2007)CrossRefGoogle Scholar
  16. 16.
    Hafner, J., Sawhney, H.S., Equitz, W., Flickner, M., Niblack, W.: Efficient color histogram indexing for quadratic form distance functions. IEEE Trans. Pattern Anal. Mach. Intell. 17(7), 729–736 (1995)CrossRefGoogle Scholar
  17. 17.
    Khan, M.A., Ulmke, M.: Non-linear and non-Gaussian state estimation using log-homotopy based particle flow filters. In: 2014 Workshop on Sensor Data Fusion: Trends, Solutions, Applications, SDF (2014)Google Scholar
  18. 18.
    Khan, M.A., Ulmke, M.: Non-linear and non-Gaussian state estimation using log-homotopy based particle flow filters. In: Sensor Data Fusion: Trends, Solutions, Applications (SDF), pp. 1–6. IEEE (2014)Google Scholar
  19. 19.
    Kidron, E., Schechner, Y.Y., Elad, M.: Cross-modal localization via sparsity. IEEE Trans. Sig. Process. 55(4), 1390–1404 (2007)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Kilic, V., Barnard, M., Wang, W., Hilton, A., Kittler, J.: Mean-shift and sparse sampling based SMC-PHD filtering for audio informed visual speaker tracking. IEEE Trans. Multimedia 18(12), 2417–2431 (2016)CrossRefGoogle Scholar
  21. 21.
    Kılıç, V., Barnard, M., Wang, W., Kittler, J.: Audio assisted robust visual tracking with adaptive particle filtering. IEEE Trans. Multimedia 17(2), 186–200 (2015)CrossRefGoogle Scholar
  22. 22.
    Kong, A., Liu, J.S., Wong, W.H.: Sequential imputations and Bayesian missing data problems. J. Am. Stat. Assoc. 89(425), 278–288 (1994)CrossRefzbMATHGoogle Scholar
  23. 23.
    Lathoud, G., Odobez, J.-M., Gatica-Perez, D.: AV16.3: an audio-visual corpus for speaker localization and tracking. In: Bengio, S., Bourlard, H. (eds.) MLMI 2004. LNCS, vol. 3361, pp. 182–195. Springer, Heidelberg (2005). doi: 10.1007/978-3-540-30568-2_16 CrossRefGoogle Scholar
  24. 24.
    Li, Y., Zhao, L., Coates, M.: Particle flow for particle filtering. In: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 3979–3983. IEEE (2016)Google Scholar
  25. 25.
    Liu, Q., Rui, Y., Gupta, A., Cadiz, J.J.: Automating camera management for lecture room environments. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 442–449. ACM (2001)Google Scholar
  26. 26.
    Maggio, E., Piccardo, E., Regazzoni, C., Cavallaro, A.: Particle PHD filtering for multi-target visual tracking. In: 2007 IEEE International Conference on Acoustics, Speech and Signal Processing, vol. 1, pp. I–1101. IEEE (2007)Google Scholar
  27. 27.
    Polat, E., Ozden, M.: A nonparametric adaptive tracking algorithm based on multiple feature distributions. IEEE Trans. Multimedia 8(6), 1156–1163 (2006)CrossRefGoogle Scholar
  28. 28.
    Ristic, B., Vo, B.N., Clark, D., Vo, B.T.: A metric for performance evaluation of multi-target tracking algorithms. IEEE Trans. Sig. Process. 59(7), 3452–3457 (2011)MathSciNetCrossRefGoogle Scholar
  29. 29.
    Talantzis, F., Constantinides, A.G., Polymenakos, L.C.: Estimation of direction of arrival using information theory. IEEE Sig. Process. Lett. 12(8), 561–564 (2005)CrossRefGoogle Scholar
  30. 30.
    Zhao, L., Wang, J., Li, Y., Coates, M.J.: Gaussian particle flow implementation of PHD filter. In: SPIE Defense + Security, p. 98420D (2016)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Yang Liu
    • 1
    Email author
  • Wenwu Wang
    • 1
  • Jonathon Chambers
    • 2
  • Volkan Kilic
    • 3
  • Adrian Hilton
    • 1
  1. 1.Department of Electrical and Electronic EngineeringUniversity of SurreyGuildfordUK
  2. 2.School of Electrical and Electronic EngineeringNewcastle UniversityNewcastle upon TyneUK
  3. 3.Department of Electrical and Electronics EngineeringIzmir Katip Celebi UniversityCigli-IzmirTurkey

Personalised recommendations