PCA-Enhanced Stochastic Optimization Methods

  • Alina Kuznetsova
  • Gerard Pons-Moll
  • Bodo Rosenhahn
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7476)


In this paper, we propose to enhance particle-based stochastic optimization methods (SO) by using Principal Component Analysis (PCA) to build an approximation of the cost function in a neighborhood of particles during optimization. Then we use it to shift the samples in the direction of maximum cost change. We provide theoretical basis and experimental results showing that such enhancement improves the performance of existing SO methods significantly. In particular, we demonstrate the usefulness of our method when combined with standard Random Sampling, Simulated Annealing and Particle Filter.


Cost Function Simulated Annealing Tracking Error Particle Filter Stochastic Optimization 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bentley, J.L.: Multidimensional binary search trees used for associative searching. Commun. ACM 18(9), 509–517 (1975)MathSciNetzbMATHCrossRefGoogle Scholar
  2. 2.
    Bray, M., Koller-Meier, E., Van Gool, L.: Smart particle filtering for 3D hand tracking. In: Proceedings of Sixth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 675–680. IEEE (2004)Google Scholar
  3. 3.
    Cham, T., Rehg, J.M.: A multiple hypothesis approach to figure tracking. In: CVPR (1999)Google Scholar
  4. 4.
    Choo, K., Fleet, D.J.: People tracking using hybrid monte carlo filtering. In: International Conference on Computer Vision, vol. 2, pp. 321–328 (2001)Google Scholar
  5. 5.
    Deutscher, J., Reid, I.: Articulated body motion capture by stochastic search. IJCV 61(2), 185–205 (2005)CrossRefGoogle Scholar
  6. 6.
    Gall, J., Potthoff, J., Schnorr, C., Rosenhahn, B., Seidel, H.: Interacting and annealing particle filters: Mathematics and a recipe for applications. Journal of Mathematical Imaging and Vision 28, 1–18 (2007)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Hansen, N., Ostermeier, A.: Adapting arbitrary normal mutation distributions in evolution strategies: The covariance matrix adaptation. In: Proceedings of IEEE International Conference on Evolutionary Computation, pp. 312–317. IEEE (1996)Google Scholar
  8. 8.
    Igel, C., Hansen, N., Roth, S.: Covariance matrix adaptation for multi-objective optimization. Evolutionary Computation 15(1), 1–28 (2007)CrossRefGoogle Scholar
  9. 9.
    Isard, M., Blake, A.: Condensation - conditional density propagation for visual tracking. International Journal of Computer Vision 29, 5–28 (1998)CrossRefGoogle Scholar
  10. 10.
    Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of IEEE International Conference on Neural Networks, vol. 4, pp. 1942–1948 (November/December 1995)Google Scholar
  11. 11.
    Kirkpatrick, S., Gelatt Jr., C.D., Vecchi, M.P.: Optimization by simulated annealing. Science 220, 671–680 (1983)MathSciNetzbMATHCrossRefGoogle Scholar
  12. 12.
    Kreinin, A., Merkoulovitch, L., Rosen, D., Zerbs, M.: Principal component analysis in quasi monte carlo simulation. Algo Research Quarterly 1(2), 21–30 (1998)Google Scholar
  13. 13.
    Martin, O., Otto, S.W., Felten, E.W.: Large-step Markov chains for the traveling salesman problem (1991)Google Scholar
  14. 14.
    Neal, R.M.: Mcmc using hamiltonian dynamics. In: Handbook of Markov Chain Monte Carlo, vol. 54, pp. 113–162 (2010)Google Scholar
  15. 15.
    Nummiaro, K., Koller-Meier, E., Van Gool, L.: An adaptive color-based particle filter. Image and Vision Computing 21(1), 99–110 (2003)CrossRefGoogle Scholar
  16. 16.
    Sminchisescu, C., Triggs, B.: Covariance scaled sampling for monocular 3D body tracking. In: CVPR, vol. 1 (2001)Google Scholar
  17. 17.
    Vandekerckhove, J.: General simulated annealing algorithm (2006),

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Alina Kuznetsova
    • 1
  • Gerard Pons-Moll
    • 1
  • Bodo Rosenhahn
    • 1
  1. 1.Institute for Information Processing (TNT)Leibniz University HanoverGermany

Personalised recommendations