Journal of Mathematical Imaging and Vision

, Volume 48, Issue 3, pp 451–466 | Cite as

An Adversarial Optimization Approach to Efficient Outlier Removal

  • Jin Yu
  • Anders Eriksson
  • Tat-Jun Chin
  • David Suter


This paper proposes a novel adversarial optimization approach to efficient outlier removal in computer vision. We characterize the outlier removal problem as a game that involves two players of conflicting interests, namely, model optimizer and outliers. Such an adversarial view not only brings new insights into some existing methods, but also gives rise to a general optimization framework that provably unifies them. Under the proposed framework, we develop a new outlier removal approach that is able to offer a much needed control over the trade-off between reliability and speed, which is usually not available in previous methods. Underlying the proposed approach is a mixed-integer minmax (convex-concave) problem formulation. Although a minmax problem is generally not amenable to efficient optimization, we show that for some commonly used vision objective functions, an equivalent Linear Program reformulation exists. This significantly simplifies the optimization. We demonstrate our method on two representative multiview geometry problems. Experiments on real image data illustrate superior practical performance of our method over recent techniques.


Mixed-integer optimization Convex relaxation Model fitting Outlier removal 


  1. 1.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981) CrossRefMathSciNetGoogle Scholar
  2. 2.
    Alizadeh, F., Goldfarb, D.: Second-order cone programming. Math. Program. 95(1), 3–51 (2003) CrossRefMATHMathSciNetGoogle Scholar
  3. 3.
    Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge (2004) CrossRefMATHGoogle Scholar
  4. 4.
    Enqvist, O., Josephson, K., Kahl, F.: Optimal correspondences from pairwise constraints. In: ICCV (2009) Google Scholar
  5. 5.
    Enqvist, O., Olsson, C., Kahl, F.: Stable structure from motion using rotational consistency. Tech. rep., Lund University (2011) Google Scholar
  6. 6.
    Govindu, V.: Robustness in motion averaging. In: ACCV (2006) Google Scholar
  7. 7.
    Hartley, R., Kahl, F.: Optimal algorithms in multiview geometry. In: ACCV (2007) Google Scholar
  8. 8.
    Hartley, R., Schaffalitzky, F.: L minimization in geometric reconstruction problems. In: CVPR (2004) Google Scholar
  9. 9.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2003) Google Scholar
  10. 10.
    Huber, P.J.: Robust estimation of a location parameter. Ann. Math. Stat. 35(1), 73–101 (1964) CrossRefMATHGoogle Scholar
  11. 11.
    Kahl, F., Hartley, R.: Multiple-view geometry under the L -norm. IEEE Trans. Pattern Anal. Mach. Intell. 30(9), 1603–1617 (2008) CrossRefGoogle Scholar
  12. 12.
    Ke, Q., Kanade, T.: Quasiconvex optimization for robust geometric reconstruction. In: ICCV (2005) Google Scholar
  13. 13.
    Lee, K., Meer, P., Park, R.: Robust adaptive segmentation of range images. IEEE Trans. Pattern Anal. Mach. Intell. 20(2), 200–205 (1998) CrossRefGoogle Scholar
  14. 14.
    Li, H.: A practical algorithm for L triangulation with outliers. In: CVPR (2007) Google Scholar
  15. 15.
    Li, H.: Consensus set maximization with guaranteed global optimality for robust geometry estimation. In: ICCV (2009) Google Scholar
  16. 16.
    Lobo, M., Vandenberghe, L., Boyd, S., Lebret, H.: Applications of second-order cone programming. Linear Algebra Appl. 284(1–3), 193–228 (1998) CrossRefMATHMathSciNetGoogle Scholar
  17. 17.
    Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004) CrossRefGoogle Scholar
  18. 18.
    Monteiro, R., Adler, I.: Interior path following primal-dual algorithms. Part I: Linear programming. Math. Program. 44(1), 27–41 (1989) CrossRefMATHMathSciNetGoogle Scholar
  19. 19.
    Nguyen, T., Welsch, R.: Outlier detection and least trimmed squares approximation using semi-definite programming. Comput. Stat. Data Anal. 54(12), 3212–3226 (2010) CrossRefMathSciNetGoogle Scholar
  20. 20.
    Nistér, D.: An efficient solution to the five-point relative pose problem. IEEE Trans. Pattern Anal. Mach. Intell. 26(6), 756–770 (2004) CrossRefGoogle Scholar
  21. 21.
    Nistér, D.: Preemptive RANSAC for live structure and motion estimation. Mach. Vis. Appl. 16(5), 321–329 (2005) CrossRefGoogle Scholar
  22. 22.
    Olsson, C., Enqvist, O.: Stable structure from motion for unordered image collections. In: Scandinavian Conference on Image Analysis (2011) Google Scholar
  23. 23.
    Olsson, C., Eriksson, A., Hartley, R.: Outlier removal using duality. In: CVPR (2010) Google Scholar
  24. 24.
    Rockafellar, R.: Convex Analysis. Princeton University Press, Princeton (1997) MATHGoogle Scholar
  25. 25.
    Rousseeuw, P., Van Driessen, K.: Computing LTS regression for large data sets. Data Min. Knowl. Discov. 12(1), 29–45 (2006) CrossRefMathSciNetGoogle Scholar
  26. 26.
    Rousseeuw, P.J., Leroy, A.M.: Robust Regression and Outlier Detection. Wiley, New York (1987) CrossRefMATHGoogle Scholar
  27. 27.
    Russell, S., Norvig, P.: Artificial Intelligence: A Modern Approach, 2nd edn. Prentice Hall, New York (2003) Google Scholar
  28. 28.
    Seo, Y., Lee, H., Lee, S.: Outlier removal by convex optimization for L-infinity approaches. In: Advances in Image and Video Technology, pp. 203–214 (2009) CrossRefGoogle Scholar
  29. 29.
    Sim, K., Hartley, R.: Removing outliers using the L norm. In: CVPR (2006) Google Scholar
  30. 30.
    Triggs, B., McLauchlan, P., Hartley, R., Fitzgibbon, A.: Bundle adjustment—a modern synthesis. In: Vision Algorithms: Theory and Practice, pp. 153–177 (2000) CrossRefGoogle Scholar
  31. 31.
    Yu, J., Eriksson, A., Chin, T.J., Suter, D.: An adversarial optimization approach to efficient outlier removal. In: ICCV (2011) Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Jin Yu
    • 1
  • Anders Eriksson
    • 1
  • Tat-Jun Chin
    • 1
  • David Suter
    • 1
  1. 1.Australian Centre for Visual Technologies, School of Computer ScienceUniversity of AdelaideNorth TerraceAustralia

Personalised recommendations