Machine Vision and Applications

, Volume 29, Issue 3, pp 525–541 | Cite as

Al-Thocb HSC: a harmony search algorithm for automated calibration of industrial equipment

Adapted harmony search algorithm for camera calibration
  • Pierre Willaume
  • Pierre Parrend
  • Etienne Gancel
  • Aline Deruyver
Original Paper


Calibration is an essential task for setting up camera parameters, especially when cameras are used for industrial applications like object recognition and picking that require a fine-grained location of the observed object. However, this process is time-consuming and requires specific image processing skills, which are not always available: an operator often needs to use the equipment rapidly without costly setup operations. The calibration system needs a coherent set of images of a given model, called a mire, positioned in different ways. In this paper, we propose to automate and to optimize the calibration system by eliminating the requirement for the user to select a suitable set of images. Thus, an optimized calibration can be obtained in a minimum of time. First, we propose to retrieve the set of points of each input image in order to avoid a renewed search at each calibration. Second, we define Al-Thocb, a Harmony Search Calibration algorithm, based on Harmony Search Optimization. The algorithm optimizes the selection of the best images. The satisfaction criterion is defined by a fitness function based on the projection error. The method allows to retrieve coherent camera parameters with no need for specific user skills. It also significantly improves the accuracy of calibration through the use of the reprojection error as fitness function. To demonstrate the applicability of Al-Thocb, we evaluate the accuracy and the responsiveness of the proposed algorithm and compare it to other existing methods.


Camera calibration Optimization Harmony search Evolutionary algorithms Industry and manufacturing Automation 



This research was supported by Hager enterprise. We would also thank the ECAM-Strasbourg Europe for valuable comments and discussions.


  1. 1.
    Aguilar, J.J., Torres, F., Lope, M.: Stereo vision for 3d measurement: accuracy analysis, calibration and industrial applications. Measurement 18(4), 193–200 (1996)CrossRefGoogle Scholar
  2. 2.
    Al-salami, N.M.: Evolutionary algorithm definition. Am. J. Eng. Appl. Sci. 2(4), 789–795 (2009)CrossRefGoogle Scholar
  3. 3.
    Angeline, P.J.: A historical perspective on the evolution of executable structures. Fundam. Inform. 35(1–4), 179–195 (1998)MATHGoogle Scholar
  4. 4.
    Ayache, N., Lustman, F.: Trinocular stereovision for robotics. IEEE Trans. Pattern Anal. Mach. Intell. 13(1), (1991)Google Scholar
  5. 5.
    Broggi, A., Caraffi, C., Fedriga, RI., Grisleri, P.: Obstacle detection with stereo vision for off-road vehicle navigation. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05)-Workshops. IEEE (2005).
  6. 6.
    Darwin, C.: The Origin of Species. John Murray, London (1859)Google Scholar
  7. 7.
    Eberhart, R.C., Shi, Y.: Comparison between genetic algorithms and particle swarm optimization. In: International Conference on Evolutionary Programming, pp. 611–616 . Springer, Heidelberg (1998)Google Scholar
  8. 8.
    Geem, Z.W.: Harmony search algorithm for solving sudoku. In: International Conference on Knowledge-Based and Intelligent Information and Engineering Systems, pp. 371–378. Springer, Heidelberg (2007)Google Scholar
  9. 9.
    Geem, Z.W.: State-of-the-art in the structure of harmony search algorithm. In: Geem, Z.W. (ed.) Recent Advances In Harmony Search Algorithm. Studies in Computational Intelligence, vol. 270. Springer, Berlin, Heidelberg (2010)Google Scholar
  10. 10.
    Geem, Z.W., Kim, J.H., Loganathan, G.: A new heuristic optimization algorithm: harmony search. Simulation 76(2), 60–68 (2001)CrossRefGoogle Scholar
  11. 11.
    Goldberg, D.E., Holland, J.H.: Genetic algorithms and machine learning. Mach. Learn. 3(2), 95–99 (1988)CrossRefGoogle Scholar
  12. 12.
    Guermeur, P., Louchet, J.: An evolutionary algorithm for camera calibration. In: Proceeding of International Conference on Robotics Distance Learning & Intelligence Communication Systems (ICRODIC’03), pp. 799–804. Rethymno, Greece (2003)Google Scholar
  13. 13.
    Heikkila, J., Silvén, O.: A four-step camera calibration procedure with implicit image correction. In: 1997 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1997 Proceedings, pp. 1106–1112. IEEE, (1997)Google Scholar
  14. 14.
    Ikeuchi, K.: Generating an interpretation tree from a CAD model for 3d-object recognition in bin-picking tasks. Int. J. Comput. Vis. 1(2), 145–165 (1987)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Izaguirre, A.: Contribution à l’intégration de la vision dans la commande d’un robot. Ph.D. Thesis, Université Paul Sabatier, Toulouse (1983)Google Scholar
  16. 16.
    Jeddi, B., Vahidinasab, V.: A modified harmony search method for environmental/economic load dispatch of real-world power systems. Energy Convers. Manag. 78, 661–675 (2014)CrossRefGoogle Scholar
  17. 17.
    Kim, B.I., Choi, J.W., Na, K.J., Ha, S.Y.: Calibration method of stereo camera for vehicle. In: Proceedings of the International Conference on Image Processing, Computer Vision, and Pattern Recognition (IPCV). The Steering Committee of The World Congress in Computer Science, Computer Engineering and Applied Computing (WorldComp), p. 357 (2016)Google Scholar
  18. 18.
    Kumar, R., Kumar, S., Lal, S., Chand, P.: Object detection and recognition for a pick and place robot. In: 2014 Asia-Pacific World Congress on Computer Science and Engineering (APWC on CSE). IEEE, pp. 1–7 (2014)Google Scholar
  19. 19.
    Mahdavi, M., Fesanghary, M., Damangir, E.: An improved harmony search algorithm for solving optimization problems. Appl. Math. Comput. 188(2), 1567–1579 (2007)MathSciNetMATHGoogle Scholar
  20. 20.
    Masaki, I.: Machine-vision systems for intelligent transportation systems. IEEE Intell. Syst. 13(6), 24–31 (1998)CrossRefGoogle Scholar
  21. 21.
    Mitchell, M.: An Introduction to Genetic Algorithms. MIT Press, Cambridge (1998)MATHGoogle Scholar
  22. 22.
    Moh’d Alia, O., Mandava, R.: The variants of the harmony search algorithm : an overview. Artif. Intell. Rev. 36(1), 49–68 (2011)CrossRefGoogle Scholar
  23. 23.
    Murray, D., Little, J.J.: Using real-time stereo vision for mobile robot navigation. Auton. Robots 8(2), 161–171 (2000)CrossRefGoogle Scholar
  24. 24.
    Ni, W.F., Wei, S.C., Lin, T., Chen, S.B.: A self-calibration algorithm with chaos particle swarm optimization for autonomous visual guidance of welding robot. In: Tarn, T.J., Chen, S.B., Chen, X.Q. (eds.) Robotic Welding, Intelligence and Automation. Advances in Intelligent Systems and Computing, vol. 363. Springer, Cham (2015)Google Scholar
  25. 25.
    Rupp, S., Elter, M., Breitung, M., Zink, W., Kueblbeck, C.: Robust camera calibration using discrete optimization. Int. J. Appl. Sci. 13, 250–254 (2006)Google Scholar
  26. 26.
    Saka, M.P.: Optimum Design of Steel Skeleton Structures, pp. 87–112. Springer, Berlin (2009)Google Scholar
  27. 27.
    Streichert, F.: Introduction to evolutionary algorithms. Paper to be presented 4 April (2002)Google Scholar
  28. 28.
    Sturm, P.F., Maybank, S.J.: On plane-based camera calibration: a general algorithm, singularities, applications. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1. IEEE (1999)Google Scholar
  29. 29.
    Tsai, R.: A versatile camera calibration technique for high-accuracy 3d machine vision metrology using off-the-shelf tv cameras and lenses. IEEE J. Robot. Autom. 3(4), 323–344 (1987)CrossRefGoogle Scholar
  30. 30.
    Ulrich, M., Wiedemann, C., Steger, C.: Combining scale-space and similarity-based aspect graphs for fast 3d object recognition. IEEE Trans. Pattern Anal. Mach. Intell. 34(10), 1902–1914 (2012)CrossRefGoogle Scholar
  31. 31.
    Wang, C.M., Huang, Y.F.: Self-adaptive harmony search algorithm for optimization. Expert Syst. Appl. 37(4), 2826–2837 (2010)CrossRefGoogle Scholar
  32. 32.
    Willaume, P., Parrend, P., Gancel, E., Deruyver, A.: The graph matching optimization methodology for thin object recognition in pick and place tasks. In: 2016 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 1–8. IEEE (2016)Google Scholar
  33. 33.
    Willaume, P., Parrend, P., Gancel, E., Deruyver, A.: Skeletonization and 3d graph approach for thin objects recognition in pick and place tasks. In: IAPR International Conference on Machine Vision Applications, pp. 1–4 (2017)Google Scholar
  34. 34.
    Yildiz, A.R., Öztürk, F.: Hybrid taguchi-harmony search approach for shape optimization. In: Recent Advances in Harmony Search Algorithm, pp. 89–98. Springer, Berlin (2010)Google Scholar
  35. 35.
    Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  • Pierre Willaume
    • 1
    • 2
    • 3
    • 4
  • Pierre Parrend
    • 1
    • 2
    • 4
  • Etienne Gancel
    • 2
    • 3
  • Aline Deruyver
    • 1
    • 4
  1. 1.ICube LaboratoryUniversité de StrasbourgStrasbourgFrance
  2. 2.ECAM Strasbourg-EuropeSchiltigheimFrance
  3. 3.Hager GroupObernaiFrance
  4. 4.Complex System Digital Campus (UNESCO Unitwin)ParisFrance

Personalised recommendations