Applied Intelligence

, Volume 28, Issue 1, pp 83–99 | Cite as

Learning-enhanced simulated annealing: method, evaluation, and application to lung nodule registration

  • Shaohua SunEmail author
  • Feng Zhuge
  • Jarrett Rosenberg
  • Robert M. Steiner
  • Geoffrey D. Rubin
  • Sandy Napel


Simulated Annealing (SA) is a popular global minimization method. Two weaknesses are associated with standard SA: firstly, the search process is memory-less and therefore can not avoid revisiting regions that are less likely to contain global minimum; and secondly the randomness in generating a new trial does not utilize the information gained during the search and therefore, the search can not be guided to more promising regions. In this paper, we present the Learning-Enhanced Simulated Annealing (LESA) method to overcome these two difficulties. It adds a Knowledge Base (KB) trial generator, which is combined with the usual SA trial generator to form the new trial for a given temperature. LESA does not require any domain knowledge and, instead, initializes its knowledge base during a “burn-in” phase using random samples of the search space, and, following that, updates the knowledge base at each iteration. This method was applied to 9 standard test functions and a clinical application of lung nodule registration, resulting in superior performance compared to SA. For the 9 test functions, the performance of LESA was significantly better than SA in 8 functions and comparable in 1 function. For the lung nodule registration application, the residual error of LESA was significantly smaller than that produced by a recently published SA system, and the convergence time was significantly faster (9.3±3.2 times). We also give a proof of LESA’s ergodicity, and discuss the conditions under which LESA has a higher probability of converging to the true global minimum compared to SA at infinite annealing time.


Simulated annealing Knowledge base Guided search Search history memorization 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Sun S, Rubin GD, Paik D, Steiner RM, Napel S (2007) Registration of lung nodules using a semi-rigid model: method and preliminary results. Med Phys 34(2):613–626 CrossRefGoogle Scholar
  2. 2.
    Thompson DR, Bilbro GL (2005) Sample-sort simulated annealing. IEEE Trans Syst Man Cybern Part B Cybern 35(3):625–632 CrossRefGoogle Scholar
  3. 3.
    Metropolis N, Rosenbluth AW, Rosenbluth MN, Teller AH, Teller E (1953) Equation of state calculations by fast computing machines. J Chem Phys 21(6):1087–1092 CrossRefGoogle Scholar
  4. 4.
    Kirkpatrick S, Gelatt CD, Vecchi MP (1983) Optimization by simulated annealing. Science 220:671–680 CrossRefMathSciNetGoogle Scholar
  5. 5.
    Cerny V (1985) A thermodynamical approach to the travelling salesman problem: an efficient simulation algorithm. J Optim Theory Appl 45:41–55 zbMATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Ingber L (1989) Very fast simulated re-annealing. Math Comput Model 12:967–973 zbMATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    Ingber L, Rosen B (1992) Genetic algorithms and very fast simulated reannealing: a comparison. Math Comput Model 16(11):87–100 zbMATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    Selim SZ, Alsultan K (1991) A simulated annealing algorithm for the clustering problem. Pattern Recognit 24(10):1003–1008 CrossRefMathSciNetGoogle Scholar
  9. 9.
    Mantawy AH, Abdel-Magid YL, Selim SZ (1998) A simulated annealing algorithm for unit commitment. IEEE Trans Power Syst 13(1):197–204 CrossRefGoogle Scholar
  10. 10.
    Yang CC, Yen J, Chen H (1998) Intelligent Internet searching engine based on hybrid simulated annealing. In: Proceedings of the thirty-first Hawaii international conference on system sciences, vol 4, pp 415–422 Google Scholar
  11. 11.
    Hirano T, Okada Y, Yoda F (1997) Structural character recognition using simulated annealing. In: Proceedings of the fourth international conference on document analysis and recognition, vol 2, pp 507–510 Google Scholar
  12. 12.
    Betke M, Makris NC (1995) Fast object recognition in noisy images using simulated annealing. In: Proceedings of the international conference on computer vision, vol 1, pp 523–530 Google Scholar
  13. 13.
    Fan Z, Zhou J, Wu Y (2004) Multibody motion segmentation based on simulated annealing. In: 2004 IEEE computer society conference on computer vision and pattern recognition (CVPR’04), vol 1, June 2004, pp 776–781 Google Scholar
  14. 14.
    Gu J, Toumoulin C, Shu H (2003) Spatio-temporal registration in coronary angiography. In: Proceedings of the 25th annual international conference of the IEEE engineering in medicine and biology society, vol 1, September 2003, pp 584–587 Google Scholar
  15. 15.
    Jolly MPD, Lakshmanan S, Jain AK (1996) Vehicle segmentation and classification using deformable templates. IEEE Trans Pattern Anal Mach Intell 18(3):293–308 CrossRefGoogle Scholar
  16. 16.
    Maglogiannis I, Zafiropoulos E (2004) Automated medical image registration using the simulated annealing algorithm. Lect Notes Comput Sci 3025:456–465 CrossRefGoogle Scholar
  17. 17.
    Mahfouz MR, Hoff WA, Komistek RD, Dennis DA (2003) A robust method for registration of three-dimensional knee implant models to two-dimensional fluoroscopy images. IEEE Trans Med Imaging 22(12):1561–1574 CrossRefGoogle Scholar
  18. 18.
    Pajares G, de la Cruz JM (2004) On combining support vector machines and simulated annealing in stereovision matching. IEEE Trans Syst Man Cybern Part B 34(4):1646–1657 CrossRefGoogle Scholar
  19. 19.
    Tandjung SS, Gunawan TS, Nang CM (2000) Motion estimation using adaptive blocksize observation model and efficient multiscale regularization. In: Proceedings the international conference on image processing, vol 2, pp 566–569 Google Scholar
  20. 20.
    Tubic D, Zaccarin A, Pouliot J (2000) Automated prostate implant seed localization from fluoroscopic images using simulated annealing. In: Proceedings of the 22nd annual international conference of the IEEE engineering in medicine and biology society, vol 4, pp 2782–2786 Google Scholar
  21. 21.
    Hajek B (1988) Cooling schedules for optimal annealing. Math Oper Res 13(2):311–329 zbMATHMathSciNetCrossRefGoogle Scholar
  22. 22.
    Catoni O (1992) Rough large deviation estimates for simulated annealing: application to exponential schedules. Ann Probab 20:1109–1146 zbMATHCrossRefMathSciNetGoogle Scholar
  23. 23.
    Ingber L (1996) Adaptive simulated annealing (ASA): lessons learned. Cont Cybernet 25:33–54, Special issue on simulated annealing applied to combinatorial optimization zbMATHGoogle Scholar
  24. 24.
    Junior HA (2003) Fuzzy control of stochastic global optimization algorithms and very fast simulated reannealing. Optimization OnLine Google Scholar
  25. 25.
    Yang S, Machado JM, Ni G, Ho SL, Zhou P (2000) A self-learning simulated annealing algorithm for global optimizations of electromagnetic devices. IEEE Trans Magn 36(4):1004–1008 CrossRefGoogle Scholar
  26. 26.
    Yip P, Pao Y-H (1994) A guided evolutionary simulated annealing approach to the quadraticassignment problem. IEEE Trans Syst Man Cybern 24(9):1383–1386 CrossRefGoogle Scholar
  27. 27.
    Chou C, Lee T (2002) A guided simulated annealing method for crystallography. Acta Crystallogr A 58:42–46 CrossRefGoogle Scholar
  28. 28.
    Chou CI, Han RS, Li SP, Lee TK (2003) Guided simulated annealing method for optimization problems. Phys Rev E 67(6):066704 CrossRefGoogle Scholar
  29. 29.
    Wong KP, Wong YW (1997) Hybrid genetic/simulated annealing approach to short-term multiple-fuel-constrained generation scheduling. IEEE Trans Power Syst 12(2):776–784 CrossRefGoogle Scholar
  30. 30.
    Moh J-S, Chiang D-Y (2000) Improved simulated annealing search for structural optimization. AIAA J 38(10):1965–1973 Google Scholar
  31. 31.
    Luck J, Little C, Hoff W (2000) Registration of range data using a hybrid simulated annealing and iterative closest point algorithm. In: Proceedings of the IEEE international conference on robotics and automation, vol 4, April 2000, pp 3739–3744 Google Scholar
  32. 32.
    Purushothama GK, Jenkins L (2003) Simulated annealing with local search a hybrid algorithm for unit commitment. IEEE Trans Power Syst 18(1):273–278 CrossRefGoogle Scholar
  33. 33.
    Rammal R, Toulouse G, Virasoro MA (1986) Ultrametricy for physicists. Rev Mod Phys 58(3) Google Scholar
  34. 34.
    Bracewell R (1999) The Fourier transform and its applications, 3rd edn. McGraw–Hill, New York Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  • Shaohua Sun
    • 1
    Email author
  • Feng Zhuge
    • 1
  • Jarrett Rosenberg
    • 2
  • Robert M. Steiner
    • 2
  • Geoffrey D. Rubin
    • 2
  • Sandy Napel
    • 2
  1. 1.Electrical EngineeringStanford UniversityStanfordUSA
  2. 2.RadiologyStanford UniversityStanfordUSA

Personalised recommendations