Skip to main content
SpringerLink
Log in

A reinforcement learning hyper-heuristic in multi-objective optimization with application to structural damage identification

  • Research Paper
  • Published:
Structural and Multidisciplinary Optimization Aims and scope Submit manuscript

Abstract

Multi-objective optimization allows satisfying multiple decision criteria concurrently, and generally yields multiple solutions. It has the potential to be applied to structural damage identification applications which are oftentimes under-determined. How to achieve high-quality solutions in terms of accuracy, diversity, and completeness is a challenging research subject. The solution techniques and parametric selections are believed to be problem specific. In this research, we formulate a reinforcement learning hyper-heuristic scheme to work coherently with the single-point search algorithm MOSA/R (Multi-Objective Simulated Annealing Algorithm based on Re-seed). The four low-level heuristics proposed can meet various optimization requirements adaptively and autonomously using the domination amount, crowding distance, and hypervolume calculations. The new approach exhibits improved and more robust performance than AMOSA, NSGA-II, and MOEA/D when applied to benchmark test cases. It is then applied to an active damage interrogation scheme for structural damage identification where solution diversity/completeness and accuracy are critically important. Results show that this approach can successfully include the true damage scenario in the solution set identified. The outcome of this research can potentially be extended to a variety of applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  • Antunes CH, Lima P, Oliveira E, Pires DF (2011) A multi-objective simulated annealing approach to reactive power compensation. Eng Optim 43(10):1063–1077

    Google Scholar 

  • Bai R, Blazewicz J, Burke EK, Kendall G, McCollum B (2012) A simulated annealing hyper-heuristic methodology for flexible decision support. 4OR 10(1):43–66

    MATH  Google Scholar 

  • Bandyopadhyay S, Saha S, Maulik U, Deb K (2008) A simulated annealing-based multiobjective optimization algorithm: AMOSA. IEEE Trans Evol Comput 12(3):269–283

    Google Scholar 

  • Burke EK, Hyde MR, Kendall G, Ochoa G, Ozcan E, Woodward JR (2009) Exploring hyper-heuristic methodologies with genetic programming. Computational intelligence. Springer, Berlin, pp 177–201

    Google Scholar 

  • Burke EK, Gendreau M, Hyde M, Kendall G, Ochoa G, Özcan E, Qu R (2013) Hyper-heuristics: A survey of the state of the art. Journal of the Operational Research Society 64(12):1695–1724

    Google Scholar 

  • Cao P, Shuai Q, Tang J (2018a) A multi-objective DIRECT algorithm toward structural damage identification with limited dynamic response information. J Nondestruct Evaluat Diagn Progn Eng Syst 1(2):021004

    Google Scholar 

  • Cao P, Qi S, Tang J (2018b) Structural damage identification using piezoelectric impedance measurement with sparse inverse analysis. Smart Mater Struct 27(3):035020

    Google Scholar 

  • Cao P, Fan Z, Gao RX, Tang J (2019) Harnessing multi-objective simulated annealing toward configuration optimization within compact space for additive manufacturing. Robot Comput Integr Manuf 57:29–45

    Google Scholar 

  • Cowling P, Kendall G, Soubeiga E (2000) A hyperheuristic approach to scheduling a sales summit. International Conference on the Practice and Theory of Automated Timetabling. Springer, Berlin, pp 176–190

    Google Scholar 

  • Cha YJ, Buyukozturk O (2015) Structural damage detection using modal strain energy and hybrid multiobjective optimization. Comput Aid Civ Infrastruct Eng 30(5):347–358

    Google Scholar 

  • Deb K, Thiele L, Laumanns M, Zitzler E (2002a) Scalable multi-objective optimization test problems. Congress on Evolutionary Computation. IEEE Press, pp 825–830

    Google Scholar 

  • Deb K, Pratap A, Agarwal S, Meyarivan TAMT (2002b) A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans Evol Comput 6(2):182–197

    Google Scholar 

  • Deb K, Agrawal S, Pratap A, Meyarivan T (2000) A fast elitist non-dominated sorting genetic algorithm for multi-objective optimization: NSGA-II. International conference on parallel problem solving from nature. Springer, Berlin, pp 849–858

    Google Scholar 

  • Dinh-Cong D, Nguyen-Thoi T (2021) An effective damage identification procedure using model updating technique and multi-objective optimization algorithm for structures made of functionally graded materials. Engineering with Computers, in press.

  • Emmerich M, Beume N, Naujoks, B (2005) An EMO algorithm using the hypervolume measure as selection criterion. In: International Conference on Evolutionary Multi-Criterion Optimization, pp 62–76. Springer, Berlin

  • Guizzo G, Fritsche GM, Vergilio SR, Pozo ATR (2015) A hyper-heuristic for the multi-objective integration and test order problem. In Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation, pp 1343–1350. ACM

  • Goldberg DE (1990) Probability matching, the magnitude of reinforcement, and classifier system bidding. Mach Learn 5(4):407–425

    Google Scholar 

  • Gomes GF, Mendéz YAD, da Cunha SS, Ancelotti AC (2018) A numerical–experimental study for structural damage detection in CFRP plates using remote vibration measurements. J Civ Struct Heal Monit 8(1):33–47

    Google Scholar 

  • Hitomi N, Selva D (2015) The effect of credit definition and aggregation strategies on multi-objective hyper-heuristics. In: ASME 2015 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (pp: V02BT03A030-V02BT03A030). American Society of Mechanical Engineers

  • Hitomi N, Selva D (2016) A classification and comparison of credit assignment strategies in multiobjective adaptive operator selection. IEEE Trans Evol Comput 21(2):294–314

    Google Scholar 

  • Ishibuchi H, Sakane Y, Tsukamoto N, Nojima Y (2010) Simultaneous use of different scalarizing functions in MOEA/D. In: Proceedings of the 12th annual conference on Genetic and evolutionary computation, pp 519–526. ACM

  • Ishibuchi H, Masuda H, Tanigaki Y, Nojima Y (2015) Modified distance calculation in generational distance and inverted generational distance. In International conference on evolutionary multi-criterion optimization, pp 110–125. Springer, Cham

  • Kaveh A, Laknejadi K (2013) A new multi-swarm multi-objective optimization method for structural design. Adv Eng Softw 58:54–69

    MATH  Google Scholar 

  • Kirkpatrick S, Gelatt CD, Vecchi MP (1983) Optimization by simulated annealing. Science, New Series 220(4598):671–680

    MATH  Google Scholar 

  • Krempser E, Fialho Á, Barbosa HJ (2012) Adaptive operator selection at the hyper-level. In: International Conference on Parallel Problem Solving from Nature, pp 378–387. Springer, Berlin

  • Li M, Yang S, Liu X (2016) Pareto or non-pareto: bi-criterion evolution in multiobjective optimization. IEEE Trans Evol Comput 20(5):645–665

    Google Scholar 

  • Lu C, Xiao S, Li X, Gao L (2016) An effective multi-objective discrete grey wolf optimizer for a real-world scheduling problem in welding production. Adv Eng Softw 99:161–176

    Google Scholar 

  • Maturana, Fialho, Á, Saubion F, Schoenauer M, Sebag M (2009) Extreme compass and dynamic multi-armed bandits for adaptive operator selection. In: 2009 IEEE Congress on Evolutionary Computation, pp 365–372. IEEE

  • Lipowski A, Lipowska D (2012) Roulette-wheel selection via stochastic acceptance. Physica A 391(6):2193–2196

    Google Scholar 

  • McClymont K, Keedwell EC (2011) Markov chain hyper-heuristic (MCHH): an online selective hyper-heuristic for multi-objective continuous problems. In: Proceedings of the 13th annual conference on genetic and evolutionary computation, pp 2003–2010. ACM

  • Mohd Zain MZ, Kanesan J, Chuah J, Dhanapal S, Kendall G (2018) A multi-objective particle swarm optimization algorithm based on dynamic boundary search for constrained optimization. Appl Soft Comput 70:680–700

    Google Scholar 

  • Maashi M, Kendall G, Özcan E (2015) Choice function based hyper-heuristics for multi-objective optimization. Appl Soft Comput 28:312–326

    Google Scholar 

  • Nareyek A (2003) Choosing search heuristics by non-stationary reinforcement learning. Metaheuristics: Computer decision-making. Springer, Boston, pp 523–544

    Google Scholar 

  • Özcan E, Misir M, Ochoa G, Burke EK (2012) A reinforcement learning: great-deluge hyper-heuristic for examination timetabling. Modeling, analysis, and applications in metaheuristic computing: advancements and trends. IGI Global

    Google Scholar 

  • Qin W, Zhuang Z, Huang Z, Huang H (2021) A novel reinforcement learning-based hyper-heuristic for heterogeneous vehicle routing problem. Comput Ind Eng 156:107252

    Google Scholar 

  • Shuai Q, Zhou K, Zhou S, Tang J (2017) Fault identification using piezoelectric impedance measurement and model-based intelligent inference with pre-screening. Smart Mater Struct 26(4):045007

    Google Scholar 

  • Suman B (2004) Study of simulated annealing based algorithms for multiobjective optimization of a constrained problem. Comput Chem Eng 28(9):1849–1871

    Google Scholar 

  • Suman B, Kumar P (2006) A survey of simulated annealing as a tool for single and multiobjective optimization. J Oper Res Soc 57(10):1143–1160

    MATH  Google Scholar 

  • Smith KI (2006) A study of simulated annealing techniques for multi-objective optimisation. University of Exeter, Thesis

    Google Scholar 

  • Szőllős A, Šmíd M, Hájek J (2009) Aerodynamic optimization via multi-objective micro-genetic algorithm with range adaptation, knowledge-based reinitialization, crowding and ε-dominance. Adv Eng Softw 40(6):419–430

    MATH  Google Scholar 

  • Thierens D (2007) Adaptive strategies for operator allocation. Parameter Setting in Evolutionary Algorithms. Springer, Berlin, pp 77–90

    Google Scholar 

  • Tiachacht S, Bouazzouni A, Khatir S, Wahab MA, Behtani A, Capozucca R (2018) Damage assessment in structures using combination of a modified Cornwell indicator and genetic algorithm. Eng Struct 177:421–430

    Google Scholar 

  • Wang KW, Tang J (2008) Adaptive Structural System with Piezoelectric Transducer Circuitry. Springer

    Google Scholar 

  • Wang WX, Wang X, Ge XL, Deng L (2014) Multi-objective optimization model for multi-project scheduling on critical chain. Adv Eng Softw 68:33–39

    Google Scholar 

  • Ye HL, Zhang Y, Yang QS, Xiao YN, Grandhi RV, Fischer CC (2017) Optimal design of a three tape-spring hinge deployable space structure using an experimentally validated physics-based model. Struct Multidisc Optim 56(5):973–989

    Google Scholar 

  • Ye HL, Zhang Y, Yang QS, Zhang B (2019) Quasi-static analysis and multi-objective optimization for tape spring hinge. Struct Multidisc Optim 60(6):2417–2430

    Google Scholar 

  • Zavala GR, Nebro AJ, Luna F, Coello CAC (2014) A survey of multi-objective metaheuristics applied to structural optimization. Struct Multidisc Optim 49(4):537–558

    Google Scholar 

  • Zavala G, Nebro AJ, Luna F, Coello CAC (2016) Structural design using multi-objective metaheuristics. Comparative study and application to a real-world problem. Struct Multidisc Optim 53(3):545–566

    Google Scholar 

  • Zhang Q, Li H (2007) MOEA/D: a multiobjective evolutionary algorithm based on decomposition. IEEE Trans Evol Comput 11(6):712–731

    Google Scholar 

  • Zhang Q, Zhou A, Zhao S, Suganthan PN, Liu W, Tiwari S (2008) Multiobjective optimization test instances for the CEC 2009 special session and competition. University of Essex, Colchester, UK and Nanyang technological University, Singapore, special session on performance assessment of multi-objective optimization algorithms, technical report, 264

  • Zitzler E, Thiele L (1999) Multiobjective evolutionary algorithms: A comparative case study and the strength Pareto approach. IEEE Trans Evol Comput 3(4):257–271

    Google Scholar 

  • Zitzler E, Brockhoff D, Thiele L (2007) The hypervolume indicator revisited: On the design of Pareto-compliant indicators via weighted integration. International Conference on Evolutionary Multi-Criterion Optimization. Springer, Berlin, pp 862–876

    Google Scholar 

  • Zhou A, Qu BY, Li H, Zhao SZ, Suganthan PN, Zhang Q (2011) Multiobjective evolutionary algorithms: a survey of the state of the art. Swarm Evol Comput 1(1):32–49

    Google Scholar 

  • Zarchi M, Attaran B (2019) Improved design of an active landing gear for a passenger aircraft using multi-objective optimization technique. Struct Multidisc Optim 59(5):1813–1833

    Google Scholar 

  • Zhang S, Ren Z, Li C, Xuan J (2020) A perturbation adaptive pursuit strategy based hyper-heuristic for multi-objective optimization problems. Swarm Evol Comput 54:100647

    Google Scholar 

Download references

Acknowledgements

This research is supported in part by a Space Technology Research Institutes Grant (No. 80NSSC19K1076) from NASA’s Space Technology Research Grants Program and in part by NSF under Grant CMMI-1825324.

Author information

Authors and Affiliations

  1. Department of Mechanical Engineering, University of Connecticut, 191 Auditorium Road, Unit 3139, Storrs, CT, 06269, USA

    Pei Cao, Yang Zhang & J. Tang

  2. Department of Mechanical Engineering-Engineering Mechanics, Michigan Technological University, Houghton, USA

    Kai Zhou

Authors
  1. Pei Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kai Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. J. Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to J. Tang.

Ethics declarations

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Replication of results

All the algorithms and analyses are implemented through MATLAB. All related data including simulation data and experimental data are available from the corresponding author upon request.

Additional information

Responsible Editor: Zhen Hu

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Topical Collection: Advanced Optimization Enabling Digital Twin Technology.

Guest Editors: C. Hu, V. A. González, T. Kim, O. San, Z. Hu, and P. Zheng.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cao, P., Zhang, Y., Zhou, K. et al. A reinforcement learning hyper-heuristic in multi-objective optimization with application to structural damage identification. Struct Multidisc Optim 66, 16 (2023). https://doi.org/10.1007/s00158-022-03432-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00158-022-03432-5

Keywords

Search

Navigation