Evolutionary constrained multi-objective optimization: a review

Abstract

Solving constrained multi-objective optimization problems (CMOPs) is challenging due to the simultaneous consideration of multiple conflicting objectives that need to be optimized and complex constraints that need to be satisfied. To address this class of problems, a large number of constrained multi-objective evolutionary algorithms (CMOEAs) have been designed. This paper presents a comprehensive review of state-of-the-art algorithms for solving CMOPs. First, the background knowledge and concepts of evolutionary constrained multi-objective optimization are presented. Then, some classic constraint handling technologies (CHTs) are introduced, and the advantages and limitations of each CHT are discussed. Subsequently, based on the mechanisms used by these algorithms, the CMOEAs are classified into six categories, each of which is explained in detail. Following that, the benchmark test problems used to evaluate the algorithm’s performance are reviewed. Moreover, the experimental comparison and performance analysis of different types of algorithms are carried out on different test problems with different characteristics. Finally, some of the challenges and future research directions in evolutionary constrained multi-objective optimization are discussed.

1 Introduction

CMOPs are a type of problem widely existing in science and technology life, for example: engineering design problems [1,2,3], scheduling optimization problems [4,5,6], path planning problems [7,8,9], and resource optimization problems [10,11,12]. Such problems often require optimizing multiple conflicting objectives while satisfying complex constraints, and thus a single solution cannot satisfy the tradeoff between multiple objectives. Moreover, due to the existence of the complex fitness landscape, traditional mathematical methods easily get trapped in local optima, making it difficult to explore the best solutions. On the contrary, an evolutionary algorithm is a population-based optimization algorithm with strong search ability and can output a set of tradeoff solutions, which is very suitable for solving various complex optimization problems [13,14,15]. Therefore, researchers tend to use evolutionary algorithms to solve CMOPs, and various types of CMOEAs are gradually increasing.

Generally speaking, a CMOEA typically comprises two integral components: the multi-objective evolutionary algorithm (MOEA) and the CHT. MOEAs play a pivotal role in fostering the evolutionary progression of populations and can be categorically classified into three distinct groups based on their environmental selection mechanisms: indicator-based methods [16,17,18], decomposition-based methods [19,20,21], and dominance-based methods [22,23,24]. These evolutionary algorithms can achieve excellent results in multi-objective problems without constraints, but they lack the capability to handle constraints. Consequently, if MOEAs are directly applied to solve CMOPs, there is a high likelihood of obtaining a set of solutions that do not satisfy the specified constraints. In order to handle constraints, many efforts are dedicated to the design of CHTs, these techniques can make choices between solutions that meet constraints and those that do not [25, 26]. Therefore, they are typically embedded into the environmental selection process of MOEAs to form CMOEAs to address CMOPs. At the same time, to evaluate the capability of the designed CMOEAs in addressing problems, a wide array of benchmark test problem sets have been developed in recent years [27, 28].

Fig. 1

The classification of existing CMOEAs

The burgeoning prevalence of evolutionary constrained multi-objective optimization across diverse application domains has generated heightened scholarly interest in its investigation. This article aims to facilitate a more thorough comprehension of evolutionary constrained multi-objective optimization among researchers. To achieve this objective, the review will focus on various aspects pertinent to evolutionary constrained multi-objective optimization.

• Several classical CHTs are introduced in detail, and the advantages and limitations of each CHT are discussed. The specific discussion of CHTs is expected to help researchers quickly understand how to deal with constraints when solving CMOPs.

• Following the delineation presented in Fig. 1, this study systematically introduces the six categories of CMOEAs individually. The purpose of this exposition is to facilitate researchers in cultivating a more profound comprehension of the prevailing technologies employed for addressing CMOPs. Furthermore, the discussion endeavors to proffer potential guidance for pioneers actively engaged in the resolution of CMOPs.

• The benchmark test problem sets for testing the performance of algorithms are introduced and analyzed, with the aim of promoting performance comparisons between different CMOEAs and assisting researchers in developing new algorithms and designing more diverse test problem sets.

• The performance of different types of multi-objective evolutionary algorithms on constraint test problem sets with different characteristics is investigated, and some solutions for different test sets are given, hoping to provide new ideas for researchers.

• At present, CMOEAs have shown excellent performance in general CMOPs, but as more features gradually emerge in the CMOPs encountered in real life, more challenges also need to be considered, such as CMOPs containing large-scale characteristics, dynamic characteristics, and multi-modal characteristics. Therefore, this paper lists some possible directions for future research to promote more development on evolutionary constrained multi-objective optimization.

The subsequent sections of this paper are structured as follows: Section 2 provides an introduction to the background knowledge concerning evolutionary constrained multi-objective optimization. Section 3 discusses the most prevalent CHTs. Section 4 classifies and introduces CMOEAs. Following this, Sect. 5 outlines well-established benchmark test problem sets. In Sect. 6, the performance of different algorithms is analyzed. Section 7 accentuates potential avenues for future research. Finally, Sect. 8 concludes the article with a summary.

2 Background knowledge

A CMOP is characterized by the presence of multiple objectives to be optimized concurrently, alongside multiple constraints that necessitate satisfaction during the optimization process. Generally, it is expressed using the following mathematical formula [29], in which the definitions of the parameters used are shown in Table 1.

$$\begin{aligned} \begin{array}{l} \min \vec {F}(\vec {x})=\left( f_{1}(\vec {x}), f_{2}(\vec {x}), \ldots , f_{m}(\vec {x})\right) ^{\text {T}} \\ \text{ s.t. } \left\{ \begin{array}{l} g_{i}(\vec {x}) \le 0, i=1, \ldots , l \\ h_{i}(\vec {x})=0, i=1, \ldots , k \\ \vec {x}=\left( x_{1}, x_{2}, \ldots , x_{D}\right) ^{\text {T}} \in \mathbb {R} \end{array}\right. \end{array} \end{aligned}$$

(1)

where $\vec {F}$ denotes the objective vector, encompassing a set of m objective functions that require optimization. $\vec {x}$ represents the decision variable, which contains D dimensions, and the search space is represented by $\mathbb {R}$. $g_{i}(\vec {x})$ and $h_{i}(\vec {x})$ represent the i-th inequality constraint and i-th equality constraint that must be fulfilled, respectively. l and k represent the number of inequality constraints and equality constraints, respectively.

Table 1 Notation used in the problem formulation

For a decision variable $\vec {x}$, the total constraint violation is determined by the sum of its violations with respect to each constraint.

$$\begin{aligned} C V(\vec {x})=\sum \limits _{i=1}^{l+k} cv_{i}(\vec {x}) \end{aligned}$$

(2)

where $cv_{i}$ represents the degree of violation of the decision vector $\vec {x}$ concerning the i-th constraint. This degree of violation is typically computed using the following formula:

$$\begin{aligned} cv_{i}(\vec {x})=\left\{ \begin{array}{ll} \max \left( 0, g_{i}(\vec {x})\right) , &{} i=1, \ldots , l \\ \max \left( 0,\left| h_{i}(\vec {x})\right| -\delta \right) , &{} i=1, \ldots , k \end{array}\right. \end{aligned}$$

(3)

where $\delta$ stands for a minute positive value, often employed to alleviate the strictness of equality constraints. For a decision variable $\vec {x}$, if it adheres to all constraints, its total constraint violation value equates to zero, rendering it a feasible solution. Conversely, if any violation occurs, it is categorized as an infeasible solution.

Pareto dominance: In the context of two feasible solutions, denoted as $\vec {x}_1$ and $\vec {x}_2$, $\vec {x}_1$ is considered to Pareto dominate $\vec {x}_2$ if the condition $f_a(\vec {x}_1) \le f_a(\vec {x}_2)$ holds for every $a \in \{1, ..., m\}$, and additionally, $f_b(\vec {x}_1) < f_b(\vec {x}_2)$ is satisfied for at least one $b \in \{1, ..., m\}$.

Non-dominance: For any two solutions $\vec {x}_1$ and $\vec {x}_2$, they are considered non-dominated by each other if there exists $a \in \{1, ..., m\}$ such that $f_a(\vec {x}_1) \le f_a(\vec {x}_2)$, and simultaneously, there exists $b \in \{1, ..., m\}$ (where $a \ne b$) such that $f_b(\vec {x}_2) \le f_b(\vec {x}_1)$. In other words, neither solution $\vec {x}_1$ Pareto dominates $\vec {x}_2$, nor does $\vec {x}_2$ Pareto dominate $\vec {x}_1$.

Pareto optimal solution: A solution $\vec {x}_1$ is deemed a Pareto optimal solution if there are no other solutions in the decision space $\mathbb {R}$ that Pareto dominate $\vec {x}_1$. Furthermore, it is noteworthy that all Pareto optimal solutions exhibit a non-dominance relationship with each other.

Pareto optimal set: The ensemble of all Pareto optimal solutions is designated as the Pareto optimal set (PS). In the domain of evolutionary constrained multi-objective optimization, the presence of constraints prompts the partitioning of the search space into two distinct regions: feasible and infeasible. The primary objective in addressing a CMOP is to identify a set of Pareto optimal solutions within the feasible regions, satisfying all constraints. This set, comprising exclusively of feasible Pareto optimal solutions, is termed the constrained Pareto optimal set (CPS). Conversely, when all constraints are disregarded, the set of all Pareto optimal solutions constitutes the unconstrained Pareto optimal set (UPS). It is noteworthy that every solution within CPS is inherently feasible, whereas the feasibility of solutions within UPS remains uncertain.

Pareto front: The mapping of the PS onto the objective space is termed the Pareto front (PF). Similarly, the projection of the CPS onto the objective space is referred to as the constrained Pareto front (CPF), and the projection of the UPS onto the objective space is known as the unconstrained Pareto front (UPF). In unconstrained multi-objective optimization, the objective is to attain the UPF, aiming for an optimal spread of solutions. Conversely, in solving CMOPs, the objective is to cover the CPF with excellent convergence and distribution. More specifically, achieving good convergence implies that solutions closely approach the PF, while good distribution denotes that solutions exhibit a desirable spread and evenness in the objective space [30]. To give more clear explanation of the ideal CPF, Fig. 2 presents a schematic diagram. Obviously, Fig. 2a is an undesirable solution set because its solutions does not converge to the CPF and there exist infeasible solutions. By contrast, Fig. 2b shows an ideal solution set, where all solutions have converged and are evenly distributed along CPF.

Fig. 2

Ideal solution set and non-ideal solution set

Classification of CMOPs: CMOPs are commonly categorized into four types, distinguishing them based on the relative positions of the UPF and CPF in the objective space:

1) Type I: UPF and CPF completely overlap; in this case, the solutions in UPF are all feasible solutions, as shown in Fig. 3a.

2) Type II: In this scenario, the CPF is fully encompassed within the UPF, where the CPF constitutes a subset of the UPF, and the two exhibit partial overlap. As depicted in Fig. 3b, this implies that on the UPF, certain solutions are feasible, while others are infeasible.

3) Type III: As illustrated in Fig. 3c, there is a partial overlap between the UPF and the CPF, featuring shared elements as well as distinct ones. Correspondingly, the UPF is partially composed of feasible solutions.

4) Type IV: In the depicted scenario in Fig. 3d, the UPF and the CPF are entirely distinct and separated. Notably, the UPF is situated in an infeasible region, rendering it entirely infeasible.

Fig. 3

Classification of CMOPs

3 Summary of existing CHTs

CHTs are key techniques for handling various constraint conditions. Therefore, in this section, some commonly used CHTs are introduced in detail, mainly including: penalty function method [31, 32], constrained domination principle (CDP) [33, 34], $\epsilon$ constrained method [35, 36], random sorting method (SR) [37, 38], and multi-objective method [39, 40].

3.1 Penalty function method

The penalty function method typically formulates the penalty term based on the degree of constraint violation exhibited by the individual. Subsequently, by incorporating this penalty term into the objectives, a new fitness function is defined, effectively transforming the constrained optimization problem into an unconstrained optimization problem.

In the penalty function method, a critical consideration is the determination of the penalty coefficient, as it significantly influences the trade-off between objectives and constraints. A small penalty coefficient may lead the population to struggle in discovering feasible solutions, while an excessively large coefficient can steer the population toward local feasible regions. Researchers have devised various methods to set the penalty coefficient, resulting in three distinct types of penalty function methods: static penalty method, dynamic penalty method, and adaptive penalty method. These methods cater to different strategies for adjusting the penalty coefficient based on specific criteria and considerations.

In the static penalty method, the penalty factor remains a constant throughout the optimization process. While this approach offers simplicity and ease of implementation, it has the drawback of maintaining a fixed proportion between objectives and constraints during the entire evolution, which may not support an optimal balance in the early and later stages of evolution. In contrast, the dynamic penalty method involves a penalty factor that continuously adjusts during the evolution process. Typically, in the initial stages of evolution, the penalty factor assumes a smaller value to prioritize a relatively larger proportion of objectives, aiming to enhance population diversity. Subsequently, the penalty factor gradually increases, leading to a gradual rise in the proportion of constraints, ultimately bolstering the feasibility of the population. This dynamic adjustment is advantageous for achieving a more balanced trade-off between goals and constraints throughout the evolutionary process. However, determining the rules for dynamic changes can be challenging, as different problems may necessitate distinct adjustment strategies. In the adaptive penalty function method, it collects information from the evolution process and then feeds back to the population based on the collected information. Through this method, the penalty factor is adaptively adjusted. This type of method can guide evolution based on population evolution information, so it can achieve promising performance on more complex problems compared to the first two types of methods. However, how to utilize information and which information to utilize are worth considering.

3.2 CDP

CDP was proposed by Deb et al. [41]. When comparing two solutions, it usually deals with the objectives and constraints separately. Generally, it uses the following criteria to compare solution $\vec {x}_1$ and solution $\vec {x}_2$.

• If solution $\vec {x}_1$ is a feasible solution, but solution $\vec {x}_2$ is an infeasible solution, then solution $\vec {x}_1$ is better than solution $\vec {x}_2$.

• If both solutions $\vec {x}_1$ and $\vec {x}_2$ are infeasible, and if the constraint violation degree of solution $\vec {x}_1$ is less than $\vec {x}_2$, then $\vec {x}_1$ is better than $\vec {x}_2$.

• If the solutions $\vec {x}_1$ and $\vec {x}_2$ are both feasible solutions, then $\vec {x}_1$ is better than $\vec {x}_2$ if $\vec {x}_1$ Pareto dominates $\vec {x}_2$.

From the above comparison criteria, it can be seen that the operation of CDP is relatively simple and easy to embed into various MOEAs. Moreover, CDP exhibits a preference for feasible solutions, enabling the population to quickly locate feasible regions and converge rapidly. However, this also makes it difficult for the population to cross large infeasible obstacles and search for the real CPF.

3.3 $\epsilon$ constrained method

The $\epsilon$ constrained method, introduced by Takahama and Sakai [35], addresses certain limitations of the CDP. This method introduces a parameter $\epsilon$ to relax constraints, deeming infeasible solutions with constraint violations less than $\epsilon$ as feasible. Consequently, this approach enables the retention of exceptional infeasible solutions. Typically, the value of $\epsilon$ gradually diminishes, and when it reaches 0, the $\epsilon$-constrained method is equivalent to CDP. When employing the $\epsilon$ constrained method to compare individuals $\vec {x}_1$ and $\vec {x}_2$, the following comparison criteria are applied:

• If $CV(\vec {x}_1) \le \epsilon$, while $CV(\vec {x}_2) > \epsilon$, then solution $\vec {x}_1$ is better than solution $\vec {x}_2$.

• If $CV(\vec {x}_1) > \epsilon$, $CV(\vec {x}_2) > \epsilon$, and $CV(\vec {x}_1) < CV(\vec {x}_2)$, then $\vec {x}_1$ is better than $\vec {x}_2$.

• If $CV(A) \le \epsilon$, $CV(B) \le \epsilon$, then $\vec {x}_1$ is better than $\vec {x}_2$ if $\vec {x}_1$ Pareto dominates $\vec {x}_2$.

The $\epsilon$ constrained method, by relaxing the constraint boundary, allows for the utilization of information from high-quality infeasible solutions and facilitates the overcoming of certain infeasible obstacles. Consequently, this approach enhances population diversity, aiding the population in approaching the true CPF. However, setting an appropriate change rule for the parameter $\epsilon$ is challenging, as it involves multiple other parameters. If the value of $\epsilon$ is set improperly, the population is susceptible to becoming trapped in a local feasible region or may encounter difficulty in finding a feasible solution.

3.4 SR

In the context of SR, a probability parameter pf is introduced. This means that when two individuals are being compared, there is a probability of pf for the population to exclusively compare their objective function values using the Pareto dominance principle. Conversely, there is a probability of $(1-pf)$ for the population to use the CDP to prioritize the comparison based on their constraint violation degrees.

Undoubtedly, the operation of SR is relatively straightforward, and to some extent, it can overcome the shortcomings of CDP. In comparing individuals, SR has a chance to exclusively consider the objective function values between individuals without taking constraints into account. As a result, the utilization of SR contributes to the enhancement of population diversity. Nevertheless, the careful consideration of the probability parameter pf is crucial. Different problems may necessitate distinct probability parameters, and setting this parameter improperly may inadvertently guide the population towards local feasible or infeasible regions. Therefore, the selection of an appropriate value for pf is critical to the effectiveness of the SR method in maintaining population diversity.

3.5 Multi-objective method

In multi-objective methodologies, constraints are treated as one or more additional objectives, thereby transforming the constrained multi-objective optimization problem into an unconstrained multi-objective optimization problem. Subsequently, multi-objective techniques are employed to address the problem. Typically, the transformed objective function takes the following general form:

$$\begin{aligned} \begin{array}{l} \min \vec {F}(\vec {x})=\left( f_{1}(\vec {x}), f_{2}(\vec {x}), \ldots , f_{m}(\vec {x}), CV(\vec {x})\right) ^{\text {T}} \\ \end{array} \end{aligned}$$

(4)

or

$$\begin{aligned} \begin{array}{l} \min \vec {F}(\vec {x})=\left( f_{1}(\vec {x}), \ldots , f_{m}(\vec {x}), cv_{1}(\vec {x}), \ldots , cv_{l+k}(\vec {x})\right) ^{\text {T}} \\ \end{array} \end{aligned}$$

(5)

The multi-objective method treats constraints as objectives, avoiding the difficulty of directly solving CMOPs. It treats constraints and objectives equally, which is beneficial for balancing objectives and constraints, and can improve population diversity. However, additional objective functions are difficult to design, and inappropriate objective functions can actually increase the difficulty of problem-solving. Moreover, due to the increase in the objective function, this method may lead to many-objective optimization problems, making the algorithm inefficient. Additionally, there is a risk of insufficient algorithmic selection pressure, causing the algorithm to fail to converge.

Fig. 4

The schematic of the different CHTs used to compare individuals

In order to more simply illustrate the different mechanisms used by several CHTs, the advantages and disadvantages of individual $\vec {x}_1$ and $\vec {x}_2$ under different constraint handling mechanisms are shown in Fig. 4. Among them, $\vec {x}_1$ is located in the infeasible area and $\vec {x}_2$ is located in the feasible area, but the objective values of $\vec {x}_1$ is better than $\vec {x}_2$. It is evident that when using CDP to compare two individuals, $\vec {x}_2$ is superior to $\vec {x}_1$ because $\vec {x}_2$ is a feasible solution and $\vec {x}_1$ is an infeasible solution. While $\vec {x}_1$ is located within the constraint boundary of the $\epsilon$ constrained method and belongs to the $\epsilon$ feasible solution. Therefore, $\vec {x}_1$ is superior to $\vec {x}_2$ when comparing the two using the $\epsilon$ constrained method. However, if $\vec {x}_1$ is located outside the $\epsilon$ constrained boundary, $\vec {x}_1$ is inferior to $\vec {x}_2$. For the multi-objective method, since the objectives of $\vec {x}_1$ is superior to $\vec {x}_2$, and the constraint violation value of $\vec {x}_2$ is superior to $\vec {x}_1$, these two individuals are non-dominated by each other and have the same quality. In addition, for the penalty function method, whether $\vec {x}_1$ is better than $\vec {x}_2$ depends on the degree of punishment, if the degree of punishment is relatively small, then $\vec {x}_1$ is better than $\vec {x}_2$, otherwise, $\vec {x}_1$ will be worse than $\vec {x}_2$. Similarly, for SR, $\vec {x}_1$ is better than $\vec {x}_2$ with the probability of pf and worse than $\vec {x}_2$ with the probability of ($1-pf$). Moreover, a comprehensive summary of the advantages and disadvantages associated with each CHT is provided in Table 2.

Table 2 Comparison of the advantages and limitations of different CHTs

4 Summary of existing CMOEAs

Currently, an increasing number of algorithms have emerged for solving CMOPs, and different mechanisms were designed and used in these algorithms. Therefore, in this section, we classify and introduce currently popular CMOEAs based on the inherent mechanisms of algorithms, as depicted in Fig. 1. Existing CMOEAs are categorized into six types: 1) methods based on the new fitness function [42, 43], 2) methods based on enhanced CHT [44], 3) methods based on constraints relaxation, 4) methods based on two-stage optimization [45, 46], 5) methods of creating other populations for assistance [47,48,49], and 6) methods of altering the reproduction operators [50,51,52].

4.1 Methods based on the new fitness function

This category of methods establishes a new fitness function when solving CMOPs, then using it to evaluate individual quality. A well-designed fitness function aids the algorithm in overcoming difficulties encountered in solving the original CMOPs.

$\bullet$ Methods based on adaptive parameters: Woldesenbet et al. [53] devised a new fitness function by incorporating two penalty functions: one based on an individual’s objective function values and the other on the degrees of constraint violation. The balance between the two penalties was adaptively controlled by the proportion of feasible solutions in the current population. Similarly, Jiao et al. [54] determined a new fitness value by considering both the actual objective function values of an individual and the degrees of constraint violation. The weight assigned to objectives and constraints was dynamically controlled by the feasible rate of the current population. Additionally, Fan et al. [55] developed an adaptive fitness function by integrating information on individual constraint violation values, objective values, current evolutionary algebra, and maximum evolutionary algebra. This adaptive function dynamically generated penalty factors based on the evolutionary state of the population, effectively balancing objectives and constraints. Ma and Wang [32] introduced a method to transfer infeasible solutions based on the distribution of feasible solutions around them. The degree of transfer was adaptively controlled by the proportion of feasible solutions in the current parent and offspring populations. Subsequently, infeasible solutions after the transfer were penalized according to their degrees of constraint violation. This adaptive transfer and penalty mechanism aimed to balance diversity and feasibility in the early stages and achieve a balance between diversity and convergence in the later stages of evolution.

$\bullet$ Methods based on dynamic parameters: Maldonado and Zapotecas-Martínez [56] utilized evolutionary algebra to integrate constraint violation degrees and objective function values, deriving new fitness values to dynamically balance constraints and objectives. To facilitate the rapid traversal of the infeasible region in the early stages and subsequent convergence to the feasible region in the middle and later stages of evolution, Vaz et al. [57] introduced a three-stage punishment method. This method assigned different fitness values at distinct stages by employing varying penalty factors. Specifically, during the early stages, no punishment was applied, and the fitness value was derived directly from the objective function value. In the middle stages, a slight penalty was imposed on constraints to guide the population towards regions proximate to the feasible region. In the later stages, a comparatively high penalty coefficient was set to facilitate the population’s entry into the feasible region. Wang et al. [43] innovatively measured the effects of constraints on objectives by considering their interaction and direct effects. This information was then combined with the original objective functions to design a new fitness function. This approach allowed the algorithm to utilize information from infeasible solutions effectively in the pursuit of optimal regions. To maintain a desirable population diversity, Long et al. [58] retained a certain number of infeasible solutions throughout the population’s evolution. Additionally, a new fitness function was devised to assess the quality of infeasible solutions, aiming to achieve a balance between constraints and objectives.

$\bullet$ Methods based on rankings: Jadaan et al. [59] sorted individuals based on objectives and constraints respectively. Subsequently, an adaptive penalty was applied to constraints based on the ranking values, thereby constructing a new fitness function. This strategy aims to achieve a balanced consideration of objectives and constraints. Ma et al. [42] sorted individuals in the population using two methods: Pareto dominance principle and CDP, respectively, and then weighted the two rankings to obtain a new fitness function. Moreover, the weights assigned to these rankings were determined based on the proportion of feasible solutions in the current population, enabling an adaptive balance between objectives and constraints. In this algorithm, the weight of the constraints is always greater than the objectives, which leads to the population not achieving excellent results on some complex problems. To address this, Yu et al. [60] proposed a dynamic selection preference algorithm, wherein weights were set using evolutionary algebra. In the early stages of evolution, the weight of objectives exceeded that of constraints, allowing for rapid traversal of large infeasible regions.

4.2 Methods based on enhanced CHT

This type of method is to compensate for the shortcomings of existing CHTs. Generally, new CHTs are designed or new mechanisms or population information are added to the original CHTs to enhance their problem-solving ability.

$\bullet$ Methods based on CDP: The CDP stands as one of the most commonly employed CHTs. Deb et al. integrated CDP with NSGA-II to address CMOPs [41]. While CDP demonstrates commendable convergence capabilities, it may face challenges in complex problems, where it is susceptible to local optimality. Fan et al. [44] introduced an enhancement to CDP by integrating angle information and proportion information of feasible solutions. This modification aimed to readjust the dominance relationship between individuals. By incorporating such information, the method could preserve high-quality infeasible solutions, aiding the population in traversing extensive infeasible regions. This enhancement contributed to achieving a balance between diversity, convergence, and feasibility in the optimization process. Wang and Xu [61] further integrated infeasible solution information into angle-based evolutionary algorithms, hoping to preserve those infeasible solutions with good diversity to help the population escape from local feasible regions. Ning et al. [46] sorted the individuals of the population based on Pareto dominance principle and constraint violation degrees respectively. After that, the final constrained non-domination ranking of each individual was the maximum of the above two kinds of ranking, so as to achieve the balance between the objectives and constraints, and then embedded it into the decomposition based framework to solve CMOPs. Ghiasi et al. [62] proposed a multi-objective variation of the Nelder-Mead simplex method and combined it with CDP to improve the quality and distribution of solutions. Gu et al. [63] proposed an enhanced version of CDP and integrated it into the multi-objective evolutionary algorithm based on decomposition (MOEA/D) framework. In this approach, an acceptable deviation value was assigned to each solution to align it with the most suitable weight vector. The deviation value was dynamically adjusted, particularly in the early stages of evolution, to preserve high-quality infeasible solutions. Zhou et al. [64] applied CDP to local populations to increase population diversity. In addition, a population status detection mechanism was designed to more rationally allocate computing resources, and diversity-enhanced CDP was developed to enhance exploration capability of the population.

$\bullet$ Methods based on SR: Geng et al. [37] employed SR to leverage objective information and achieved a balance between objectives and constraints. This utilization of objective information aimed to prevent the population from converging to local optima. Recognizing the challenge of setting the probability parameter in SR, Ying et al. [65] designed an adaptive version of SR. This adaptive approach dynamically controlled the probability parameter based on the current evolutionary stage and the differences between the degrees of constraint violation among individuals in the population. The goal was to achieve a flexible and dynamic balance between objectives and constraints during the optimization process.

4.3 Methods based on constraints relaxation

Methods that involve relaxing constraints, particularly using the $\epsilon$ constrained method, are primarily geared toward exploring infeasible areas. The relaxation of constraints allows the algorithm to discover high-quality information, enhance population diversity, and facilitate the exploration of the entire CPF. In these approaches, the $\epsilon$ constrained method serves as a key tool for managing constraints and guiding the exploration of the solution space.

$\bullet$ Methods for dynamically adjusting constraint boundary: In order to improve population convergence by controlling infeasibility, Saxena et al. [66] designed the $\epsilon$ domination principle and applied it to the NSGA-II framework. Similarly, Zapotecas et al. [67] implemented the $\epsilon$ constrained method within the MOEA/D framework. This application involved relaxing the constraint boundary to address a constrained multi-objective airfoil shape optimization problem. Yang et al. [68] combined the $\epsilon$ constrained method with adaptive selection operations to adaptively select different evolutionary operations based on performance during population evolution. Becerra et al. [69] employed the $\epsilon$ constrained method to relax the constraint boundary in their optimization approach. This relaxation facilitated the population’s ability to traverse infeasible regions, enabling it to approach the proximity of the true CPF. After that, the rough sets theory was used to extend these solutions to increase their diversity, so that the population could be distributed over the complete CPF. Martinez and Coello [70] introduced a novel selection mechanism based on the $\epsilon$ constrained method. This mechanism was integrated into the MOEA/D. The method aimed to explore relevant information within the neighborhood and identify promising solutions with the minimum objective values, all while adhering to the dynamic constraint boundary. To address challenges posed by problems with narrow feasible regions and multiple disconnected feasible regions, Sun et al. [71] combined constraints relaxation technique with differential evolution algorithm, and designed different offspring generation strategies and environment selection mechanisms for solutions inside and outside the constraint boundary to help the population improve its ability to explore search space and find feasible regions.

$\bullet$ Methods for adaptive adjustment of constraint boundary: In [72], Fan et al. improved the $\epsilon$ constrained method. In this method, the proportion of the current population feasible solutions was incorporated into the general $\epsilon$ constrained method to adjust the constraint boundary in an adaptive and dynamic manner. In order to make the range of constraint boundary more reasonable, Yang et al. [73] developed an enhanced version of the $\epsilon$ constrained method. This improvement involved adaptive adjustments to the constraint boundary by utilizing the maximum and minimum constraint violation values observed among infeasible individuals. In the work presented by Yang et al. [74], a dynamic constraint processing mechanism was introduced. This mechanism partitioned the search process of a population into two modes: constrained search mode and unconstrained search mode. The division was based on evolutionary algebra and the proportion of feasible solutions within the population. In the constrained search mode, an enhanced $\epsilon$ constrained method was employed. This improved method could adaptively adjust the $\epsilon$ value using information such as the maximum constraint violation value and the proportion of feasible solutions in the population. Wang et al. [75] ranked the difficulty of each constraint according to the degree of constraint violation values of the individuals, and then relaxed the boundary of each constraint to maintain the diversity of the population. In an effort to guide the population away from locally feasible or infeasible areas, Zhu et al. [76] devised a detection and escape mechanism, when evolutionary stagnation in the population was detected, an enhanced $\epsilon$ constrained method was employed. This improved method facilitated the relaxation of the constraint boundary, assisting the population in escaping from the stagnant areas. To leverage information from infeasible solutions, Zhou et al. [77] introduced a mechanism aimed at preserving the diversity of infeasible solutions. This mechanism was designed to capitalize on the information offered by infeasible individuals located outside the dynamic constraint boundary. By maintaining the diversity of infeasible solutions, the algorithm sought to extract valuable information from this subset of the population, potentially enhancing the optimization process in the presence of constraints.

4.4 Methods based on two-stage optimization

Methods of this type often partition the evolutionary process of a population into two distinct stages, each serving a different evolutionary purpose. The information retained by the population in the initial stage is strategically utilized to guide its search for the CPF in the subsequent stage. This two-stage approach is designed to enhance the optimization process, leveraging specialized strategies at different phases to achieve improved performance in constrained multi-objective optimization problems.

$\bullet$ Methods of not considering constraints information in the first stage: Fan et al. [78] introduced a push and pull search framework known as PPS. In the push phase of this framework, all constraints were ignored, allowing the population to traverse all infeasible regions and approach the vicinity of the UPF. Subsequently, in the pull phase, an improved $\epsilon$ constrained method was implemented to guide the population’s search from the UPF to the CPF. After that, they then combined the PPS framework with the MOEA/D framework [79], which applied PPS to each of the decomposed subproblems, and these subproblems were solved through the use of collaborative search. Similarly, Garcia et al. [80] combined PPS technology with cellular genetic algorithm, then embedded them into the MOEA/D framework to solve CMOPs. The advantage of PPS is that the population is not affected by the infeasible region in the first stage and quickly approaches the UPF, but in the second stage, all constraints are considered at the same time, which may cause difficulties in the population search. In order to overcome this problem, Ma et al. [81] prioritized all constraints after the population reached UPF in the first stage. In the second stage, constraints were processed in turn according to their priorities, rather than all at once, thus reducing the difficulty of constraint processing and make it easier for the population to search for CPF. Ming et al. [82] developed a general two-stage framework, similarly, the purpose of the first stage is to focus on approximating the UPF, and the feasible solutions encountered during the search are saved to the archive, and in its second stage, any CMOEAs can be embedded, the archive obtained in the first stage serves as its initial population, then the embedded CMOEAs are used to search for CPF. To leverage the relationship between the UPF and the CPF for solving CMOPs, Liang et al. [83] introduced a two-stage evolution process. In the learning stage, feasibility information and dominance relationship information within the population were utilized to ascertain the relationships between the two Pareto fronts. Subsequently, in the evolution stage, the knowledge acquired in the learning stage was employed to guide the population, enabling the use of distinct evolution strategies. This approach aimed to assist the population in searching for the genuine CPFs by capitalizing on the learned relationships between the UPF and CPF. Zhang et al. [84] investigated an optimization framework that maps the objective space to the decision space. They employed an artificial bee colony framework and divided the optimization process into two stages. In the initial stage, fast non-dominated and crowded distances were utilized to drive the population toward the Pareto front. In the second stage, a multi-objective mechanism employing the Tchebycheff method was applied to enhance population diversity. Zou et al. [85] designed a two-step auxiliary population, in which the first step ignored constraints to evolve toward UPF, and then the constraint relaxation technique was used in the second step to induce the population to search from UPF to CPF, assisting the main population to search for more promising regions. To harness the information from promising feasible and infeasible solutions, Bao et al. [86] proposed an archive-based two-stage algorithm, comprising exploration and exploitation stages, each employing two populations. In the exploration phase, the first population disregarded constraints to search the UPF, while the second population incorporated overall constraints as an additional objective to explore promising regions. Subsequently, in the exploitation phase, both populations considered all constraints to search for promising feasible regions and collaborate in the exploration of the CPF. In order to be able to take full advantage of the information hidden in feasible non-dominated solutions (FNDS), Zhang et al. [87] proposed a two-stage evolution strategy, in the first stage, the population does not consider whether it satisfies constraints, in the second stage, the population was divided into three parts: FNDS, solutions dominated by FNDS, solutions not dominated by FNDS, and different environmental selection strategies were designed to help these three parts work together to find CPF. Ming et al. [88] devised a two-stage, two-population algorithm for addressing CMOPs. In the initial stage, two populations underwent independent evolution, one considering constraints and the other without. Subsequently, in the second stage, the primary population furnished location information pertaining to the feasible domain for the auxiliary population. This facilitated the exploration of high-quality infeasible solutions by the auxiliary population. The infeasible solutions discovered by the auxiliary population, in turn, played a pivotal role in aiding the primary population in converging towards the CPF.

$\bullet$ Methods of incorporating constraints information in the first stage: To strike a balance between constraint satisfaction and objective optimization, Tian et al. [89] devised a two-stage algorithm. In the first stage, equal emphasis was placed on objectives and constraints, encouraging the population to traverse infeasible regions. Subsequently, in the second stage, the significance of constraints was heightened relative to objectives. This adjustment aimed to guide the population into the feasible region and facilitate the discovery of the CPF. Liu and Wang [90] devised a straightforward yet efficient two-stage framework. In the first stage, a CMOP was converted into a constrained single-objective optimization problem, intending to guide the population toward promising feasible regions. Subsequently, in the second stage, any approximate CMOEA could be integrated. This integration aimed to obtain a well-distributed population, enhancing the exploration of the CPF. In the proposed purpose-directed two-stage algorithm by Yu et al. [91], a dual-stage strategy was employed. In the first stage, the feasibility of the population served as a secondary index, with the primary goal of striking a balance between convergence and diversity. In the second stage, certain infeasible solutions with low constraint violation values were selectively utilized based on the evolutionary state of the population. The principal objective in this stage was to sustain the feasibility and diversity of the population. Yu and Lu [92] identified that the algorithm’s search capability could be enhanced through the utilization of corner points. Consequently, they introduced a two-stage algorithm centered around corner point search. In the initial stage, the focus was on the exploration of corner points, while the subsequent stage involves approximating the actual CPF of the population with the aid of corner points. In the research conducted by Liang et al. [93], a two-stage algorithm based on interactive niche was explored. In the initial stage, two populations were independently designed to utilize niches for generating offspring populations, thereby sustaining population diversity. In the subsequent stage, the two populations were combined, and interactive niche technology was employed to facilitate collaboration in generating offspring, aiming to extract more promising information. Xiang et al. [94] balanced the objectives and constraints by considering the type of problem. In the first stage, only the objectives were optimized, with the aim of determining the type of problem. In the second stage, different evolutionary strategies were used to encourage the population to search for CPF based on the determined type. In order to solve CMOPs with deceptive characteristics, Peng et al. [95] studied a collaborative framework using an improved direct weight vectors, which mainly consisted of two stages. In the first stage, two subpopulations were used to explore feasible regions and the entire search space, respectively, and they shared directions that are conducive to objectives optimization with each other. In the second stage, the entire population was dedicated to finding the Pareto optimal solutions. Wang et al. [96] introduced a novel two-stage algorithm that incorporated a two-population mechanism in the first stage to enhance algorithm performance through complementary environmental selection. In the second stage, a feasibility-based approach was explored to promote a more even distribution of the population within the feasible area.

4.5 Methods of creating other populations for assistance

This type of method usually creates one or more auxiliary populations, which have different purposes from the main population, to help the main population search for CPF.

$\bullet$ Auxiliary population without considering constraints: In order to explore each objective in more detail, Wang et al. [97] created m subpopulations dedicated to optimizing m objectives, and archive population as the main population to save feasible non-dominated solutions searched by all auxiliary populations, so as to obtain well-distributed CPF. Similarly, Chafekar et al. [98] created multiple auxiliary subpopulations, each of which used genetic algorithms to optimize a single objective, and there is information interaction among these subpopulations. Liu and Wang [99] used a MOEA/D framework to decompose a CMOP into multiple subproblems, and then created an archive population for each subproblem to store individuals with good objective values and low constraint violation values, and there was information crossing between these subpopulations during the generation of offspring. Liu et al. [100] designed two cooperative populations, in which the first population only optimized the objective values by ignoring constraints, while the second population only optimized the constraint violation values and ignored the objectives. The two populations helped each other evolve through interaction and migration between solutions. Tian et al. [101] proposed a coevolutionary algorithm involving an auxiliary population that disregards all constraints, enabling it to approach the UPF. Simultaneously, the main population utilized CDP to explore the CPF. The interaction between the auxiliary and main populations facilitated the latter in traversing extensive infeasible regions and further exploring the CPF. While the auxiliary population swiftly reached the proximity of the UPF by ignoring constraints, its effectiveness diminishes in later evolutionary stages, particularly for problems with distant UPF and CPF, leading to suboptimal resource utilization. To solve this problem, Liang et al. [102] proposed a population size reduction mechanism. This innovative strategy involved gradually diminishing the size of the auxiliary population, thereby preventing unnecessary consumption of computing resources. Additionally, an archive was employed to retain feasible solutions discovered by the auxiliary population throughout the evolutionary process. This archival mechanism ensured that the auxiliary population can provide valuable contributions to the main population by exploring regions that may have been overlooked. The combined effects of population size reduction and the archive enhance the efficiency and exploration capabilities of the coevolutionary algorithm.

$\bullet$ Auxiliary population gradually considering constraints: Liang et al. [103] proposed an innovative infeasible-proportion control mechanism. This mechanism gradually diminished the proportion of infeasible solutions within the auxiliary population. Consequently, the auxiliary population shifted gradually from the infeasible region towards the feasible region, allowing it to collaborate with the main population in the joint exploration of the CPF. Li et al. [104] designed two collaborative evolutionary archive populations. The main archive was convergence archive, aiming to promote the evolution of populations towards CPF, and the other auxiliary archive was diversity archive, which mainly explored feasible and infeasible regions that are not been explored by the main archive, so as to maintain the diversity of populations. Inspired by this, Xia and Dong [105] designed a collaborative matching mechanism for two archives, and developed a high-quality solution selection mechanism for convergence archive to help it approach CPF from different directions. Moreover, diversity archive used a dynamic selection mechanism when updated based on the evolution state of the convergence archive. Wang et al. [106] proposed a co-evolutionary approach involving two populations, the primary population was dedicated to emphasizing feasibility and maintaining diversity while exploring the CPF. In contrast, the auxiliary population initially neglected constraints during its early evolution stages, reserving consideration for later phases. Additionally, the auxiliary population strategically focused its search efforts on corner and center points, contributing to an expedited convergence process. Ming et al. [107] designed an auxiliary population using adaptive penalty function method with the aim of preserving infeasible solutions with competitive ability. On the contrary, the main population preferred feasibility, and a new adaptive fitness function was designed for it to maintain a balance between convergence and diversity. Liu et al. [108] designed a new auxiliary archive that uses a non-dominance sorting mechanism and an angle-based selection mechanism to maintain population diversity, with the aim of approaching the feasible region from the side of the infeasible region to assist the main population search for CPF. Qiao et al. [109] studied an innovative auxiliary population by designing a dynamic constraint processing mechanism for it. In this way, the auxiliary population can gradually increased the constraints for processing, reducing the difficulty of handling all constraints simultaneously. Moreover, a dynamic resource allocation mechanism was applied to the main population and auxiliary population based on performance feedback from both populations.

$\bullet$ Methods based on multi-task optimization: Qiao et al. first used the idea of evolutionary multi-task to solve CMOPs in [110], in which two tasks were constructed, the main task was for the original CMOP, whose main purpose was to find CPF, and the auxiliary task was for the constrained MOP, which mainly provided excellent objective function information for the main task. The two task populations searched for CPF in a co-evolutionary way. In [111], a dynamic auxiliary task was designed to assist the main population search for CPF, and its constraint boundary was dynamically reduced, and an improved $\epsilon$ method, which incorporated the multi-objective method into the general $\epsilon$ constrained method to increase the diversity of the population within the constraint boundary, was applied to the auxiliary task to continuously maintain the correlation with the main task and improve the valid information for it. Afterwards, this algorithm was improved to enhance its search ability to handle more complex decision space constraints problems [27]. Similarly, Jiao et al. [112] proposed a multiform optimization framework to solve CMOPs, whose auxiliary tasks used an improved $\epsilon$ constrained method to gradually reduce the boundary, with the aim of using excellent infeasible individuals, and two task populations approached CPF from the direction of the feasible region and the infeasible region respectively. In [113], two tasks were created by Qiao et al., the primary task involved considering all constraints with the aim of searching for the CPF. Meanwhile, the auxiliary task, by disregarding constraints, facilitated the main task in traversing large infeasible regions through information transfer. Additionally, an adaptive offspring generation strategy based on the performance feedback from both tasks had been studied to help both tasks produce superior offspring individuals. In [114], three tasks with different functions were created, among which the main task was to search for CPF. The first auxiliary task was a global task that ignored all constraints to help the main population cross infeasible regions. The second auxiliary task was a local auxiliary task that focused on providing diversity guidance around the main population. Through information transfer between the three tasks, the performance of the main population is effectively improved. Currently, more and more CMOEAs with auxiliary populations were proposed, and Zhang et al. [115] conducted a detailed discussion on the existing auxiliary tasks. Furthermore, they designed an algorithm with auxiliary population, in which the main population utilized a dynamic boundary reduction technique and the auxiliary population focused on optimizing the objective to assist the main population in searching for CPF.

4.6 Methods of altering the reproduction operators

This kind of method mainly helps the population to produce excellent offspring by designing efficient reproduction operators, so as to explore the decision space deeply to search for well-distributed CPF.

$\bullet$ Methods based on differential evolution: Xu et al. [50] introduced a novel approach to differential evolution by incorporating infeasible solution-guided mutation operations. This method involved integrating nearby infeasible solutions into the variation operation of differential evolution. By leveraging the information from these high-quality infeasible solutions, the algorithm aimed to guide the evolution of the population toward promising regions in the search space. To achieve offspring individuals characterized by both strong convergence and diversity, Yu et al. [51] devised a competitive algorithm leveraging the metaphors of differential evolution. In this algorithm, a novel mutation mechanism was introduced to distinctly address feasible and infeasible solutions. The goal was to encourage the generated offspring to adhere to various constraints while approximating the genuine CPF. Ramesh et al. [116] proposed an enhanced version of the differential evolution algorithm for addressing multi-objective reactive power scheduling problems. Their approach involved incorporating a recombination operation based on simulated binary crossover and a dynamic crowding distance mechanism into the differential evolution algorithm. This modified algorithm aimed to improve the optimization process for reactive power scheduling in the context of multiple objectives. Qian et al. [117] integrated two differential evolution mutation strategies with an adaptive mechanism. Subsequently, based on feedback information from population performance, the mutation strategies that drive the population towards promising directions were adaptively selected. Furthermore, this adaptive approach was combined with the $\epsilon$ constrained method to harness outstanding infeasible solutions. In order to prevent the premature convergence of the population in pursuit of convergence, Qu and Suganthan [118] developed a diversity enhanced constrained multi-objective differential evolution algorithm, which combined the population with diversity archive to enhance the diversity of the generated differential vectors and offspring, thereby encouraging the population to explore a wider range of regions. Wang et al. [119] incorporated the idea of Pareto dominance into the adaptive differential evolution algorithm, which could adaptively generate scaling factors and crossover probabilities conducive to population evolution according to the evolutionary state of the population. In addition, an elite archive solution set was created to save the non-dominance solutions in the evolutionary process, and a diversity measurement strategy was designed to maintain the diversity of Pareto optimal solutions. To safeguard both diversity and convergence within the population, Liu et al. [120] formulated an adaptive grouping strategy aimed at augmenting population diversity. Additionally, the studied delved into multiple differential evolution crossover strategies, intending to enhance the algorithm’s search capability and expedite population convergence.

$\bullet$ Methods based on other optimization strategies: He et al. [121] designed a pairwise offspring generation strategy, which selected feasible or infeasible solutions with excellent performance to generate promising feasible solutions or infeasible solutions that can help the population cross infeasible regions, so as to improve the efficiency of the algorithm. This algorithm was successfully applied to solve constrained large-scale multi-objective optimization problems. Miyakawa et al. [122] emphasized the role of infeasible solutions in population evolution, and designed a direct matching strategy based on the inverted penalty-based boundary intersection scalarizing function, in which each solution was assigned a unique search direction based on its position in the objective space to enhance the diversity of the population. Further, they designed a new direct matching method to further exploit information of useful infeasible solutions by controlling the dominance regions between solutions [123], in which the selection area of infeasible solutions was controlled to increase the efficiency of superior information utilization. Ming et al. [52] enhanced the competitive swarm optimizer, introducing an optimizer that amalgamates competitive and cooperative elements to address CMOPs. The competitive swarm optimizer played a role in facilitating rapid convergence of the population. In the cooperative swarm optimizer, a mutual learning strategy was investigated to assist the population in escaping local optima.

5 Benchmark test problems

The purpose of the test function is to test the performance of the designed algorithm. At present, a large number of test problem sets have been proposed. In this part, the characteristics of some classic test sets are mainly introduced, providing convenience for researchers to better understand the test problems.

$\bullet$ The SRN [124], TNK [125], and OSY [126] are among the first three multi-objective test problems with constraints, each featuring two objectives. In SRN, some of its UPF becomes infeasible due to constraints, resulting in CPF being a subset of UPF. TNK incorporates a nonlinear constraint boundary, and its CPF is characterized by discontinuity. OSY’s Pareto optimal region comprises five distinct parts. However, these three test problems have some limitations, including: 1) low dimensionality; 2) most objective and constraint functions are not entirely nonlinear, simplifying the process of finding optimal solutions; 3) the complexity introduced by the constraints in these problems is not adjustable.

$\bullet$ The CTP test problem set, introduced by Deb et al. in 2001 [127], offers the advantage of adjustable constraint difficulty. Notably, the constraints are formulated in the objective space. The constraint functions in this set are designed to present two distinct types of challenges: 1) difficulty in approaching the CPF; 2) difficulty in exploring the entire decision space. The CTP test problem set contains 7 test problems, in which the number of constraints of CTP1 can be adjusted, and the terrain near CPF is very complex. Each constraint is an implicit nonlinear function of the decision variables, and the existence of constraints makes the search area near CPF infeasible. Moreover, the number of constraints can be increased, and the introduction of more features, such as multimodal and deceptive features, further increases the complexity of the test problem. CTP2-CTP7 make it difficult for the population to optimize the entire decision space. The constraints of these problems contain six parameters, and constraint test problems of different difficulty are obtained by adjusting these parameters. Compared with the above three test problems, CTP considers more features, but there are still some defects, such as insufficient dimensionality, a relatively large feasible area, and all questions only contain two objectives. Additionally, the number of constraints introduced in CTP1 can be changed, but the shape of its CPF does not change when the number of constraints is increased. For CTP2-CTP7, although the shapes of their CPFs can be changed, the feasible regions they contain are very large because only one constraint is included.

$\bullet$ The CF test problem set, introduced by Zhang et al. in 2008 [128], consists of 10 test problems featuring nonlinear PSs in the decision space and discontinuous geometry in their CPFs. In comparison with the CTP test problem set, the CF test set introduces a highly complex variable coupling relationship, posing challenges for algorithms to converge when solving CPF test problems. For CF1-CF3 and CF8-CF10, their CPFs are part of the UPFs, with CF8-CF10 containing three objective functions that need optimization. Conversely, CF4-CF7 present search difficulties near CPFs, with the Pareto optimal solution situated on the constraint boundary. Despite its complexity, the CF problem suite also exhibits drawbacks, including the lack of adjustable difficulty, a large feasible area, and a fixed number of objectives.

$\bullet$ The NCTP test problem set [129] contains 18 test problems, and the search space is $[0, 5]^D$, and the number of dimensions of each test problem includes 10 and 30. The NCTP test problem set considers the Rosenbrock function as distance function when constructing the test problems, which increases the difficulty of convergence, and the high-dimensional decision space is constructed, in addition, additional constraints are added to reduce the proportion of feasible regions in the decision space. The NCTP test set introduces several parameters, and test problems with different characteristics can be obtained by controlling the values of these parameters. Furthermore, for NCTP1-3, NCTP7-9, and NCTP13-15, the proportion of feasible domains is less than 0.1%, posing challenges for algorithms to find feasible solutions.

$\bullet$ C-DTLZ test problem set [130] is established based on unconstrained DTLZ test problems [131]. The number of objectives in C-DTLZ test set is extensible and can be set to any number, and the search space is $[0, 1]^D$. The C-DTLZ test set introduces three types of constraints, each contributing distinct challenges. Firstly, constraints are designed to present an infeasible obstacle for the CPF search, preventing the population from effectively approaching the CPF. However, it’s noteworthy that the UPF remains feasible despite these constraints. Secondly, constraints define several isolated feasible regions along the UPF, leading to a scenario where portions of the CPF become infeasible due to the introduced constraints. In this case, the CPF is disconnected but constitutes a part of the UPF. Lastly, constraints are crafted to render the entire UPF infeasible, while the CPF is comprised of the boundaries of the feasible region. The C-DTLZ test set is deliberately designed to incorporate challenges such as multimodality, irregularity, and deviation. Despite its varied difficulties, a notable limitation is the lack of internal independence in the position variables of the test problem on the PS.

$\bullet$ For the DC-DTLZ test problem set [104], similarly, there are three types of constraints. The first type of constraints limit the feasible region to a pair of narrow conical strips, and the algorithm is easy to fall into the local feasible region, thus failing to find all CPF fragments. The second type of constraint narrows the feasible region to the thin band region above PF, and the constraint violation value will fluctuate in the region close to PF, rather than linearly increasing or decreasing, making it more challenging to solve. The third type of constraint is a combination of the first two, which limits the feasible region of the problem to a pair of segmented conical fringes. In addition, when approaching PF, the degree of constraint violation also fluctuates the same as the second type of constraint.

$\bullet$ The LIR-CMOP test problem set [132], comprising 14 test problems, is characterized by narrow feasible regions within a search space of $[0, 1]^D$, with the number of decision variables being extensible. The intricate relationship between the shape and distance variables in these problems adds to their complexity, making them challenging to solve. The true CPF for the LIR-CMOP test set is often hindered by numerous infeasible regional blocks, posing challenges in searching for CPF during population evolution. Notably, the constraint function in each problem involves a controllable shape function and a distance function. The shape function allows for the adjustment of CPF shape, making it either convex or concave, while the distance function is employed to modulate convergence difficulty.

$\bullet$ The MW test set [133] comprises 14 testing problems with diverse characteristics, featuring discontinuous and very small feasible regions. Notably, the CPFs of some problems are composed of only a few isolated Pareto optimal solutions, adding to the difficulty of searching for CPFs in MW test problems. The construction of MW test problems involves a global control process and a local adjustment process. The former is responsible for controlling the size of feasible regions, while the latter adjusts the complexity of the boundaries of feasible regions. This dual-process construction allows for the generation of CPFs with different geometric shapes, making the characteristics of the MW test set more complex. Furthermore, in the MW test set, the distance function can be extended to any number of decision variables, providing the flexibility to adjust the number of decision variables in all MW test problems. Additionally, the constraints in the MW test set are constructed in the objective space.

$\bullet$ Unlike previous testing problems that construct constraints in the objective space or in the decision space, the DOC test set [90] contains 9 test problems that attempt to introduce complex constraints in both spaces. The DOC test set has rich features, such as containing both objective space constraints and decision space constraints, inequality constraints and equality constraints. In addition, CPFs of DOC test set problems have a variety of characteristics, such as: continuous, discontinuous, concave, convex, linear, mixed, and multimodal. Furthermore, the feasible region generated by the constraints introduced by the DOC test set in the decision space also exhibits characteristics such as nonlinearity, minima, and multimodality. One limitation of this test set is that the dimensions of each test problem are fixed and cannot be arbitrarily adjusted.

$\bullet$ The DAS-CMOP test set [134] consists of 9 testing problems with adjustable difficulties and scalable objective numbers, which are mainly characterized by three types of difficulties: the first type of constraint function provides difficulties in diversity; the second type of constraint function introduces difficulties in feasibility; the third type of constraint function can lead to difficulties in convergence of the problem. The difficulty of DAS-CMOP can be defined by a triplet, where each parameter is used to adjust the level of each difficulty. By combining the three main types of constraints with triplets of different difficulty levels, constraints of different difficulty levels can be generated.

$\bullet$ The ZXH_CF test set [28], comprising 16 extensible constraint test problems with a search range of [$\varepsilon$, $1-\varepsilon$] ($\varepsilon =10^{-10}$), introduces constraints that contribute to both convergence difficulty and diversity difficulty. The former involves introducing infeasible obstacles near the CPF and considering the correlation between the position and distance variables, while the latter restricts the optimal feasible region to obtain CPFs of different shapes. Among these 16 test problems, ZXH_CF1 and ZXH_CF2 are the simplest, featuring specific rules for CPFs, such as linear or concave shapes, and infeasibility only in non-optimal regions. Conversely, the CPFs of other problems are irregular, making the problems more challenging to solve. Additionally, the constraints of the ZXH_CF test set are imposed in the decision space, and both the number of objectives and the number of decision variables can be extended. Furthermore, both PSs and PFs have clear analytical expressions, and their shapes are easy to describe, providing a visual way to analyze the search behavior of the algorithm and enhancing the interpretability of the experimental results.

$\bullet$ In the DCMOP test set [95], constraints with deceptive characteristics are defined, that is, the constraint violation values of infeasible regions closer to the feasible region are actually higher, which brings new challenges for CMOEAs. The DCMOP test set contains 6 test problems, among which the UPF of DCMOP1 is infeasible, and the region between UPF and CPF is the deceptive constraints. DCMOP2 is similar to DCMOP1, except that part of the region between UPF and CPF is the deceptive constraints. In DCMOP3, a portion of its UPF is surrounded by deceptive constraints. In DCMOP4, the region closest to the optimal feasible region is the deceptive constraints. The UPF of DCMOP5 is surrounded by deceptive constraints. The CPF of DCMOP6 is surrounded by deceptive constraints.

$\bullet$ Unlike most test problems, which show a simple positive relationship between objectives and constraints, SDC test set [27] attempt to show a more complex relationship between objectives and constraints, making the test problems more consistent with the characteristics of the real world problems. The SDC test set contains 15 test problems, and the number of decision variables related to constraints can be expanded, so the difficulty of the constraint function can be adjusted. In addition, to help researchers more conveniently verify the generalization performance of algorithms on variant functions, the SDC test set provides several different parameter interfaces to make it easier to generate new variant problems. The SDC test set has the following characteristics: 1) constraints are constructed in the decision space and have various characteristics of constraint landscape such as uni-modal or multi-modal; 2) scalable decision variables are related to constraint functions, allowing for adjustment of the difficulty of constraints; 3) scalable decision variables are associated with distance functions, allowing for adjustment of convergence difficulty; 4) the overlap between UPF and CPF can be adjusted to make it easier to test the algorithm’s ability to locally adjust population distribution.

$\bullet$ In [135], a constrained multi-objective test problem set considering multi-modal characteristics is designed, called CMMF, which includes 17 test problems. In CMMF, constraints are designed in the decision space, and each problem has two or more equivalent Pareto optimal solution sets. Based on the differences in convergence of different CPSs and the difficulty in searching feasible regions, CMMF is divided into four types, each with varying degrees of difficulty. In addition, the number of decision variables for CMMF test problems can be expanded.

$\bullet$ The RWCMOP test problem set, developed by Kumar et al. [136], serves as a valuable benchmark for evaluating the performance of algorithms on practical problems. It encompasses 50 distinct real-world problems, spanning various domains such as 21 mechanical design problems, 3 chemical engineering problems, 5 process design and synthesis problems, 6 problems in power electronics, and 15 problems in power system optimization. This diverse collection reflects the complexity and challenges encountered in real-world applications, making the RWCMOP test set an essential resource for assessing the effectiveness of constrained multi-objective optimization algorithms.

Based on the above introduction, the characteristics of various test sets, including the number of objectives, dimension size, the type of constraints, and the features of CPFs, are summarized in Table 3.

Table 3 The characteristics of some classic test problem sets

6 Performance comparison

In this section, the performance of some mainstream CMOEAs belonging to different types is investigated, including DSPCMDE [60], MOEADDAE [76], PPS [78], CCMO [101], MTCMO [111], and CMOCSO [52]. Among them, DPSCMDE belongs to the method based on the new fitness function, MOEADDAE employs the $\epsilon$ constrained method to relax the constraint boundary, PPS is a two-stage method, CCMO and MTCMO both create other populations to assist the evolution of the main population. However, MTCMO adopts a multitasking approach, and finally, CMOCSO generates more promising offspring by altering the reproduction operator.

In addition, three test sets, LIR-CMOP [132], MW [133], and SDC [27], containing different characteristics, are selected to test the ability of these algorithms to solve CMOPs. Specifically, the constraints of the MW test set belong to the objective space constraints, while LIRCMOP and SDC belong to the decision space constraints. However, the constraints of the SDC test problem are more complex. From a global perspective, the constraints of MW and LIRCMOP show a positive correlation with the objectives, while the constraints of SDC show a chaotic and complex relationship with the objectives. Therefore, they can comprehensively investigate the performance of different algorithms.

What’s more, the population size is set to 100, the maximum number of evaluations is 100000, and each algorithm is run 20 times on each problem. The dimensions of the three test sets are set to 30, 15, and 30 respectively. Please note that all experiments are conducted on the PlatEMO platform [137]. Furthermore, the performance metric IGD is used to compare the superiority and inferiority of algorithm results.

The specific evaluation of IGD experimental results is shown in Table 4, where the standard deviation of 20 runs of IGD is represented in parentheses. If the algorithm does not find a feasible solution continuously for 20 runs on a problem, it is represented in the form of "NAN (FR)". FR represents the proportion of runs that find feasible solutions in 20 runs. In addition, for each problem, if an algorithm performs best on this problem, it is represented by a deep orange color, and if it performs second best, it is represented by a light orange background color. Table 5 summarizes the proportion of different algorithms achieving the best and second best IGD values on these three test sets.

Table 4 The average IGD(std) value obtained through different algorithms. The best and second best average IGD of all algorithms in each test function are highlighted in dark orange and light orange, respectively

From the experimental results, it can be observed that CMOCSO performs the best on the LIR-CMOP test set, achieving the best or second best results on 13 out of a total of 14 test problems. CMOCSO uses two offspring generation methods, competitive and cooperation. Specifically, the competitive method uses a competitive particle swarm optimizer to generate offspring, and combined with the $\epsilon$ constrained method to accelerate the convergence of the population to CPF during environmental selection. The cooperation method is through arithmetic crossover operation to generate offspring, which does not consider constraints in environment selection and can help the population explore UPF. The second best performing algorithm is PPS, which is a two-stage algorithm. In the first stage, all constraints are not considered to make the population evolve towards UPF. Afterwards, the $\epsilon$ constrained method is employed to gradually reduce the constraints boundary, promoting the population to gradually recover from UPF to CPF. Based on the experimental results, focusing on the collaborative use of $\epsilon$ constrained method and UPF information may contribute to the effective resolution of the LIR-CMOP test set.

Table 5 Percentage of best and second-best IGD results obtained by different CMOEAS on the MW and RIL-CMOP test suites. The best and second best percentages are shaded with a dark orange and light orange background, respectively

For the MW test set, CCMO achieves the best results, followed by MTCMO, both of which create a auxiliary population to help the main population explore CPF together. The auxiliary population of CCMO does not consider constraints at all, so it can explore the information of UPF. In addition, the interaction between the two populations can help the main population cross large infeasible regions and reach the optimal feasible region. MTCMO designs an improved $\epsilon$ constrained method for the created auxiliary population, which combines typical $\epsilon$ constrained method with multi-objective method, greatly enhancing the diversity of the population and allowing for exploration of more regional information. The proportion of MTCMO to obtain the best and second best results is slightly lower than that of CCMO, which also indicates that the information of UPF is crucial to solving the MW problems. Based on the experimental results, creating suitable auxiliary populations and mining UPF information may help the algorithm more effectively solve the MW test problems.

For the SDC test set, there is a complex relationship between constraints and objectives, unlike the LIRCMOP and MW test sets, whose UPFs and CPFs are in the same direction. While for SDC test problems, searching for UPF may lead to the population moving further away from CPF. Therefore, methods such as PPS, CCMO, and CMOCSO that help populations search for CPFs by exploring UPFs information are not achieves satisfactory results. On the contrary, DSPCMDE performs best on the SDC test set because it does not focus on the relationship between UPF and CPF, but designs a new fitness function to reduce the difficulty of searching the original problem. MTCMO achieves the second best result, because its improved $\epsilon$ constrained method designed for auxiliary population can expand the search range, exploring the regions between UPF and CPF, making it easier for the main population to search for CPF. Based on the experimental results, designing new fitness functions and increasing the exploration of the regions between UPF and CPF will help the algorithm to solve the SDC test problems more effectively.

Fig. 5

Rankings of six compared algorithms by Friedman’s test

Figure 5 shows the overall results of different algorithms on all test problems, which are obtained by the Friedman’s test. The lower the ranking value, the better the overall performance of the algorithm. From the Fig. 5, it can be seen that CMOCSO achieves the minimum ranking value, followed by MTCMO. However, due to the diversity of problem characteristics, different algorithms are suitable for different test problems, so on the whole, the performance difference of different algorithms is not significant. Based on this, targeted strategies can be designed according to the characteristics of the problem to solve the problem more accurately and efficiently.

7 Future research directions

Although a large number of algorithms have been designed to solve CMOPs, there are still some possible research directions and challenges regarding evolutionarily constrained multi-objective optimization. Therefore, in this section, some potential research topics are discussed

$\bullet$ Computationally expensive constrained multi-objective optimization: When the current algorithms solve CMOPs, a lot of computing resources are used [138, 139]. In fact, when faced with real world problems, there are not so many computing resources used [140], only a few thousand computing resources are allowed, such as computational fluid dynamics [141], design of trauma systems [142], and computational fluid dynamics [141]. To address this issue, surrogate models are often employed to simulate the fitness terrain of the original problem, and then surrogate models are used to evaluate individual performance to reduce the number of real function evaluations. There are a large number of computationally expensive CMOPs in the real world [143], therefore, more efficient strategies using surrogate models should be designed to solve such problems.

$\bullet$ Multi-modal constrained multi-objective optimization: Multi-modal problems are characterized by the existence of multiple Pareto optimal solutions that correspond to the same objective vector. In these optimization problems, diverse sets of Pareto optimal solutions, or modes, coexist, sharing identical objective values. Therefore, for CMOPs with multi-modal characteristics, there are two or more CPSs in the decision space corresponding to the same CPF in the objective space, and all CPSs need to be explored [144, 145]. Due to the existence of multi-modal characteristics, the fitness landscape of the problem becomes more complex. In order to deal with such problems, it is necessary to have a very good diversity of algorithms and can identify different multi-modal regions. Therefore, efficient diversity preservation mechanisms and multimodal adaptive recognition mechanisms need to be designed and combined with CMOEAs to solve CMOPs with multimodal characteristics.

$\bullet$ Large-scale constrained multi-objective optimization: Most current CMOEAs address small-scale CMOPs with a relatively small number of decision variables. However, the efficiency of these algorithms decreases significantly when solving CMOPs with large-scale characteristics, mainly because the decision space becomes particularly large as the number of dimensions increases, resulting in the algorithm easily converging to local regions [146]. This kind of problem is also often encountered in real life, such as configuring software optimization [147], multi-objective vehicle routing problem [148], and time-varying ratio error estimation problem [149]. In order to solve this problem, effective dimension reduction technique [150], decision variable grouping technique [151], and efficient reproduction operators [52] are possible solutions.

$\bullet$ Multi-task constrained multi-objective optimization: Multi-task optimization requires an algorithm to solve two or more problems simultaneously in a single run [152, 153]. Current algorithms that use the idea of multi-task to solve CMOP only implement the search for a single CPF, and some optimization algorithms force the population to search both UPF and CPF, but essentially only solve a CMOP. In fact, there are many constrained multi-objective optimization problems where the experience can be used by each other, so it is necessary to explore CPFs for multiple problems at the same time. To solve such problems, efficient knowledge transfer mechanisms need to be designed, and algorithms need to identify which knowledge needs to be transferred, when it needs to be transferred, and whether more auxiliary populations are needed to search for multiple CPFs simultaneously is worth considering.

$\bullet$ Dynamic constrained multi-objective optimization: In real-life scenarios, dynamic environments are frequently encountered, leading to changes in the objectives or constraints of CMOPs. Examples of such dynamic environments include optimal control problems [154], optimization in fluid catalytic cracking operations [155], and the optimization of operational indices in the beneficiation process [156]. In these contexts, the optimization landscape dynamically evolves, requiring adaptive approaches that can efficiently respond to the changing objectives or constraints over time. Adaptive algorithms designed for dynamic CMOPs play a crucial role in addressing the challenges posed by such dynamic environments. Such CMOPs containing dynamic characteristics will be more difficult to solve compared with general CMOPs, because a dynamic environment needs to be considered. As the environment changes, the feasible region of the problem will also be changed, so its CPF will also change. This leads to the population searching for the initial CPF needing to search the decision space again, and even the information obtained needs to be discarded. Indeed, when dealing with dynamic CMOPs, real-time detection of environmental changes becomes crucial. The algorithm needs to continuously assess whether the environment has undergone alterations, identify the specific changes that have occurred, determine the feasibility of reusing the information from the previous population after the change, and strategize on how to effectively leverage this information. Dynamic CMOPs demand adaptive and responsive optimization techniques that can promptly adapt to evolving conditions, ensuring the algorithm remains effective and efficient in the face of dynamic environments. Unfortunately, there are few algorithms to solve such problems [157, 158], so more strategies need to be designed to focus on this field.

8 Conclusion

This paper presents a comprehensive review of evolutionary constrained multi-objective optimization, encompassing fundamental concepts within the constrained multi-objective optimization domain, some common constraint handling mechanisms and their advantages and limitations. Then, the existing classical CMOEAs are classified and introduced in detail, including the CMOEAs based on the new fitness function, CMOEAs based on enhanced CHT, CMOEAs based on constraint relaxation, CMOEAs based on two-stage optimization, CMOEAs of creating other populations for assistance, and CMOEAs of altering the reproduction operators. Then, some characteristics of classical constrained multi-objective test problem sets are introduced, and the performance of some mainstream CMOEAs on test sets with different characteristics is investigated and analyzed. Finally, this paper delves into potential research hotspots within the field of evolutionary constrained multi-objective optimization, paving the way for future investigations. By offering a comprehensive overview of the existing knowledge, this review aims to provide researchers with a foundational understanding of evolutionary constrained multi-objective optimization. It is our aspiration that this contribution will not only serve as a valuable resource for researchers seeking insights into the current state of the field but also stimulate further advancements and innovations in this evolving domain.