Sensitivity analysis of specific metrics for automated evaluation of layout changeability

Manufacturing companies are facing a turbulent market environment. Challenges for these companies lie in balancing efficient, economical action with maintaining agility, responsiveness and competitiveness. Thus, it is becoming even more important that as much potential as possible is leveraged as early as the planning phase of a factory. Automated layout design with focus on changeability could help to increase the agility and responsiveness for the factories. Approaches for automated design and optimization of layout variants are already prevalent. Ultimately, the major challenge remains in the automation of the layout evaluation taking qualitative criteria in consideration. Burggräf et al. provide a first approach for the quantification of the qualitative layout evaluation criterion changeability through specific metrics. Within the scope of this work those metrics are examined with regard to their reliability. Anomalies or deviations in the evaluation results shall be identified and suggestions for improvement will be proposed based on the obtained insights.


Introduction
A central component of factory planning projects is the layout design. It contributes significantly to the success of a project [1]. Layout design is a successive process in which individual layout variants are first created and then optimized on the basis of specific characteristics. The selection of the final layout is a complex decision problem, which is influenced by the individual experience and understanding of the planning problem [2]. Based on the individual experience, the planner can also consider qualitative aspects in the evaluation of the layout variants. These significant aspects include the intuitive knowledge about the evaluation of changeability. Those are in turn considered essential for the anticipatory planning of future factory requirements, as well as the economic efficiency and reaction speed of a factory [3]. Moreover, the subjectivity of the evaluation can play a decisive role in the layout selection. Thus, the lack of possibility of objective comparison of the layout variants can result in the non-selection of the optimal layout [4].
The increasingly dynamic environment of factories poses an additional major challenge to conventional layout design, as agility and responsiveness are limited by the time-consuming planning process [5]. Moreover, insufficient planning processes without of an objective evaluation approach pose the risk that the overall most favorable variant is not identified [6]. Automated layout design can help to overcome these challenges [7]. In the course of this, a number of approaches for the automated design and optimization of layout variants already exist. However, the replacement of subjective decision processes of a factory planner by an objective approach increases the complexity of automated layout design. In this context, Burggräf et al. have developed a methodology for the evaluation of layout changeability which formalizes the qualitative aspect of changeability by means of a mathematical model. With the help of the consideration of the quantified evaluation of changeability, the automated layout design could be completed to meet the requirement of agility and responsiveness of the factory [4].

3
Within the scope of this work, the methodology developed by Burggräf et al. for the evaluation of layout changeability will be examined. In the course of this, a better understanding of the evaluation result will be created to minimize risks in the application of the methodology. Therefore, the validity and reliability of the layout changeability evaluation will be inspected. The aim of this paper is to determine approaches for the improvement of the automated evaluation by the additional or alternated metrics.

Theory background-heuristics of layout changeability
The goal of automated layout design is the identification of the most suitable layout variant in a given situation under certain premises. For this purpose, approaches from operations research serve as a basis for the generation of layout variants [8]. With the help of design and optimization algorithms, it is possible to generate numerous layout variants, which are then optimized based on defined criteria. Therefore, the layout selection process represents a multi-criteria decision-making problem [9]. The generated variants are analyzed using evaluation metrics to determine the expression of the individual target criteria. Hereby, the consideration of quantitative criteria constitutes the significant majority [10]. The results of the evaluation metrics are included in the overall evaluation of the layout variant [4].
Wiendahl et al. formulate five different factors that ensure the changeability of a production system. In this context, the term changeability enabler is often referred to. These criteria can be transferred as requirements for a changeable layout. Thus, following Wiendahl et al., changeability is achieved through a combination of scalability, modularity, mobility, compatibility and universality [11].
According to Burggräf et al. the layout changeability is influenced by different factors. The complex multi-criteria decision problem at hand has been divided into several levels, as Fig. 1 shows. The overall goal is the layout evaluation. This goal is subordinated to the changeability, which was identified as a decisive qualitative criterion. The concretization of the changeability is carried out by the five change enablers scalability, modularity, mobility, compatibility and universality. The degree of layout changeability is evaluated with a specific metric for each enabler with exception to mobility, which is divided into the mobility potential and Fix Points. In accordance with the Analytical Hierarchy Process a weighing of each criterion on the basis of a pairwise comparison is needed, which is a common approach in factory layout design [12,13]. Based on the criterions evaluation results and weighing, it should be possible to establish a ranking of the layout variants and consequently to identify the most favorable variants. The variables of the metrics are imported from a data and geometry model of a digital factory twin. The data and geometry model represent the so-called planning space, which contains the building information as well as the requirements and framework conditions for the layout. These data of the digital twin can be complemented and defined by the user. With the help of the changeability formalization, the basic requirement of an automated layout design is fulfilled considering qualitative factors in the evaluation process [4].
In the following, the previous work of Burggräf et al. will be reflected, which is relevant for the further chapters of this study. The evaluation metrics uncovering the human evaluation of changeability for automated factory layout planning are briefly explained in terms of their mode of operation. The input and output factors of each equation will be described qualitatively. A detailed quantitative presentation of the evaluation metrics can be found in the reference study [4].
The scalability (C1) of the layout is evaluated in the model under consideration on the basis of the area required in the future. For this purpose, the ratio of the current area and future unused area to the predicted area required in relation to the number of all departments is determined. The current area is composed of both the currently used area of a specific department and the associated buffer zones. Thereby, buffer zones are intended to keep space free for future demand. A differentiation is made whether the area of the layout is increasing or decreasing.
The evaluation of the modularity (C2) factor is based on the proportion of the total area that consists of modules. Modules are defined spatially, in order to exchange modules without effort in need of a change. Both the number of different modules and their size are defined in the data model. A number of three different square modules m is recommended. The modularity represents the ratio of the modular area to the total area, which ranges between zero and one.
The mobility is evaluated using two equations that examine mobility potential (C3a) and possible fix points (C3b) in the layout. The mobility potential evaluates the distribution of areas with an equal degree of mobility, which represents the freedom of movement of assets. For this purpose, different mobility degrees are defined in the data model. These range from high, medium, low to no mobility and provide an indication of the possibility of moving an area including the associated operating resources within the factory. The metric for evaluating the mobility potential examines the clustering of areas with the same mobility degree. For this purpose, the sum of the largest contiguous areas with uniform mobility degree is set in relation to the total area. Fix points are defined as elements that cannot be moved or changed. These are mostly elements of the building structure, such as ramps or elevators. These monuments, as well as areas with low mobility, should be placed at the edge of the layout. Consequently, the metric measures the shortest distance to the nearest static element of the building, such as to static or fire protection walls. The sum of the distances provides the evaluation result.
The changeability enabler compatibility (C4) is assessed by the media supply within the layout. The compatibility metric is intended to provide information on the uniformity of the media profile. Each area of a layout has certain requirements for the supply of media. The media profile results from requirements on the media supply as well as its availability. Not every area has the same requirements for media supply. Since it is not justifiable for economic reasons to make all media accessible at every location, different media profiles exist within the factory. In the course of this, it is recommended to spatially aggregate areas with uniform media profiles into clusters, in order to allow a homogeneous media distribution within each cluster. The aggregation is evaluated using the formula for assessing compatibility. For this purpose, the sum of the largest areas with a certain media profile is determined in relation to the total area. The aggregation of media profiles can be achieved by connecting two areas with the same media profile that are separated by another space.
The metric for evaluating universality (C5) provides an equation that takes into account the floor bearing load. The functional independence of the layout is realized by the area provided. The floor bearing load is a decisive factor for the possible uses of the layout. Consequently, this aspect must be taken into account when evaluating the layout's functional independence. In this case, the metric examines the aggregation of areas with the same bearing load, similar to the mobility and compatibility potential. Here, the sum of the largest contiguous areas with the same floor bearing load is set in relation to the total area.

Methodology-The scope of the sensitivity analysis
The identification of the most favorable layout variants is based on a variety of mathematical models. These models can be classified as part of information technology. According to Hevner, such models should be examined with respect to their functionality, completeness, consistency, accuracy, performance, reliability, and applicability [14]. In the course of a sensitivity analysis, an understanding of the influence of the variation of the input factors on the change of the resulting system response is provided. Based on Cullen et al. a sensitivity analysis can be used to investigate the reaction of a real-world system to an underlying mathematical model. A quantitative as well as qualitative statement about the expected change due to a parameter variation shall be made. In this respect, possible risks in the application of the approach are to be identified and minimized by improving and supplementing the existing metrics [15]. According to SieBertz et al. a distinction is made in the field of sensitivity analysis between the investigation of the qualitative influence, the influence of individual factors and the influence of the variation of all factors as well as their direct comparison. Consequently, the areas of investigation of the sensitivity analysis are divided into the so-called factor screening, the local as well as the global sensitivity analysis [16].
The presentation of the results in this work focuses on the findings obtained through the local sensitivity analysis. The local sensitivity analysis examines the individual evaluation metrics of the layout changeability. The analysis is intended to provide a more profound insight into the operation of the individual metrics for the evaluation of layout changeability. Therefore, the factor screening and the global sensitivity analysis will not be discussed in detail.
In the course of the local sensitivity analysis, all factors that influence the result are examined for their definition range. The definition range describes the set of all elements under which the metric is defined. The variables of the present metrics are taken from the underlying data and geometry model of the digital factory twin. These data are defined by the user. Based on these findings, the variable factors of the metrics are determined [17].
The variation of the factors provides the basis of the results, which may vary in positive or negative direction. In addition, the specifications, which were stored in the data and geometry model, are changed in order to determine a possible influence on the evaluation result. In the event, where the analysis results disclose a deficit in the evaluation metrics, suggestions for improving the metrics are made. This may involve adjustments to the existing metrics, or it may involve additions that evaluate other aspects of the target criteria that were previously neglected [17].

Key Findings-Improved performance metrics
This section presents the results of the metrics analysis. The objective is to compensate for possible deficiencies in the evaluation metrics by recommending actions to improve the existing metrics. In particular, the improvement shall enhance the significance of the metrics for evaluating the five factors scalability, modularity, mobility, compatibility and universality. In order to achieve a more valid result, the existing metrics were adjusted, or additional metrics were introduced. The variation of the metrics input factors provides the basis of the following results. For this purpose, the variation of the factors has to be determined. The variation can be in either positive or negative direction. In addition, the specifications, which were stored in the data and geometry model, are changed in order to determine a possible influence on the evaluation result.
For the investigation, three layout variants were created on a basis of a maximum of 2500m 2 planning area. The variants differ in their building shape as well as in the size of their fire compartments. Layout 1 has a square floor plan and is not restricted by predefined fire compartments. Layouts 2 and 3 have more complex building shapes. They are divided into three fire compartments. Each layout variant has six different areas A to F. Buffer zones are assigned to some areas, which are intended to provide expansion possibilities for the future. The entire free area of layout 3 is occupied by buffer zones. Figure 2 illustrates the three different layouts.

Scalability
The scalability evaluation presented in Sect. 2 is based on the future floorspace demand, which is commonly derived from a forecast value. Although this value is defined in the data model, it is in the nature of forecast values that they only reflect reality to a limited extent under certain circumstances. In the course of this, a deviation of A Future is to be expected. The evaluation of scalability presented in Fig. 3 has shown a difference in the evaluation, whether the actual area undercuts or exceeds the assumed future area. The formula has evaluated the underestimation of the required area more favorably than an overestimation. An underestimation of the future area should be considered more critical, as the scaling of the layout might reach its limits. Overestimation of the required area has a negative effect due to economically unused area, however, the scalability is guaranteed. The relative deviation of the future area is caused by the relative evaluation of the formula, which only calculates the ratio of the areas. Since this is a systematic error, it should be compensated. Based on these findings, the necessity of a correction factor for scalability in case Fig. 2 Representation of the layouts for the local sensitivity analysis of a demand underestimation of the areas can be derived. Following universality, overestimation of buffer zones promotes changeability by overestimating future area. Since the metric is based on areas of a current layout variant, the application area is limited to brownfield planning only.
Formula 1 presents a possible adjustment of the evaluation result with the help of a correction factor. The analysis of the scalability evaluation has shown that the correction factor minimizes the deviation in case of an underestimation of future area demands. In the case where the actual future demand for area exceeds the predicted demand, the case > 0 occurs. In case of underestimation of the area demand, the overestimation can be corrected by subtracting the correction factor 2 . (1)

Modularity
The evaluation of the modularity factor is based on the proportion of the total area that consists of modules. Burggräf et al. assume three different square shaped modules m in their work. Modularity represents the ratio of the modular area A m to the total area A total , which ranges between zero and one. Contrary to the initial form of the metric, layout areas should not be considered as single modules, but should rather be able to be constructed from multiple modules. Therefore, a higher degree of flexibility can be achieved within a certain area, due to the ability of faster paced changes. For the analysis four module combinations have been created, which differ between their overall and relative size to each other. Three different Layouts 1-3 are used to evaluate the coverage ration of each module combination. Figure 4 illustrates the modular design of Layout 1 for the module combinations A-D. The different modules have been fitted into the layout in order to archive the highest modular design, respectively the highest coverage ratio.
In this case changing the module size can affect the modular area and, respectively, the coverage ratio. Furthermore, It can be observed, that if more different module sizes exist, the more likely it is to map the total area congruently with modules. A high number of modules is associated with increasing individualization of the layout. However, this contradicts the principle of modularity, which is based on uniform standardized elements. The result of the analysis shows, that the smaller the dimensions of the modules are, the higher the coverage ratio. Nevertheless, the dimensions should have an application-dependent minimal dimension. An infinitesimal approach would not be consistent with the idea of modularity, in order to realize a most uniform surface of interchangeable elements. In addition, it has a positive effect on the evaluation result, if the module dimensions do not represent multiples of each other. Overall, no specific need for action to change the metric for mobility has been identified. Therefore, the metric remains in its original form, as shown in Formula 2:

Mobility
The evaluation of mobility is solely based on the mobility potential and fixed points. It is conceivable that the overall mobility of an area is increased as part of an investment in order to increase the changeability of the production system. In the course of this, it is of interest to investigate to what extent the increase in the overall mobility of the operating equipment is reflected in the evaluation of the mobility potential. Consequently, in the layouts investigated, the overall mobility of certain areas was systematically improved and compared with the original state, as Fig. 6 shows.
The results of the analysis have shown that the evaluation of the mobility potential is only based on the aggregation of areas with the same mobility as well as possible restrictions due to fixed points within the layout. The current actual overall mobility of the layout areas is not considered in the evaluation. However, the overall mobility is decisive for the mobility of the layout areas. If the relocation of certain layout areas is not possible, the evaluation of the mobility aggregation and potential restrictions within the layout would be partly obsolete. The overall mobility is considered a basic requirement for the evaluation of mobility potential as well as the consideration of fixed points. Consequently, it is necessary to quantify the qualitative overall layout mobility.
In the course of this, an addition to the mobility evaluation shall be presented with the aim to determine a more reliable result. In order to support the validity of the mobility evaluation, the actual mobility is considered in the form of a weighting. In this context, a mobility degree is assigned to each area, as in the calculation for mobility potential. The mobility degree is quantified with a weighting g ∈ ℕ . The lowest mobility degree g − is assigned the value 1.00. The highest mobility degree is described by g + , which is by definition the maximum number of different mobility degrees in the layout. To calculate the overall mobility, the sum of all weighted layout areas is put into the ratio of the total area A total , which is multiplied by the weighting of the highest mobility grade g + . The additional equation is presented in Formula 3: with the same degree of mobility plays an important role in achieving changeability, the evaluation of the mobility potential should still be carried out [4]. Together with the mobility potential, the mobility degree provides an evaluation of the relative as well as spatial distribution of mobility within the layout. Furthermore, it is possible to include the evaluation of fixed points in the calculation of the mobility degree. In this case, the fixed points are assigned exceptionally a weighting g = 0 . Consequently, the distance to the optimal evaluation result increases by an increasing number of fixed points. This addition to the existing metrics is intended to increase the informative value and reliability of the mobility evaluation.

Compatibility
According to Burggräf et al. it is essential to keep the number of different media profiles as low as possible and to concentrate them spatially [4]. For this reason, the influence of the number of different existing media profiles in the layout on the evaluation result is analyzed. In the course of this analysis, four different media profiles are defined in the original state of the layouts. This number is then iteratively reduced. The media profile with the smallest area is joined to the media profile of the nearest area. This process is repeated until there is only one general media profile in the layout. The iterative reduction of the number of media profiles and the associated evaluation result of the compatibility potential are shown in Fig. 7.
The results of the analysis have shown that an improvement of the evaluation result only occurs, if the largest area of a media profile p is expanded. In addition, the situation might occur, that all media profiles are already spatially aggregated. In this respect, the best result is already achieved by the evaluation metric. Under these circumstances, a further reduction of media profiles has no effect on the evaluation result. Furthermore, it can be observed that a high number of different media profiles favors the aggregation of areas with the same media profile. Provided that a media profile is only present in one area, this profile is completely spatially concentrated. Consequently, an increase in the number of media profiles with a consistent number of areas n leads to an optimum of the evaluation result.
This finding is contrary to the core of compatibility, to create as uniform and standardized interfaces as possible. The more media profiles there are in the layout, the more likely it is that the interfaces will be individualized. Consequently, the number of media profiles should be kept as low as possible in order to keep the objective of layout convertibility in mind. However, in the context of economic efficiency of the production system, it must be taken into account that minimizing the media profiles to just a single profile is hardly feasible.
The improvement of the compatibility potential cannot be pursued by minimizing the number of media profiles alone. The spatial concentration of the media profiles is equally crucial to achieve a high evaluation result. In particular, the number of resources provided by a media profile, such as electricity, compressed air, water or steam, contributes to overall layout changeability. In order to specify the metric, both tapphe number of media profiles and the number of resources bundled by the respective profile should be included in the evaluation. The adjustment is shown in Formula 4:

Universality
The floor bearing load of the building is recorded by the geometry model. A change of the bearing loads is accompanied by an extraordinary high effort. For this reason, a mathematical analysis of the floor bearing load modification is refrained from. Furthermore, the structure of the metric is identical to the metrics evaluating the aggregation of the mobility degree and the media profiles. However, the validity of the evaluation result should be interpreted based on the analysis results already obtained. The optimal result of the metric is achieved by aggregating all areas with the same bearing load t . This implies not only large contiguous areas with the same floor bearing load guarantee the functional independence of the layout. There is an eventuality that multiple areas with different bearing loads could also achieve a high rating. This situation would allow only limited room for movement in the layout design since the areas would depend on the permitted bearing loads. In the course of this, the result of the metric would be in contradiction to the functional independence of the layout. An adjustment to the existing metric is proposed, which is similar to the adjustments beforehand. As the number of different load capacities has a great negative impact on the layout universality, this factor should be considered in the evaluation. A higher number of different load capacities n leads to a decrease of the evaluation result due to the correction factor 1 n . The adjustment of the metric is presented in Formula 5: Another factor that plays a decisive role in the universality of the layout is its ceiling height. A uniform, sufficient ceiling height ensures functional independence for the layout. As a result, individual layout areas with height requirements are not tied to specific ones within the building. These requirements may be due to machinery or equipment with exceptional heights.
In the course of this, a way to include ceiling height in the metrics for evaluating universality is presented. The objective is to provide a statement about the independence of ceiling height. This is done by means of a relative evaluation of the proportion of space that meets the clearance requirements. Provided that the room height of the building is greater than the maximum required height of the layout areas h max , universality is guaranteed. This value is at best a forecast, which is determined on the basis of future scenarios. The sum of the areas A H,i that meet this criterion is The results of the local sensitivity analysis provide an important insight into the origin of the evaluations. The interpretations of the respective results derived therefrom must be taken into account when applying the metrics. Nevertheless, the planner has to consider that optimal layout changeability can only be achieved by a combination of all enablers. An overview of the derived adjustments to the systematic of the layout changeability evaluation is illustrated in Fig. 8.
The metric referring to the layout scalability has been adjusted to resolve a systematic issue, which had been identified in the course of this study. The evaluation of the layout modularity has proven to reliably identify the highest modular covering ratio of the layout. Nevertheless, it has been observed, that the definition of the module sizes has a significant impact on the evaluation outcome. In order to enhance the evaluation of the overall mobility, a third metric has been added. The adjustment to compatibility evaluation now includes the number of different media profiles within the layout, which shall be minimized. An additional (4.6) Ceiling Hight C5b = metric considering the ceiling height has been added to the layout universality evaluation. The load capacity evaluation has been adjusted in order to achieve a more uniform load capacity distribution within the layout.

Conclusion and outlook
Automated layout design offers a great potential, increasing the agility and responsiveness of factories. Approaches for automated design and optimization of layout variants based on quantitative criteria are already widespread in research but find sparse application in industry due to the lack of qualitative criteria. Ultimately, the greatest challenge lies in layout evaluation automation by integrating qualitative criteria. The identification and selection of the most favorable layout variants represents a multi-criteria decision problem that does not depend solely on quantitative criteria such as transport costs and distances. Rather the complexity of layout planning is increased by considering qualitative criteria, particularly changeability. Thus, the approach of Burggräf et al. provides a foundation for the automated evaluation of layout changeability. The explicit and reliable formalization of layout changeability is a crucial aspect of automated layout design. In the course of the performed analysis, all factors affecting the evaluation results were identified as well as their respective influence. Based on the analysis of the validity of the scoring results, recommendations for action are suggested to Fig. 8 Overview of the adjusted systematic of the layout changeability evaluation increase the reliability and quality of certain metrics' scoring results. In particular, the existing metrics for Scalability C1, Compatibility C4 and Load Capacity C5a were adjusted in order to archive more significant increments in the layout evaluation. Furthermore, two new metrics Mobility Potential C3c, and Ceiling Height C5b were introduced, considering qualitative criteria which has not been factored in previously. The validation of the additions to the existing metrics implies an overall increase in the validity of the evaluation results. The layout changeability is identified more precisely with the help of the introduced metrics.
However, the holistic approach does not take into account all indicators which ensure layout changeability. The changeability often relies on individual factors, depending on the specific use case. For the formalization, a trade-off must be drawn between factors that increase the ability to change in a particular case and those that can be applied in any setting. In this regard it should be emphasized that some metrics are restricted to either brownfield or greenfield planning.
Building on the results and improvements of this study, a representative survey with participants from industry and science should be conducted, in order to validate the present key findings of automated qualitative layout evaluation. Furthermore, a second validation of the metrics should be performed in a real-world environment or simulation to verify the applicability of the established metrics.
Funding Open Access funding enabled and organized by Projekt DEAL.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.