Approach to Reference Models for Building Performance Simulation: Establishing Common Understanding

In the fields of business process modeling, logistics, and information model development, Reference Models (RMs) have shown to enhance standardization, support the common understanding of terminology and procedures, reduce the modeling efforts and cost through the paradigm “Design by Reuse”, and enable knowledge transfer. Utilizing RMs in Building Performance Simulation (BPS) shows potential to achieve similar benefits. However, there is no universally agreed understanding of RMs. In a previous scientific publication, we provided a comprehensive overview of the diversely interpreted definitions, benefits, and attributes of RMs and related terms. Additionally, to transfer the approach of RMs to BPS, a definition for RMs applicable to BPS has been provided, and the identified RM qualities were matched with BPS’s challenges. However, a sound evaluation of the success of transferring RMs to BPS is lacking. Therefore, this scientific contribution firstly includes the analysis conducted in the previous scientific contribution constituting a common understanding about RMs and their elements for BPS. Secondly, by conducting expert interviews, the applicability and validity of the developed concept of RMs for BPS are surveyed. In total, ten experts (seven BPS experts and three RM experts) evaluated the quality of creating transparency about the understanding of RMs and the level of success of their transfer toward BPS. The experts consistently see a great benefit of RMs in BPS, but for BPS experts the transfer and possible application of RMs in BPS is not sufficiently clear. Accordingly, the key output of the conducted survey is that a clearer and more detailed application example, e.g., describing at a more easy-to-understand level of detail an exemplary class of the provided example of an RM, is required for a more profound transfer of RMs to BPS.


Introduction
Modeling and simulation of buildings and Heating, Ventilation, and Air Conditioning (HVAC) systems have become an established practice, both in the research and industry, to manage the increasing complexity of Building Energy System (BES) interactions and tackle the global targets to their decarbonization [1]. These applications are designated with the term Building Performance Simulation (BPS). The use of simulation and computational models can support the BES life cycle [2], from the design process until the commissioning and maintenance phases.
In general, model-based engineering, which adopts models instead of directly realizing a solution [3], leads to frontloading efforts during the development process. Therefore, it supports an early-stage concept verification and, hence, faster and more efficient time-to-market, improving the chances to detect errors early.
Nevertheless, BPS and likewise model-based engineering induce several challenges. The design of reliable and accurate mathematical models is time-consuming and compels experts and cost; therefore, there is a need for model reusability [4]. Moreover, besides the intrinsic multi-disciplinary approach to BES [5], the increase in system complexity (e.g., integration of elements of the so-called Internet of Things) has resulted in a closer interaction of various disciplines [4]-architecture, engineering, as well as IT and data science. Consequently, the models' transparency, their ease of share, and common understanding among different experts become fundamental. Furthermore, facilitating knowledge transfer of BPS processes and procedures would further spur their adoption across the whole BES life cycle [6]. Eventually, a higher model abstraction and modular approach helps reducing the comprehension efforts as well as enhance simulation program debugging and, additionally, model maintenance and portability [4].
An RM is a conceptual framework "for understanding the significant concepts, entities, and relationships of some domain, and therefore, a 'foundation' for the considered area" [7]. RMs have been applied primarily in the fields of information systems [8,9], virtual enterprises [7], production and logistics simulations [10,11], business administration, and informatics [12]. In these sectors, the most recurring benefit is the saving of time and cost efforts during the development phase of new models, because an RM enables the Design by Reuse paradigm [10,13,14]. Moreover, by standardizing and systematizing best practices [15], they have proven to increase the quality of the to-be-realized model [8,11]. Another recognized benefit is that the use of RMs leads to recommendations for actions to derive measures for improvements [10,16]. In addition, an RM fosters the communication between different experts by bringing together the subjective views [12] and, thus, builds a foundation for a common terminology and common procedures [7,11]. Less noted but still relevant is that RMs guide simulation of logistic processes enabling easier interaction with the simulation models [15]. RMs enable knowledge transfer [17] and support educational purposes, such as employee training [9].
We expect that similar benefits could be achieved by transferring the concept of RMs to BPS. For example, a reference model for BPS can be developed that provides both reusable simulation modules and a reusable approach for developing simulation models for BPS. Among other things, this can lead to an optimized knowledge transfer by making the models easier to understand and utilizable for multiple entities. Therefore, in a previous scientific contribution [35], we have identified challenges of BPS and proposed RMs as a potential way to face these challenges. In order to utilize RMs in a certain field, a clear understanding of their architecture and characteristics is mandatory. A comprehensive literature research showed that this universally agreed understanding of RMs cannot be found, as also stated by Bartsch, Thomas, and Camarinha-Matos and Afsarmanesh [7,8,12]. Consequently, we investigated the common understanding and specific understanding in different implementations as well as identified qualities of RMs. On this background, we proposed a definition applicable for the field of BPS. Finally, an example of an RM for simulation-based test benches was provided. However, the developed common understanding about reference models for the BPS domain, including the provided definition and example of use, was lacking application and validation. Accordingly, within this scientific contribution, this lack is addressed and, based on an expert survey, the developed approach of RMs for BPS is critically reviewed and their applicability and usability for BPS is explored. The ten experts consulted are composed of seven simulation engineers working directly in the field of BPS for the company Bosch Thermotechnology and three experts in the field of reference modeling.
This scientific contribution, as an extension of the previous one, includes the overall research conducted on (1) different perceptions on RMs, (2) the architecture and elements of RMs, (3) a suitable definition for BPS, and (4) a survey on the intended increased transparency on RMs and the level of success of their transfer to the field of BPS. An additional result of the expert survey is the validation and extension of the identified challenges of BPS found in the literature. Therefore, the overall research goal of this scientific contribution is not only to present the approach positioned in a previous scientific contribution, but also to provide experts' evaluation about the applicability and value proposition of the proposed approach for RMs for BPS. Investigating on the following research questions shall give insights about lacking elements for transferring the approach and making it utilizable in the field of BPS: • What are attributes, aims, and benefits of RMs? • How are RMs to be defined for BPS? • Are RMs a possible approach for further value generation and countering challenges of BPS?
This paper is structured as follows: "Understanding of RMS" presents the state of the art in understanding the term RM. "Transferring the RM Approach to BPS" documents the authors' suggested definition for the term RM in the field of BPS as well as the related attributes and qualities, showing the benefits of RMs. The understanding of RM is supported by an application example in "An RM for Simulation-Based Test Benches". "Expert Survey" includes results of the experts' survey regarding challenges for BPS, the intended increase in transparency on RMs, and their value proposition for BPS. Finally, in "Recapitulation and Outlook", conclusions are drawn with an outlook for future scientific work.

Understanding of RMS
The term Reference Model (RM) emerged in the literature at the end of the 1980s for the development of industrial enterprise models and pertains to a class of words that are often used but seldom clearly understood [8]. Reference modeling is the process of developing an RM to be used for different applications [9]. From a pure etymological perspective, the term reference model consists of the words reference and model. These have, respectively, the meaning of "quoting something" and "remarkably good example that can be imitated" [18,19]. Nonetheless, an agreed understanding of RMs is lacking, and diverse definitions are offered, depending also on the application field. Actually, the denomination RM is sometimes used without any well-founded qualification [20].
A model itself is an abstract formal representation of a portion of the real world [17]. A model can be used to understand, explain, design, and implement a system [7,21].
Van der Aalst et al. report that RMs provide generic solutions for developing specific models [13]. Bartsch adds that RMs are understood as a specific manifestation of a general type of abstract model having certain characteristics [12]. Furthermore, Pajk et al. declare that "RMs are generic conceptual models that formalize recommended practices for a certain domain" [22]. Accordingly, Rabe et al. define an RM to be a conceptual framework that includes a standard description of processes and best-in-class practices [11]. In Camarinha-Matos and Afsarmanesh (2008) the authors state an RM to be an "abstract representation of a large number of possible systems" [7]. Eventually, Thomas (2006) offers a user-centered definition: An RM is a user-accepted model that can be exploited (and re-used) in supporting the construction of another model [8]. Based on this definition, an RM requires that at least one application of it can be found.
It can be noted that while there is no universally agreed definition, there are nonetheless commonalities to be found regarding their characteristics and benefits. Based on the investigated contributions, the application of an RM is generically illustrated in Fig. 1. First, the required elements of the RM are selected. At the same time, also the required elements of the model to be created are to be identified. These two steps support each other iteratively, hence they already represent an initial application of the RM. Subsequently, the latter is to be applied profoundly by substituting already developed elements from the RM into the model to be created. Possibly, not all required elements are covered by the RM. These, therefore, may need to be developed afterward, but with an overall significantly lower effort.
Nevertheless, in order to use the full potentials of RMs in the field of Building Performance Simulation (BPS), a deep understanding of RMs and their conception is required. To meet the need for more transparency and to later guide the identification of an RM definition applicable to the field of BPS, in the following the authors present the RMs' attributes Subsequently, the authors present the clustering process that leads to the aggregated RM attributes' synthesis into qualities.

RM Attributes and Qualities
By investigating eighteen relevant contributions, a total of 41 attributes of RMs are identified and clustered to nine qualities (Fig. 2). These qualities, providing a profound understanding of RMs' characteristics, are reusable, flexible, reliable, designed systematically, generally valid, required, user-centered, comprehensive, and educative.
The attributes adaptable, applicable, customizable, and configurable enable the RM to its quality of reusability (Q1). On one hand, there is a need for a high abstraction level-abstract from specific features [9,12,23]-as the RM should be applicable to various homogeneous fields. On the other hand, a high level of detail is required [24] to offer guidelines to ensure the RM's ease of use [17,23]. This conflict of goals goes together with the inconsistency in literature about whether an RM should be tool-independentonly referring to them [25]-or tool-related [24].
A modular and hierarchical structure consisting of a composition of submodels, allowing a wide range of choices [13], leads to flexibility (Q2) [14,15].
The quality of generally valid (Q3) consists of the attributes universal, transferable, and valid in a specific field when meeting corresponding specified conditions [16,17,23]. Therefore, there is no claim to an absolute universal validity, but to a general validity in a class of applications [8].
Noteworthy, qualities Q1, Q2, Q3 present fuzzy boundaries as their attributes overlap. This is the case, e.g., for the attribute customizable, which can be entirely associated neither to the quality flexible, nor reusable, nor generally valid. Moreover, there is a strong interrelation of the quality Q3 with Q1 as being generally valid is necessary for the RM to be reusable.
In order for the user to be confident in applying an RM, the quality of reliability (Q4) has to be ensured [17]. Accordingly, an RM should be credible, e.g., by observing best practices [9,17,24], as well as disclosing the sources cited and the authorship [7]. It should, in the best case, already be validated or at least validateable [12,23]. Finally, it is necessary that the user accepts the model as a reference and that the RM is applied at least in one case [8].
To justify the RM use, the quality of required (Q6) is crucial. Attributes of this quality are the usefulness and utility of the RM [12,24] and, if applicable, its innovativeness [9,12].
Being user-centered (Q7) is a fundamental quality of RMs. This quality implies ease of use [15], visualization character [12,24], and providing a definition of the meaning and the purpose of the RM [17] together with its correct and efficient use [11]. Ultimately, there is a need for syntactic (well-defined linking and combination of elements) and semantic (well-defined content) completeness [16,24]. This semantic completeness is often supported by a formal description technique [9,14,26].
An RM should include the quality of being comprehensive (Q8), both in its development and application [24]. By showing relationships between activities and entities [9], an RM can provide multiple perspectives and scenarios of application depending on the current boundary conditions [13,24]. Q8 also enables a continuous improvement by allowing the benchmark of the as-is and target status [7,10,24].

Prioritizing the Compiled Qualities
The occurrence of the detected qualities in the respective contributions is counted to determine their prevalence, hence allowing for ranking them (Table 1). There seems to be a particular consensus regarding the qualities reusable (Q1, 94%), generally valid (Q3, 78%), user-centered (Q7, 72%), educative (Q9, 72%), flexible (56%), and comprehensive (50%). These qualities, which reach prevalence above or equal to 50%, are, therefore, classified as common perception of RMs. At this point, it should be pointed out again that some qualities have fuzzy boundaries (see "RM Attributes and Qualities"). The quality flexible, for example, shows a high attribute overlap with reusable and generally valid.
The remaining identified qualities are not shared by the majority and are, thus, seen as additionally annotated qualities. Regardless, a model should be reliable (Q4, 33%) and systematic (Q5, 22%), thus supporting trustworthiness, reusability, and user centricity. Increased reliability, for example, can be achieved by an initial application or even validation of the developed RM. The lowest weighted quality is required (Q6, 17%). This result is to be questioned critically, as the requirement itself might already be expressed by the creation of the RM. The detection and occurrence of these qualities is intended to show the common and uncommon perception of the investigated contributions, but by no means to exclude uncommon perceptions. Instead, the goal is to provide the fundamentals for a viable definition and general understanding of RMs in order to integrate them to the field of BPS.

Transferring the RM Approach to BPS
As stated in "Understanding of RMS", a model is "something that a copy can be based on because it is an extremely good example of its type" [19]; it is an abstract formal representation of the investigated portion of the world [17]. Consequently, as a premise to this section, the authors underline that it should not be misunderstood as a building or HVAC simulation model, which represents a digital counterpart of physical phenomena and is used for predicting and understanding their dynamics. The investigation on attributes and qualities of RMs ("Understanding of RMS") indicates the presence of ambivalent perspectives, which is partly the result of different needs of different application fields. Consequently, this section presents the authors' proposed general definition of RMs (based on the state of the art), which is applicable likewise to BPS and is essential for transferring the RM methodology to the latter field.
An RM is a holistic collection of methodologies systematically structured in an architecture in which every element (e.g., guidelines, methods, procedures, and entities) is made transparent and outlined as a generic solution based on both best practices and innovative approaches.
The main objective of such an RM in the field of BPS is empowering flexible reuse of existing knowledge and practices, spurring the adoption of model-based engineering, in turn, leading to efficient development of high-quality solutions.
Based on this definition, RMs require a conceptual architecture (synonym: framework) that itself is an organized view of the RM (Fig. 3).
As reported in Camarinha-Matos and Afsarmanesh (2008), an architecture defines a specific system in an abstract way [7]. This architecture is a logical collection of elements that should be described with the help of a modeling language, which emerges to meta-models. A meta-model is a model of a model "describing the syntax of the model and generalizing the semantic" [16] with the help of a meta-language [12] (e.g., Unified Modeling Language (UML), system Modeling Language (SysML), programming languages).
As depicted in Fig. 3, meta-models can be regarded as the architecture's foundation. There may be different meta-models needed for various purposes depending on the described element.
Typical architecture elements are guidelines [7], which suggest predetermined procedures or good practice methods that can be adopted to ease a particular process [29]. Other common elements of the architecture are maturity models, which are usually linked to a process. They are perceived as benchmark tools to evaluate weaknesses and strengths of the as-is status that later can be used to guide optimization or better lead to recommendations for actions. Exemplary tools can be part of the architecture, supporting the understanding and applicability of the RM as well as enabling the RM validation. Noteworthy, Fig. 3 shows that elements can have different abstraction levels. This is the case, for example, of the element entities and their physical and non-physical sub-elements.

SN Computer Science
A systematic design (Q4) of the RM is enhanced by an elements' classification, making the latter easier to apply. Because classifications highly depend on a specific use case, they are not detailed further in this paper, which addresses a broad BPS perspective.
The RM definition provided comprises the qualities identified in "RM Attributes and Qualities". Therefore, applying the approach to an RM in BPS supports facing the challenges outlined in "Introduction".
The RM qualities of being reusable (Q1) and flexible (Q2) allow for overcoming the development efforts related to time and cost. They lead, for example, to avoid the development of a new simulation model from scratch. Generally valid (Q3) ensures an easier and wider transfer of the established practices in a certain BPS domain to another, e.g., district and urban or residential and commercial building modeling.
By collecting and systematically presenting methodologies in a conceptual architecture (Q5) and by being reliable (Q4), an RM fosters standardization, which leads to a quality increase of the resulting applications of BPS through the whole BES lifecycle, as well as cross-study benchmarks (e.g., compare modeling assumptions). Additionally, the educative (Q9) and user-centered (Q7) characteristics, by disclosing the adopted methods and procedures, can be used not only to transfer knowledge from experts to non-experts, but also as a teaching instrument for students or inexperienced employees. Q7 combined with comprehensive (Q8) lead to more overall transparency of the simulation models, promoting, in turn, their ease of sharing, understanding, debugging, and conducting maintenance.

An RM for Simulation-Based Test Benches
To enhance the understanding of the presented RM benefits and clarify how an RM application in the field of BPS would look like, in the following, the authors present a preliminary example.
The performance of newly developed building control systems can be assessed by the three complementary approaches (a) field-test, (b) emulation, and (c) simulation. The latter relies on a simulation model of the system to compute the Key Performance Indicators (KPIs) and verify as well as benchmark the tested control's quality.
This model-based concept is promising to overcome disadvantages related to approaches (a) and (b), when regarding test coverage as well as time and cost efforts. Nevertheless, since such a simulation-based approach relies on BPS, it faces the challenges outlined in "Introduction". Furthermore, there is the need for a higher standardization and systematization of approach (c) to ensure results generalization, fair cross-study comparisons, and high test quality [28]. Approaches (a) and (b) are established in the industry, while the potential of (c) tends to be not fully acknowledged.
As presented in "Transferring the RM Approach to BPS", RMs are a viable answer to the stated challenges. Therefore, an RM for implementing virtual test environments to assess the performance of building control systems is conceptualized. To ensure the RM flexibility (Q2), the architecture is modular and consists of four categories and nine classes (Fig. 4). These identified elements (of the architecture) result from grouping the procedures and methods required to develop a virtual testing concept. Moreover, to design the RM architecture, the trade-off between the level of model abstraction and granularity has been considered to ensure its reusability (Q1) [11]. As depicted in Fig. 4, the categories are (1) what-if scenario identification, (2) simulation model design, (3) performance assessment, and (4) results benchmark.
Category (1) describes the methods to identify the parameters and variables required to generate the what-if scenarios to ensure a sufficient test coverage and quality (avoid the "garbage-in, garbage-out" paradigm). In particular, this category contains the classes test location, occupants, building, Fig. 4 Conceptual architecture of the RM for simulation-based test benches and HVAC system, which identify the representative features of the built environment. Category (2) guides the design of the simulation model. Except for the class simulation environment, all the contained classes are covered also by category (1). The design of the simulation model affects the input data that are required to characterize the what-if scenarios. Consequently, the authors recognize that categories (1) and (2) must overlap to ensure the RM comprehensiveness (Q8). Noteworthy, the class simulation environment leads the implementation of the virtual test on software. The performance assessment methodologies are present in category (3) that allows defining the reference control for the benchmark and the evaluation metrics (KPI selection). Category (4) guides the result postprocessing and benchmark phases: once the raw data are available, effective result visualization and a test report are to be ensured for attracting the stakeholders' interest and supporting the design phase of the control under test.
Each class consists of a set of structured steps, which the user should follow to implement a simulation-based benchmark in a standardized and systematic way. Based on the use case scenarios, flowcharts or activity diagrams offer the user a panorama of choices (Q8) and problem solving know-how (Q9).

Expert Survey
The centerpiece of this scientific contribution, which is an extension of a previous paper, is a survey of expert opinions. A total of ten experts from the fields of reference modeling (three experts) and BPS (seven experts) are surveyed with regard to the following aspects: • Challenges for BPS • Understanding of RMs before reading the previous scientific contribution • Understanding of RMs after reading the previous scientific contribution • Degree of increased transparency about RMs for BPS • RMs' value proposition for BPS

Survey on Challenges for BPS
As a first step of the survey, the experts are asked to assess the following challenges for the field of BPS identified in the previous scientific contribution: • C1: High simulation model development efforts [4] • C2: Increasing complexity of building energy systems [4] • C3: Multidisciplinarity [4,5] • C4: Targeting simulation model reusability and standardization [4] • C5: Non-transparency of simulation model [6] • C6: Need to increase the amount of established applications of BPS along the building energy system life cycle [6] • C7: Increasing environmental requirements on building energy systems [31] With regard to high simulation model development efforts (C1), all ten experts agree for this to be a challenge for BPS. Three of them even strongly agree to this challenge. They note that the development and implementation of a simulation to increase accuracy is time-consuming and cost-intensive if no standard for reuse exists.
An increasing complexity (C2) of already complex physical interacting systems (e.g., hybrid HVAC, district and grid interactions, smart buildings) is rated as a relevant challenge of BPS by eight out of ten experts, whereby two strongly agree. However, one expert notes that modular models can already manage complexity. One Expert disagrees to C2 stating high complexity is not a challenge, but gives reason for BPS.
Likewise, there is hardly any discrepancy between expert opinions at the challenge of multidisciplinarity (C3), which follows from the close interaction of various disciplinesarchitecture, engineering, as well as IT and data science. Eight out of ten experts agree on C3 for being a challenge for BPS, while four of them even strongly agree. Two experts remark that multidisciplinarity is much more likely to motivate BPS.
Reusability of simulation models (C4) can only be reached to a very limited extend because of very little standardized model development. Seven out of nine valid expert answers coincide with this statement. One expert states: "for some cases we do have very good approaches, in other cases we don't". Still, one Expert disagrees stating that C4 is instead a prerequisite for BPS.
Seven out of ten experts agree on that non-transparent simulation models (C5) are a challenge for BPS, whereby three of the experts even strongly agree. Convoluted models make their understanding and debugging difficult making it necessary to have a complete structure that is supported by a modular structure. Two Experts neither agree nor disagree. One of those states that higher transparency of models can also lead to a possible higher confusion of the users. The other expert states models are transparent enough just need to be shared. Another expert, again, rates C5 rather as a prerequisite than as a challenge.
Six out of seven valid answers agree on the challenge of spurring the adoption and utilization of BPS across the whole BES life cycle (C6); three of them even strongly agree. One Expert neither agrees nor disagrees to C6.
The last surveyed challenge shows a large discrepancy. Out of nine valid answers, three agree that for BPS increasing environmental requirements on building energy systems is a challenge. Three neither agree nor disagree mentioning carbon accounting is not more difficult for BPS than it is for a real building. Another three disagree stating that BPS is rather an answer for increasing environmental requirements. Figure 5 shows the expert answers' average (vertical per surveyed challenge). This evaluation is intended as a graphical illustration and cannot be generalized. Since the data are qualitative, the quantification performed here is not generally representative [32]. It can be concluded that, on average, the surveyed experts consider challenges C1 to C6 to be relevant challenges for BPS, while C7 can rather be seen as a global challenge for Building Energy Systems and a motivation for BPS.
The experts not only were surveyed regarding previously identified challenges, but also had the opportunity to add further challenges with a respective prioritization (high, medium, low). Two experts independently indicate that controlling the quality of developed and utilized BPS models is a high-priority challenge. Many influencing parameters, different interpretations, different requirements, and different methods make it tricky to assess the correctness of a model. Another high-priority challenge, according to one expert, is to handle big data in terms of volume, speed, and quality. One expert also adds another high-priority challenge of meeting domain-specific requirements. Medium-prioritized challenges are (i) postprocessing and communication of the BPS model (shared by two experts), (ii) discrepancy between standardized and individual BPS models, (iii) increasing grid requirements for building energy systems, and (iv) identification of relevant requirements for the specific application domain.

Survey on Previous Understanding of RMs
All experts, three from the field of reference models and another seven from the field of BPS, were interviewed regarding their understanding about RMs and their benefits before reading our previous scientific contribution.
The three RM experts jointly note that a reference model for the field of simulation provides a structure for creating a simulation model and should, therefore, be classified at a higher level of abstraction than the simulation model itself. The enumerated advantages here are the comparison, communication, and usability of knowledge for certain domains and certain boundaries, and its explicit application. The BPS experts have a quite different idea. According to them, a reference model is a standardized model, e.g., of a building, in which different elements of the building can be put together. One of them notes that an RM is a kind of simulation model library, which contains different simulation models of different physical elements. Here, the advantages reusability, benefit of computational effort instead of actual physical effort, and optimization of efficiency and energy consumption are noted. One BPS expert had not heard of RMs before.
When comparing the concept of RMs developed in "Transferring the RM Approach to BPS" and the experts' understanding, it is clear that the RM experts have a similar understanding, but the BPS experts rather consider an RM to be a standardized simulation model.

Survey on Updated Understanding of RMs
The understanding about RMs and their benefits after reading our previous scientific contribution has not changed among RM experts, but among BPS experts a fundamental change can be observed. The three RM experts and five of the seven BPS experts now share the view that an RM is a kind of framework for the modeling process, which includes a modular toolbox of guidelines, processes, and methods. It also includes best practices and innovative approaches. An RM is built like a "Lego-brick model" from which you can take elements you need for developing a simulation model, including important information while abstracting many complexities. The RM can be a reusable process model and is described formally independent from a simulation model. The advantages noted are reusability for addressing similar challenges, an easier requirements analysis, a transfer of good practices and existing knowledge, and an easier product development. However, after reading the concept, two BPS experts still consider RMs as a "good standard" and a simulation library containing several simulation models.
We can conclude that after reading our previous scientific contribution, the perception of the experts about RMs and their advantages coincides with the contents of the paper. According to this, a comparatively basic unified understanding about RM could be achieved.

Survey on Degree of Increased Transparency About RMs and Their Value Proposition for BPS
On top of the change in understanding about RMs, experts were also interviewed regarding the degree of increased transparency about RMs explicitly for BPS. While seven experts note that the topic is prepared in a scientifically systematic and well-founded manner, all ten experts agree that the transfer of RMs to BPS requires more detail regarding the approach. It is noted that too little detail is provided on how RMs address BPS problems and challenges. Accordingly, the concept of RMs has become clearer to the experts, but the practical purpose has not. Figure 6 illustrates the average rating and the rating interval of the transparency of the presented RM approach divided into RM and BPS experts. The RM experts perceive the described approach as fairly clear, while the BPS experts show a higher discrepancy and on average rate the approach as being only moderately clear.
As practitioners, the BPS experts initially find it difficult to grasp the core of this scientific research at first. It is noted that for practitioners there is too much unnecessary description around the key statements. At the same time, the transparency is increased by the condensed figures, the systematic processing, and the presented RM Architecture overview. The architecture overview (Fig. 3) is considered useful, but the interaction and classification of the different elements were not clear.
Eight of the interviewed ten experts agree that more application-related examples, e.g., by detailing the given application example, lead to an increased transparency of how, when, and where RMs are to be used. The interaction and classification of different elements of an RM should be included and clarified in the application example of an RM for simulation-based test benches ("An RM for Simulation-Based Test Benches"). Here, the experts explicitly call for the details of selected modules of the reference model presented. Only four out of eight experts consider the provided application example to be useful for the transfer of RMs to BPS. A conceivable reason for this, confirmed by all experts, is that detailed examples and the link between the elaborated concept and the provided example are missing. The embedding of the example in a simulation environment is explicitly requested by the experts, which should make clear, e.g., which modeling language is used, which software is used, what the general procedure is, and what comprises an RM. One expert even suggests incorporating this approach into an IT tool to make it as easy to utilize as possible, as "glorious steps are always lost if not incorporated in IT".
All ten experts state they would use an RM for BPS like described in this paper in their daily business, e.g., for onboarding how to do modeling by setting up reference models for heat pumps, a higher work efficiency and profitability by avoiding model development from scratch, and for detailed requirements engineering. For developing such RMs for BPS, the surveyed experts identify the following points to include:

Recapitulation and Outlook
BPS shows optimization potential in terms of modeling effort and costs, practical reusability, and standardization as well as knowledge transfer; this can be tackled by applying RMs. However, due to the diversely interpreted definitions, benefits, and attributes of RMs in different established domains, the prerequisite for transferring RMs to BPS is the creation of terminological transparency.
To tackle the research objective of identifying attributes, aims, and benefits of RMs, the state of the art regarding the understanding of RMs is shown and analyzed. Further, an RM's attributes are collected and clustered into nine qualities, which characterize and distinguish them. The eventual transfer of RMs to BPS is achieved by providing a unified definition of RMs, valid also for BPS, based on the identified This study provides a first approach to RMs for BPS, which offers the benefit of collecting both best practices and innovative approaches with a user-centered approach, in a structured and systematic way. Additionally, this scientific contribution as an extension includes the results of an expert survey on the challenges for BPS, the understanding of RMs before and after reading our previous scientific contribution, and the degree of success of transferring the approach of RMs to the field of BPS. The expert survey conducted with BPS and RM experts provides insight into the research objective, whether RMs are a possible approach for further value generation and countering challenges of BPS. Basically, the challenges for BPS previously identified from the literature were confirmed by the experts. Particularly noteworthy are the challenges of high simulation model development efforts, multidisciplinarity, and the need to increase the amount of established applications of BPS along the building energy system life cycle. Increased environmental requirements were rated as a motivation for BPS rather than a challenge. In addition to the surveyed challenges for BPS, the experts also noted other challenges. Particularly relevant challenges are, for example, controlling the quality of developed and utilized BPS models, handling big data, and meeting domain-specific requirements.
The RM experts consider the provision of transparency about the diversely interpreted concept of RMs as valueadding. The BPS experts, as potential practitioners of this approach, confirm this view of a structured creation of transparency, but are reserved about the lack of application emphasis in the paper. All experts agree that the application examples and the detailing of the still very rough example of a RM for simulation-based test benches ("An RM for Simulation-Based Test Benches") will add value to the transfer of RMs to BPS and, thus, trigger the practitioners to take advantage of this approach. Additionally, implementing the approach into an IT tool might further spur its utilization.
The results match our intention of exploring the understanding about RMs in different domains, creating transparency about various interpretations, and defining RMs valid for the BPS domain. This scientific contribution was intended to be holistic, which, however, leads to a too abstract level for practitioners with less information how to utilize RMs. Therefore, it shows the need for further scientific work.
A maximum confidence of the qualitative study was aimed at by analyzing the data systematically and rule-governed with the help of a category system as suggested by Kuckartz (2014), an expert in qualitative content analysis [32]. Nevertheless, a subjective influence of the researchers cannot be excluded, since a qualitative content analysis is an interpretative procedure [33]. Explicit and detailed perceptions of social actors could be captured. According to Koch (2016), this is a valuable advantage over quantitative research, which delivers rather superficial and above all standardized results [34]. However, the results of a qualitative content analysis are limited. Nevertheless, hypothetical approaches to knowledge can be established. These are partly epistemologically not validated and are to be understood as a basis for potential further investigations [33].
The research goal of investigating the suitability of RMs for addressing the challenges of BPS and, thus, transferring RMs to the field of BPS could be approached. Hence, a clear structure and understanding of RMs for BPS could be established, but the limitations, as also outlined by the surveyed experts, are particularly the lack of RM solutions that can serve as pilot models for the field of BPS. Therefore, a main future research focus is the pursuit of tangible application of the introduced approach to BPS by developing mature reference models. Especially for use cases where data availability is of high performance, RMs show high potential, e.g., the introduced example of BPS and other promising building energy systems' use cases, such as data analytics in reliability prognosis.
Funding Open Access funding enabled and organized by Projekt DEAL.

Conflict of interest
The authors declare that they have no conflict of interest.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/.