Keywords

1 Introduction

Visualizing heterogeneous building information enhances the decision-making processes of stakeholders in a building’s lifecycle and is essential for (non-)professionals to understand a building’s performance [1]. Designers use visualization tools to improve their designs. Technicians use them to run simulations and perform calculations. Once a building is built, facility managers and maintenance workers could use visual information to monitor their buildings and improve the maintenance process. As state-of-the-art architectural design challenges are interdisciplinary by nature, interconnecting heterogeneous building information into a single viewpoint would stimulate cross-domain decision-making and change the rather linear way of design and planning to a more circular approach that enables improving the building throughout multiple lifecycle stages [2].

The heterogeneous nature of data in the AEC (architecture, engineering, and construction) industry introduces challenges to integrating building information in a single viewer. Data comes in different formats, is often stored at different locations, and is owned and maintained by different stakeholders. Next to this, data is generated in different phases of the building’s lifecycle [3].

The IFC (Industry Foundations Classes) schema is the de facto standard in the AEC industry and allows stakeholders to exchange building information across multiple BIM authoring tools. However, IFC has some challenges related to data integration in various building lifecycle stages. IFC was never created to support the wide range of input data necessary for complex building performance simulations during the design phase [4]. Simultaneously, IFC does not easily integrate data that is being generated in the operational phase of a building [2], such as survey results, interviews, sensor measurements, IoT data, and time-series data [1]. Finally, reasoning over IFC models is difficult as the IFC schema lacks formal semantics [5].

Semantic web technologies have successfully opted as a solution to those issues in the past. Pauwels et al. [3] conducted an extensive literature review on semantic web technologies in the AEC industry. Boje et al. [2] researched how these semantic web technologies could lead to digital twins. These semantic digital twins are proposed as a solution to integrate data related to building energy performance [6], indoor environmental quality [7], and building management systems [8].

Some projects have already developed user interfaces so that end-users can interact with the integrated data. Rasmussen et al. [9] integrated sensor data and BIM data using semantic web technologies and visualized their results in a 2D viewer. Valra et al. [10] created a 3D viewer for managing BIM that applies semantic web technologies to link IFC files to other relevant files. Work of O’Donnell et al. [11] shows how semantic web technologies could be used to create interactive dashboards for building energy management. The examples mentioned here store sensor data in the RDF format using SSN [12], SAREF [13], or similar ontologies.

However, these interfaces are rather one-dimensional, while interconnecting cross-domain knowledge promises to solve both ecological, societal, and economical challenges. To enable interdisciplinary decision-making at all scales and lifecycle stages of architectural design, a flexible framework is necessary that can integrate heterogeneous and federated building information, spread across multiple sources, into a single viewpoint. Boje et al. [2] state that such a dynamic, web-based digital twin of the built environment would improve lifecycle costs, reduce carbon emissions, and deliver better construction services, but that the current interconnection between the various stakeholders, their use cases, and their data is weak. This work aims to fill this gap by presenting a web application – LBDviz – that dynamically integrates data from different knowledge domains scattered across the internet.

2 Method

LBDviz combines a graphical user interface with a method to query federated knowledge graphs (Fig. 1). This paper first proposes a method to integrate heterogeneous building information into a web of data. A generic framework to represent building information, sensor data, and occupant feedback using the RDF (Resource Description Framework) model is presented in Sect. 3. The result of this framework is a set of RDF graphs that contain information from different stakeholders in the AEC industry. Comunica is then used to query the decentralized RDF data.

The graphical user interface is built upon the knowledge graphs and uses the IFC.js library as a basis. The library enables interaction with IFC models on the web. As shown in Fig. 1, a link between the web geometry and the linked data is created by consistently converting the GUIDs in the IFC file to literals in the RDF data so that a direct link between the web geometry and the linked data is established.

The graphical user interface serves as a gateway to interact with the data in the decentralized knowledge graphs. Multiple methods of querying data are provided, such as buttons, custom SPARQL queries, and interaction with the web geometry. Next to this, LBDviz enables querying time-series data from time-series databases.

To test this new tool, multiple case studies are presented in Sect. 5. These case studies aim to show how interconnected building information can enhance decision-making in the design and operational phase of a building.

Fig. 1.
figure 1

System architecture of LBDviz

Full size image

3 Integrating Heterogeneous Building Information

To solve the issues related to IFC, as mentioned in Sect. 1, semantic web technologies are proposed to represent building information. Earlier research successfully integrated heterogeneous building information using these technologies [7, 14]. Data is stored using the Resource Description Framework (RDF) format, resulting in labeled and directed graphs composed of triples. These triples are constructed using a subject, predicate, and object (i.e.:Building:hasRoom:LivingRoom). All resources have unique identifiers (URIs), enabling the integration of heterogeneous data on the web. RDF data can be enriched by linking the resources to ontologies: formal and explicit specifications of shared conceptualizations. In the AEC industry, ontologies typically contain snippets of domain knowledge that can be reused by other stakeholders.

3.1 Converting Building Information Models to RDF

While early-age ontologies in the AEC industry were rather monolithic ontologies based on the IFC schema (such as ifcOWL [15, 16]), recent ontologies focus on creating smaller, modular domain ontologies. The Building Topology Ontology (BOT) is developed as a core ontology in the AEC industry [17] and describes core topological concepts of a building, such as spaces, levels, and elements. Various researchers designed extensions to BOT that can be used to describe domain knowledge. Building properties could, for example, be described using BOP [18], PROPSFootnote 1, and OPM [19], while taxonomies of building elements are described using the PRODUCTFootnote 2, BEOFootnote 3, and MEPFootnote 4 ontologies. The result of the existing work is a modular set of ontologies that can be used in a plug-and-play-like approach to describe building information in the RDF format based on the needs and knowledge of the user.

Various conversion tools have been created to enhance the conversion of BIM files to the RDF format. Both Pauwels & Terkaj [20] and Hoang and Törmä [21] created a tool to convert IFC files to RDF following the ifcOWL ontology. Later initiatives introduced converters that implement the domain ontologies, such as the IFC-to-LBD converter by Oraskari et al. [5] and the Revit-bot-exporterFootnote 5 that creates RDF exports directly from Autodesk Revit. Both converters implement the BOT ontology as a core ontology, complemented by other domain ontologies. Converting BIM files to RDF following other ontologies is still a manual process. As the existing ontologies do not fully cover the domain knowledge necessary for this project, a manual conversion procedure is executed. The result of this procedure is visible in Fig. 2.

3.2 Integrating Sensor Data

Similar to the BIM domain, researchers investigated how semantic web technologies could be applied to describe sensors and the data they produce. This resulted in various ontologies related to sensors, such as SSN/SOSA [12, 22] and SAREF [13]. Although some researchers argue that sensor data could be converted to RDF, best practices suggest that sensor data could be best stored in time-series databases [23, 24].

To link the different databases to the linked building data, we propose storing some metadata of the sensor in the RDF file. The Building Performance Ontology [18] was developed to represent this metadata in an RDF format. Figure 2 shows how this metadata is linked to other building information. The query language SPARQL could now be used to query relevant information about the sensor, and the results of this query can be used to access the right data point in the right database. This method allows sensor data stored at different locations to be integrated with the linked building data [14].

3.3 Integrating Occupant Feedback

In earlier work, we developed a smartwatch application – Mintal – that can be used to obtain occupant feedback on building performance during the operational phase. The application, running on Fitbit Sense and Fitbit Versa 3, automatically converts this feedback to small RDF snippets. As the URIs of the given feedback overlap with the URIs of rooms in the RDF file of the building, this feedback will automatically be integrated with the knowledge graph. This enables querying both the measured building performance (such as sensor data) and the perceived performance (e.g., the feedback) using a single SPARQL query. The Occupant Feedback Ontology (OFO) [25] was developed to structure this feedback. The ontology is firmly attached to the Building Performance Ontology, enhancing query functionalities. Figure 2 shows how the occupant feedback is integrated with the other data.

Fig. 2.
figure 2

Integrating BIM, sensor data, and occupant feedback using the RDF model

Full size image

4 LBDviz: Visualising Linked Building Data on the Web

4.1 Overview

LBDviz is a single-page web app that consists of a full-page IFC viewer with various menu windows on top of the viewer (Fig. 3). This application aims to enable users to interact with interconnected building information from multiple sources in a single viewpoint, with a low software skill threshold for new users.

4.2 The 3D Model

LBDviz uses the IFC file extension to import geometries. IFC is the de facto standard in the AEC industry and can be exported by most proprietary software tools. It, therefore, enables importing geometries created by different stakeholders and different tools.

To build web applications around the IFC format, LBDviz uses IFC.jsFootnote 6. IFC.js is a JavaScript library that enables reading and writing IFC files in the browser. IFC.js uses Three.jsFootnote 7 to represent the geometry in the web viewer. Next to that, the library parses the IFC file so that one can directly apply JavaScript functions to the file.

These JavaScript functions were used to add typical BIM software functionalities to the viewer. After uploading an IFC file, a user can zoom, pan, or rotate the model using the mouse wheel and the right and left mouse buttons. One can crop the 3D model in all directions using cropping panes, allowing the user to navigate through the IFC model. Interaction with elements of the 3D geometry is enabled by simply clicking on them. After selecting the element, various options for advanced interaction are available in the menu bar.

Fig. 3.
figure 3

An overview of LBDviz

Full size image

4.3 Querying Federated Building Information

End-users would typically need information from multiple files. A maintenance engineer that needs to check the heating installation would, for example, need information from both the architect’s and the HVAC engineer’s models. In the semantic web context, interconnected building information would typically be maintained decentrally on the web or in various databases. To query this federated information, ComunicaFootnote 8 is applied. Comunica is a meta-query engine that can be used to execute SPARQL queries over multiple SPARQL endpoints or files.

Fig. 4.
figure 4

Querying federated building information in LBDviz

Full size image

The LBD viewer has multiple ways to execute SPARQL queries. First, the user can travel to the SPARQL tab, which has a text box in which the user can type custom SPARQL queries. A second tab – the Graph Locator tab – can be used to select one or more data sources. After executing the query, the results will be printed in a table in the Results tab. The table will dynamically create headers based on the variables that are passed on in the query (Fig. 4).

As many stakeholders in the AEC industry are unlikely to be able to create custom SPARQL queries, LBDviz also has a Find tab. This tab contains buttons with predefined SPARQL queries behind them so that every end user can run basic queries without needing expertise in semantic web technologies. These buttons can be tailored for specific roles or stakeholders. Current buttons contain basic queries, such as finding all walls, all doors, all windows, and all sensors.

Finally, end-users can enrich queries by interacting with the 3D model. After clicking on an element in the 3D model, the GUID of the element is extracted. This GUID is then inserted in predefined SPARQL queries to filter results for this specific element. This method can be used to query all the properties of a selected element or all the sensors in a selected room. Using this same GUID, LBDviz has the option to query occupant feedback on elements or rooms. These queries are also executed using a button in the Find tab. After clicking the button, the viewer will print the feedback in the Results tab.

4.4 Querying Time-Series Data

Where most data in the design phase of a building is typically static information (such as drawings), data produced in the operational phase of a building has a more dynamic nature. InfluxDB – a time-series database – can be used to store the data produced by sensors. Querying time-series data in LBDviz is a two-step procedure. First, metadata related to the sensor device is stored in the knowledge graph (Fig. 2). This metadata is queried and imputed in a Flux query, that returns the sensor readings from the time-series database. A module in LBDviz automatically creates these queries based on the interaction with the viewer. If the user selects a specific room, LBDviz will only select the sensors in that room.

5 Case Studies

Multiple case studies were executed to show how LBDviz can stimulate tackling challenges throughout different lifecycle phases of a building. During the design phase, complex building requirements can only be validated after combining information from various stakeholders. The first case study shows how requirements related to human health and well-being could be checked during the design phase so that designers receive real-time feedback on the current state of their design. These requirements are based on the WELL v2 standard [26].

Even if a building scores high on those standards, research suggests that buildings do not necessarily improve occupants’ comfort levels [27, 28]. Therefore, a second case study compares the as-designed information with the as-built information. First, designed thermal performance is compared with measured thermal performance using the PMV/PPD method. This method calculates how many people are likely to complain about the thermal comfort in a space. Afterward, measured performance is compared with perceived performance, by integrating the occupant feedback from the smartwatch app Mintal.

LBDviz promises to help tackle ecological and economical challenges. The real-time energy consumption of buildings and devices is added to the platform so that the building performance can be compared with energy consumption, and thus CO2 emissions and costs.

Finally, a feedback loop is created to show how knowledge from the past can be imputed into new design processes.

5.1 Requirement Checking

WELL v2 is a standard for buildings that assesses the human health and well-being in a building. Various parameters of this assessment relate to the building’s design and should therefore be part of the design requirements. To assess those requirements, certain rules in WELL v2 were manually converted to SPARQL queries. The Atlas building of the TU/e campus was used for this case study. The building was renovated between 2014 and 2019, which is why the design phase contained multiple BIM files, some representing the existing building and some representing the newly built parts. As many subcontractors were responsible for specific parts of this building, requirement checking is a complex task that involves information from many parties.

Figure 5 shows how LBDviz integrates various IFC models into a single viewer: one model contains existing parts of the building, one model contains newly designed walls and floors, one model contains the façade, and one model contains newly designed stairs. The IFC-to-LBD converter [5] was used to convert the IFC models to an RDF format. They were uploaded as separate files to a GitHub repository, after which we inserted their web locations into the Graph locator tab.

Fig. 5.
figure 5

Validating a WELL v2 requirement using SPARQL

Full size image

One could then use the SPARQL queries to test individual WELL v2 rules and assess certain features of human health and well-being. The query in Fig. 5 returns all stairs that comply with feature V03 Circulation Network in WELL v2, requiring that at least one staircase open to regular occupants serves all floors of the project and:

  • is designed through at least two of the following strategies: music, artwork, light levels of at least 215 lx when in use, windows that provide access to daylight, natural design elements or gamification,

  • is located physically or visibly before elevators and escalators,

  • and has a point of decision signage near the main entrance or reception desk.

Validating these requirements during the design phase requires a lot of coordination and communication between stakeholders. In the case of Atlas, four files needed to be combined to check this feature. Considering that WELL v2 has 122 feature sets, checking them during the design phase would be too time- and knowledge-intensive.

The Results tab in Fig. 5 successfully returns the stairs that comply with feature V03. It integrates the information from multiple RDF files through the Comunica engine. As LBDviz directly queries from web sources, an update of these web sources will also result in updated query results in LBDviz. This has been validated by deleting the stair from the RDF file of one of the subcontractors and deleting the reception desk from the RDF file of the architect. In both cases, the query in LBDviz did not return the stair anymore, as it did not comply with V03.

5.2 Designing High-Performing Buildings

Researchers found a gap between the as-designed and as-built performance of LEED [28] and BREEAM [27] certified buildings. The lack of tools to monitor, control and optimize buildings only increases this gap. LBDviz aims to support facility managers by giving them advanced insights into their building’s performance. The second case study, therefore, adds sensor data and occupant feedback to the information in Sect. 5.1.

WELL v2 describes various features in the thermal comfort category. Some of those features are decided upon during the design phase, such as whether the building has enough radiant heating and cooling capacity. However, many of those features relate to complex operational decision-making and are influenced by the users of a building. Feature T01 Thermal Performance, for example, describes requirements related to the indoor temperature, while feature T02 Verified Thermal Comfort describes rules for post-occupancy surveys and occupant satisfaction.

These performance indicators can be simulated during the design phase of a building [29], although the lack of information on technical systems limits these predictions. To test how the as-built performance compares to the as-designed performance, a method was developed to calculate the Predicted Mean Vote – an indicator for thermal performance – in real-time using the interconnected data (such as presented in Fig. 2) [14]. As we can store both the predicted PMV and the measured PMV during the operational phase in the knowledge graph, both can be queried, next to other parameters. Figure 6 shows a comparison of the simulated and measured thermal performance of the building. It shows both the final assessment score (based on [30]) as well as individual components of the assessment. Designers can use this as a direct feedback loop to their designs.

Fig. 6.
figure 6

Comparing the as-designed and as-built building performance

Full size image

Next to this, LBDviz can help in finding the gap between measured and perceived building performance. Figure 7 shows how LBDviz integrates the web geometry with temperature readings and occupant feedback. The sensor data is acquired through the method described in Sect. 4.4. The occupant feedback is provided using a smartwatch and translated to RDF as described in Sect. 3.3. Simultaneously querying both the sensor data and the feedback results in a mixed chart that designers and facility managers can use to better understand positive and negative occupant feedback in relation to measured data. Next to checking features in the WELL v2 certificate, the interconnected data can be used to 1. Find anomalies in the sensor data streams, 2. Group rooms with similar complaints, 3. Find the rooms with the most complaints, and 4. Find HVAC systems that relate to properties that people complain about.

Fig. 7.
figure 7

Integrating sensor data, occupant feedback, and the 3D web geometry

Full size image

5.3 Ecological and Economical Challenges

As the current energy crisis in The Netherlands is causing an extra financial burden on real estate finances, gaining insights into the costs of maintaining high building performance is crucial. Simultaneously, keeping energy consumption low will have positive effects on greenhouse gas emissions.

LBDviz integrates real-time energy consumption of buildings and individual devices with the building information. It allows facility managers to monitor energy consumption and includes ecological and economical parameters in their decision-making processes. Figure 8 shows how feedback on illuminance and energy consumption of the building and a lighting source (using a smart energy socket) can be integrated in LBDviz. Facility managers can now see what the energy consumption (and thus costs) of certain interventions are. As the data is integrated into the knowledge graph, advanced algorithms would also be able to find an optimal pay-off between (ecological and economical) costs and indoor comfort.

Fig. 8.
figure 8

Integrating a building’s energy consumption with perceived building performance

Full size image

5.4 Design Feedback

Interconnecting building information doesn’t stop at one building. Semantic web technologies enable the integration of data from multiple buildings within (or without) a designer’s portfolio. As the performance of a building is crucial to determine the optimal layout of buildings, designers would benefit from historical feedback about this performance [29]. As the Comunica engine in LBDviz is built to integrate data from various buildings, a designer can query performance indicators and other building parameters from various buildings in the portfolio (Fig. 9) and compare these to their current design. Figure 9 shows a comparison of thermal comfort scores from multiple buildings and some possible influencers. This can be applied for many use cases, including the ones mentioned in Sect. 5, and leads to a more circular design process.

Fig. 9.
figure 9

Design feedback: comparing thermal comfort with previously designed buildings

Full size image

6 Conclusion

Interconnecting building information on multiple scales and lifecycle stages enables designers to improve their decision-making and tackle ecological, societal, and economical challenges. However, this information is often generated in different software tools, saved in different file formats, using a stakeholder’s own domain language, and is stored in decentral storage locations. Since communication between all the stakeholders in a construction project is a complex and time-intensive task, decision-making is often based on incomplete information, or worse: gut feeling.

This paper presents LBDviz, a tool that integrates heterogeneous and federated building information spread across multiple sources, into a single viewpoint. LBDviz combines a web viewer (based on IFC.js), a meta-query engine to query linked data (using Comunica), and a time-series query engine (using Flux).

The software tool was tested in multiple case studies. The first study shows how interconnecting data can help requirement checking during the design phase. First, relatively static requirements such as staircases were checked, after which more dynamic building performance indicator checks were performed. Interconnecting data across multiple lifecycle stages enabled comparing the as-designed with the as-built building performance. The perceived performance was also integrated, showing designers and facility managers whether occupants actually like their buildings. Finally, the energy consumption of buildings and devices was integrated into LBDviz, enabling decision-makers to take ecological and economic considerations into account when maintaining the performance of their buildings. Interconnecting data goes further than just one building, and we showed that our methodology can lead to a more circular design process, where designers learn from their previous buildings.

LBDviz is an ongoing project. Combining web geometry with linked data is promising to solve data interoperability challenges in all phases of a construction project. Future research should therefore focus on further developing the application’s user interface based on the requirements of real practitioners. By co-creating the application with industry experts, LBDviz might reduce the barriers to using linked data in practice. Next to that, future research should explore the data interoperability challenges of other stakeholders in the AEC industry. Breaking the barriers to interconnect with the data generated by other stakeholders will hopefully improve the decision-making processes throughout the entire lifecycle of our buildings.