In this chapter we briefly describe information methods and technologies supporting geotechnical systems analyses, i.e. using virtual reality methods for data and model integration (Sect. 5.1) and improving computational efficiency by using high-performance-computing techniques (Sect. 5.2).

The generalised approach of the developed visualisation methodology makes it possible to integrate a variety of data formats into 3D scenes and to create studies for a wide range of application fieldsFootnote 1 For example, a prototype for an experiment information system was developed for the rock laboratory in Mont Terri operated by the Federal Office of Topography surveys Switzerland (swisstopo) (Bossart et al. 2017; Jaeggi et al. 2017). This prototype combines a representation of the complex geometry of the Swiss Jura mountains with tunnel system and boreholes of the rock laboratory with a series of results of coupled numerical simulations for the planning and validation of long-time experiments, some of which are designed to run over several decades.

Fig. 5.1
figure 1

Overview of the URL Mont Terri including experimental places

Full size image

Due to the resolution, runtime and complexity of the simulations, the previously described methods for data reduction are essential for the visualisation of the results (Fig. 5.3). Figure 5.1 depicts a map of the URL Mont Terri including local labels for the experiments which have been integrated into the Mont Terri visualization study. The development was carried out in close cooperation with swisstopo as well as the expert scientists who are in charge of the experiments.

The visualisation of geotechnical processes requires three-dimensional representation options for coupled field problems as well as validation in the context of corresponding measurement results, therefore, in the following we discuss aspects of data and model integration into a VR framework.

5.1 Visual Data and Model Integration

A prototype of a Digital Twin for URL Mont Terri has been designed and implemented (Graebling et al. 2022). In Sect. 2.6 the importance for data visualization and placing models into the real geological context has been already elaborated. In order to set-up the Digital Twin a particular Task VR (virtual reality) has been established in the portfolio of the Mont Terri project which was supported by the consortium. The VR task is dedicated to both data and model integration in order to combine the two sources of information for process characterization in the subsurface. The VR task has been evolved as a starting point of a new kind of data management for the future–a visually mediated data base including predictive capacities by integrated process models.

From the technical side, several software tools are important for the development of the VR study. The DataExplorer as part of OpenGeoSys-Workflows is used for data integration (Rink et al. 2022). The Unity framework is being adopted for interactive visualization purposes and data base access interfaces have been implemented.

5.1.1 Data Integration

An important prerequisite for data integration was linking two data bases, (i) the borehole information system BIS (developed a hosted by Simultec) and (ii) the sensor data base Geoscope (developed and hosted by Sixense Soldata). The support from the two companies in helping for the data base links is greatly acknowledged.

Fig. 5.2
figure 2

Data integration for URL Mont Terri: Prototype CD-A experiment

Full size image

An illustration of data integration for URL Mont Terri is depicted in the figure collection. Figure 5.2a shows the entire tunnel system of URL Mont Terri including the positions of all active experiments (green spheres). The acronyms stand for the respective experiments. The indicated positions of the experiments can be clicked in the Unity framework and further information for connected boreholes and experiments become available. By clicking a particular borehole related geometrical information is displayed in a borehole menu (Fig 5.2b). Measurements such as displacements due to rock convergence can be mapped on the related boreholes of an experiment, therefore, structural and process data are being combined (Fig. 5.2c). Structural information such as fracture planes from surface scans can be added into the scene (Fig. 5.2d). Other documentation such as related reports, publication links or photos from cores can be displayed in the context of the selected experiment or borehole (Fig. 5.2e). Moreover, Fig. 5.2f shows sensor data for the temporal evolution of displacements measured from different sensor positions. Therefore, the experimental information systems allows to show and combine structural and process data distributed in time and over space.

5.1.2 Model Integration

Model integration of three examples of Mont Terri experiments are presented:

  • CD-A: Influence of Humidity on the Cyclic and Long-Term Deformation Behavior of the Opalinus Clay: The CD-A experiment focuses on the evaluation of the highly coupled hydraulic-mechanical (HM) processes, aiming on the identification of the significant mechanical effects in the vicinity of an excavation. To gain an increased process understanding of the excavation induced mechanical processes, a comparison of coupled hydraulic-mechanical effects in the CD-A twin niches–one containing coupled hydraulic-mechanical effects due to desaturation and the other one characterized by avoiding desaturation effects as much as possible, is carried out (Ziefle et al. 2022).

  • FE: The “Full-Scale Emplacement” experiment mimics, as realistically as possible, the construction, waste emplacement, backfilling and early post-closure evolution of high-level waste disposal in Opalinus clay by a multiple heater test. It aims at the investigation of repository-induced thermo-hydro-mechanical (THM) coupled effects on the host rock at this scale and the validation of existing coupled THM models (Müller et al. 2017; Papafotiou et al. 2019)

  • FS-B: The “Fault Slip” experiment studies the aseismic-to-seismic activation of a fault zone in a clay/shale formation of Opalinus clay and particularly the conditions for slip activation and stability of faults. Furthermore, implications of fault permeability and the coupled effects between fault slip, pore pressure are investigated (Nussbaum et al. 2017; Guglielmi et al. 2017, 2020; Shiu et al. 2021; Cappa et al. 2022).

Simulation results for the three experiments using OpenGeoSys are integrated into the VR study showing the process evolution in the real context on the URL Mont Terri.Footnote 2 In case of the CD-A experiment, the numerical simulation has been used for the experimental design (calculation of optimal distance between the two niches) (Fig. 5.3).

Fig. 5.3
figure 3

Model integration for selected experiments for URL Mont Terri

Full size image

5.2 Computational Efficiency

In order to achieve sufficient computational efficiency for the simulation of strongly coupled and often non-linear multi-field problems such as for thermo-hydro- mechanical (THM) processes, massive parallelization technologies have to be applied.

OpenGeoSys (OGS) is an open source software developed at the UFZ for the simulation of coupled thermal-hydrological-mechanical-chemical (THMC) multi-field processes in porous media. Such computations place high demands on memory and computing power. The aim is to model real-world applications by increasing spatial resolutions and to make models more realistic, for example by integrating non-linear material laws into the modelling. With a high spatial resolution, important geological structures that are relevant for the physical processes in the domain can be incorporated into the simulation model. This allows, for example, detailed simulations in large-scale areas as they are necessary for predictions about the mechanical integrity of various host rock types. The implementation required an efficient use of the available HPC resources. Within the framework of the project, scalability and parallel efficiency of individual OGS components (e.g. I/O, various local assemblers, monolithic approach vs. staggered scheme, linear equation system solvers) have been analysed. For the HPC optimisation, tracing/profiling tools were used to identify time-critical code sections and to find improvement potentials. Possible re-design steps for OGS were then derived from the analysis results and implemented. The optimised OGS code was successfully demonstrated in various application areas, such as hydrology and environmental geotechnics.

One of these time-critical code section was identified as the simulation output file operations. Before we used parallel file I/O based on the Visualization Toolkit (VTK) but its file formats were not designed with massive parallel clusters and file systems in mind and were a serious bottleneck in some application scenarios. We therefore implemented simulation result output based on HDF5 which decreased the time for writing simulation output files by more than an order of magnitude. Post-processing can still be done with common visualization tools like ParaView.

The variational phase–field model in this study has been implemented in parallel in OpenGeoSys. Unlike numerical methods that treat fracture explicitly with element edge or node splitting, the variational phase–field model treats fractures through a phase–field variable, which is a primary variable to be solved. Therefore, its parallel implementation does not require any special consideration. If one has an implementation of linear elasticity, all that is required is to add an extra equation system for phase–field evolution and iterate the linear elastic system and the phase–field evolution system. All the variational phase–field simulations in this study were run in parallel and the linear scaling has been confirmed up to 100’s of CPUs.

Fig. 5.4
figure 4

THM simulations for a complete deep geological repository, shown is the temperature distribution 100 years after emplacement (Wang et al. 2021) (left), THM simulations for the full-emplacement (FE) experiment in the Mont Terri underground laboratory, shown are temperature, saturation and vertical stress distributions (Raith et al., 2020) (right)

Full size image

The application examples depicted in Fig. 5.4 present complex thermo-hydro-mechanical (THM) simulations for geotechnical problems such as the design of deep geological repositories for the storage of energy waste (e.g. heat-generating radioactive substances) in the geological subsurface. The HPC methodology is particularly demanding here because, on the one hand, it deals with non-linear coupled multi-field problems that place high demands on the parallel solution methods. Secondly, they involve complex geometries (mapping of geological and geotechnical structures), which place high demands on the domain discretisations and corresponding domain decompositions.

5.3 Software Engineering and Benchmark Workflows

5.3.1 Containers

Linux containers are a technology for encapsulating the runtime environment of software and executing it on any Linux system. Common technologies here are Docker and Singularity. Singularity is ideal for HPC environments because it enables seamless integration into parallelised environments (e.g. with MPI) and also uses a better execution concept than Docker (Singularity containers can be executed on the system with limited user rights whereas Docker containers require administrator rights). Singularity is thus widely available on HPC systems, for example on the systems of the TU Dresden (“Taurus”), the Forschungszentrum Jülich (“Juwels”) and the Helmholtz Centre for Environmental Research (“Eve”).

In order to provide various software configurations of OpenGeoSys (e.g. different OGS versions, integration of third-party libraries to extend the range of functions) through Linux containers, a container creation framework “OpenGeoSys Container Maker” was implemented, which provides a user-friendly parameterisation of the container to be created. The framework is available to users via a web interface on the OpenGeoSys GitLab server (used for software version management, project management and software quality assurance). The container is created according to user specifications and made available as a download (Fig. 5.5).

Fig. 5.5
figure 5

Schematic workflow for container generation

Full size image

The containers include the OpenGeoSys software in binary form as well as other tools helpful for the preparation and execution of simulations, such as computational grid generators or Python scripting interfaces. Singularity containers are regular files and can be transferred onto the HPC system easily after creation via the web interface mentioned above. Parallelised simulations can now be carried out with the help of the container. We used these container-based OGS workflows successfully during the project.

5.3.2 Benchmarks and Jupyter Notebooks

A new development within the project is the provision of container applications for Jupyter Notebooks.Footnote 3 In addition to the OGS core and external modules / libraries (e.g. MFront, PHREEQC, PETSc), these containers also contain the Jupyter Notebook server application and a number of Python packages which can be added as needed (Fig. 5.6). After starting the container, the Jupyter Notebook can be accessed as a browser application and OGS can be executed using notebooks. Jupyter Notebooks also form a new basis for OGS benchmark presentation and integration. New test cases are formulated and explained in Python-based Jupyter notebooks which can intermix script logic with explanatory text and images. Moreover, the large variety of existing Python tools can be used for pre- and postprocessing of OGS simulation results. Figure 5.8 shows the OGS benchmark gallery pageFootnote 4 which is organized according to the THMC process coupling hierarchy (Fig. 5.7).

Fig. 5.6
figure 6

Illustration of the OGS-Container concept for Jupyter applications (sources: www.docker.com and www.opengeosys.org)

Full size image
Fig. 5.7
figure 7

THMC processes

Full size image

5.3.3 Benchmark Workflows

The OpenGeoSys benchmark gallery (Fig. 5.8) is organized according to the THM/RTP process hierarchy, thermo-hydro-mechanical and reactive transport processes. A specific process class is represented by a tile showing a simulation result of the related process class. After clicking a tile, the available benchmarks of a process class comes visible and can be selected. Typically, an OGS benchmark starts with a short description of the problem and showing most important results for the benchmark test. All benchmarks are linked with the OGS project file (prj-file), therefore, the benchmark settings are directly available through the gallery. All benchmarks are part of the OGS quality assurance workflow which is continuously running all tests (benchmarks and so-called unit-tests for basic functionalities) after any code changes, automatically. For new benchmarks Jupyter notebooks are available for user convenience and user-specific pre- and postprocessing operations. New process classes and/or those with new benchmarks are highlighted as featured processes on top of the benchmark gallery.

Fig. 5.8
figure 8

OGS Benchmark Gallery organized by process classes. Benchmarks of a specific process class are behind the tiles

Full size image

Big Data applications cover a wide range of requirements for the analysis of complex data. This usually includes not only the analysis itself, but also the necessary pre-processing steps for data integration and preparation as well as methods for user interaction and evaluation. Therefore, it is necessary to create and execute complex workflows even for seemingly simple analyses. Modern HPC architectures are an ideal basis for providing customised and therefore high-performance working environments for different application requirements.

Large amounts of data not only reach the limits of computability, but above all of comprehensibility. Simulation processes are usually divided into three parts: in pre-processing, the simulation model is defined and input data for the simulation is created. Typically, the simulation as the second part generates large amounts of result data, which must then be processed and analysed in post-processing. Post-processing is usually based on a visualisation of the result data and takes place either on front-end computers of the simulation cluster or on the user’s desktop computer. Only in the latter case a transfer of the simulation data is necessary. The final analysed and processed data as a result of post-processing are far less extensive than the simulation results. Performing post-processing directly during the simulation (in-situ visualisation) makes it possible to bypass the communication/data transfer bottleneck and transfer only the already analysed data to the user’s desktop computer.

The development, numerical realisation, validation and use of integrative modelling tools for the simulation of problem-specific, coupled processes as well as the efficient data presentation in the Big Data context make essential methodological contributions to the systematic analysis and prediction of processes in various areas of environmental science, especially with temporally and spatially large model dimensions. In the overarching conceptual and methodological approach to model and software development, the project results achieved by the grant recipient UFZ play a leading scientific role in the efficient and sustainable planning and management of the environmental science systems under investigation.

Work on the further development of the OpenGeoSys software platform was necessary in order to be able to reliably and efficiently carry out simulations of complex coupled processes of environmental science applications (hydrology, geotechnics) on large scales using HPC systems. This provided evidence that OpenGeoSys is suitable for modelling environmental science processes at real sites. In this context, the research work on parallelising the software should be mentioned as necessary and appropriate. With regard to scientific 3D visualisation, the work carried out on the development of workflows, numerical algorithms and software modules for the synoptic in-situ representation of heterogeneous model data from conceptually different sources has laid the foundations for the integrated consideration of structural information, results from measurement campaigns and results of numerical simulation, which were not available at the beginning of the project work.Footnote 5