Data Management

Abstract

Data management includes the development and use of architectures, guidelines, practices and procedures for accurate managing of data during the entire data lifecycle of an institutional unit or a research project. Data are defined as different information units such as numbers, alphabetic characters, and symbols that are particularly formatted and can be processed by computer. The data in the project is provided by various actors which can be GeomInt partners, their legal representatives, employees, and external partners.

Data management includes the development and use of architectures, guidelines, practices and procedures for accurate managing of data during the entire data lifecycle of an institutional unit or a research project defined as different information units such as numbers, alphabetic characters, and symbols that are particularly formatted and can be processed by computer. The data in the project is provided by various actors which can be GeomInt partners, their legal representatives, employees, and external partners. GeomInt Data is provided at GeomInt data management portal (DMP).

In GeomInt project the partners work in very close cooperation. Project-owned and connected infrastructures are synergetic used (as illustrated in Fig. 5.1). In addition to the rock mechanics laboratories of the partners CAU, IfG and TUBAF with partly unique equipment data from ongoing experiments in the underground laboratories are accessed. An essential element of the GeomInt project is the simulation and software development structures illustrated in Fig. 5.1. With regard to the use of the simulation platform OpenGeoSys, the development of which is coordinated by the UFZ and in whose further development the GeomInt partners BGR, CAU, IfG and TUBAF are involved, the simulation and development infrastructures located at the UFZ, including version management, is available to these partners.

Fig. 5.1

Network diagram to illustrate synergies and dependencies in the GeomInt network in connection with the infrastructure elements and numerical methods in the project

The collaborative work requires data management structures and guidelines. Therefore, the first step was to set up a document that includes a user agreement and a data management plan which is the basis for data management in the project.

5.1 User Agreement and Data Management Plan

The GeomInt project partners agreed to set up a user agreement which includes specifications for data structures including metadata, data formats, access authorization for data, the possible publication of data, as well as the handling of the data after the end of the project and outside the project. A first version of this user agreement was created six months after the start of the project.

The user agreement includes guidelines and definitions for the following aspects

• Which data will be generated in the project and has to be managed?

• How will data be provided and exchanged?

• What are the rights of use for the partners and for third parties?

• How to cite data?

• How to supervise the compliance of the user agreement?

As part of the user agreement, a data management plan, which is a formal document that describes how project data is managed during the research period and after completion of the project, was developed. The goal of a data management plan is to consider the aspects of data management (metadata creation, data preservation and analysis) before the start of the project. Following points are discussed in the GeomInt data management plan:

• 1. Generation and management methods (data infrastructure, external data, data integration, data formats, quality control, user groups, data processing stages, versioning, documentation and meta data, geocoding)

• 2. Data Legal Management

• 3. Data exchange and provision, citation rules

• 4. Short-term storage and data management (storages, data transfer, backup, security)

• 5. Long-term storage (characteristics, metadata and documentation, responsibility)

• 6. Resources (organizational roles and responsibilities for data management).

5.2 GeomInt Data

The project results include specific data from laboratory and in-situ experiments, software components and data sets from numerical simulations (i.e. model and result files). An estimation of the extent of the data generated in GeomInt could not be made before the project. Therefore, the data management concept had to be flexible. This uncertainty was mainly due to the fact that the evaluation of test and calculation results may lead to a change in test and calculation planning and may even lead to additional experiments or simulation calculations.

The availability of experimental and numerical data generated in the project, including existing metadata, is realized on an internal area of the Geomint homepage. The UFZ is responsible for the project data and has many years of experience in data management regarding the cooperative development of open source software (OpenGeoSys) as well as the acquisition, storage and processing of data from experiments on different scales, exploration and monitoring campaigns, numerical simulations and scientific 3D visualizations.

The UFZ has sufficient capacities and modern data management systems for data storage, which are available as a central data infrastructure for the research network. Specifically, data sets are managed by means of an ORACLE database. Access is via a web portal, where each data record must be provided with metadata before uploading. The metadata standard used is compatible with the INSPIRE Directive 2007/2/EC and also regulates the rights for access, use and transfer of the data. A tape system is also available for the long-term storage of very large amounts of data. For the provision of exploration and monitoring data, geo-services mentioned in the GDI-DE are used as far as possible. Since such services for complex modelling and simulation data do not exist so far, the provision is done via a data research portal, where data can be found by means of stored metadata.

As software components are part of non-commercial, scientific program platforms and are open source products (e.g.. OpenGeoSys), they are hosted by the responsible partner via established source code hosting services (e.g. GitHub) and is publicly available. A possible public access to project data, which goes beyond the status quo as described in technical publications, as well as the handling of the data after the end of the project is regulated in the cooperation agreement or in the cooperation contract between the project partners.

The handling of data obtained from the in-situ experiments in the underground laboratories through synergies with other projects is also regulated separately (access authorisation for these data, storage location, publication, handling of the data after the end of the project). Such an approach is necessary because specific parts of these data can be used for the scientific purposes of GeomInt, but they are generated in other projects with partly other partners.

5.3 GeomInt DMP

In this section, exemplary data sets of every project partner are described. A table of these data sets including description and link are available only for project partners at the website (Fig. 5.2). Some data sets can be found on the UFZ data investigation portal https://www.ufz.de/drp/. These data sets are uploaded to the data management portal at UFZ (DMP@UFZ).

Fig. 5.2

GeomInt DMS portal https://www.ufz.de/geomint/index.php?de=46799

Fig. 5.3

GeomInt DMS portal: Data areas for experimental, simulation and URL related information

The GeomInt data management system (DMS) is organised in three sections (Fig. 5.3):

• Experimental data (Fig. 5.3 left),

• Simulation data (Fig. 5.3 right),

• and data connected to URLs (Fig. 5.3 bottom right).

Table 5.1 summarizes the MEX related data concerning experiments and simulations. A selection will be described in the following sections.

The following codes (and related input files) are used (see Chap. 7 for detailed code introductions).

Software Codes

• LEM: In-house developed MATLAB In-house code available in executable P-file format.

• DEM: Commercial code by Itasca Ltd.

• SPH: In-house code.

• OGS (OpenGeoSys): https://www.opengeosys.org/releases/.

• HDF: In-house code.

Table 5.1 MEX data management

Input files

• LEM: In-house documentation

• DEM: Itasca Ltd. (user’s manuals)

• SPH/HDF: In-house documentation

• OGS (Benchmarks): https://www.opengeosys.org/docs/benchmarks/

• HDF: In-house documentation.

5.3.1 MEX 0-1a: Bending Fracture Test

Participating institutions of MEX 0.1a (see section 4.1): CAU, UFZ, IfG*¹ (Table 5.2).

Table 5.2 MEX 0-1a: Data overview

CAU (LEM)

The required LEM code and the input variables of the three-Point fracture toughness test on the Rockville Granite samples are uploaded to the IfG (Kiel) NextCloud server. The data is accessible through the following link: https://nextcloud.ifg.uni-kiel.de/index.php/s/pRmBPJ9gK5Se6ci.

The uploaded protected MATLAB file in a *.p format requires a MATLAB version with a built-in Voronoi Tessellation and Delaunay Triangulation functions. The input variables are prepared in a single file for the simulation of the fracture toughness in Rockville Granite. Figure 5.4 shows the relation between the load versus CMOD as described in Sect. 4.1.

Fig. 5.4

The load versus CMOD from the lattice simulation

Fig. 5.5

The load versus CMOD from the VPF simulation results

UFZ (FEM-VPF)

The source code can be found in OpenGeoSys project on github and the input files for the three point bending test performed on the Rockville Granite samples have been uploaded. The files include the unstructured finite element mesh files in vtu format and an OGS input file in xml format. As homogeneous properties such as Young’s modulus are assigned int the computational domain, the spatially constant material properties are specified in the OGS input file rather than in the mesh file. The load and crack mouth opening displacment computed from the simulations are shown in Fig. 5.5 as described in Sect. 4.1.

MEX 0-1a (UFZ) will be also provided as an OGS benchmark case at: https://www.opengeosys.org/docs/benchmarks/phase-field/pf_tpb/.

Meta Data Overview (According to Dublin Core)

See Tables 5.3, 5.4 and 5.5.

Table 5.3 MEX 0-1a (CAU)

Table 5.4 MEX 0-1a (UFZ)

Table 5.5 MEX 0-1a (IfG)

5.3.2 MEX 0-1b: Three-Point Fracture Toughness Test, Opalinus Clay

Participating institutions of MEX 0-1b (see Sect. 4.2): CAU, UFZ (Table 5.6).

Table 5.6 MEX 0-1b: Data overview

CAU Kiel

The experimental results of the three-point fracture toughness test on the Opalinus Clay samples are uploaded to the IfG (Kiel) NextCloud server. The data is accessible through the following link: https://nextcloud.ifg.uni-kiel.de/index.php/s/pJxp2eNEJb6PfiS.

The data set, which includes the time, applied force (N) and the displacement of the sample at the loading point (mm), is provided in a *.txt file. The crack mouth opening displacement (CMOD), which is determined from the image processing technique (Sect. 2.2.2), is given in a *.xlsx file. The data includes the time and the calculated CMOD (mm).

The required LEM code and the input variables of the three-Point fracture toughness test on the Opalinus Clay samples are uploaded to the IfG (Kiel) NextCloud server. The data is accessible through the following link: https://nextcloud.ifg.uni-kiel.de/index.php/s/ZBFN2rSZ99kPY9M.

The uploaded protected MATLAB file in a *.p format requires a MATLAB version with a built-in Voronoi Tessellation and Delaunay Triangulation functions. The input variables are prepared in two different files for a parallel and perpendicular embedded layer orientations. Figure 5.6 shows the comparison between the experimental and numerical data as described in Sect. 4.2.

Fig. 5.6

The load versus crack mouth opening displacement (CMOD) response of the Opalinus Clay

UFZ

The input files for OGS, which were used to simulate the three point bending test performed on the orthogonal and parallel laminations of Opalinus Clay samples, have been uploaded. The files include the unstructured finite element mesh files in vtu format and OGS input files in xml format. Also in the mesh files, the material properties are defined per element. Particularly for the orthogonal and the parallel Lamentations in the samples are represented through a contrast in the fracture toughness in the samples and can be found in the mesh files. The load and crack mouth opening displacment computed from the simulations are shown in 5.7 as described in Sect. 4.2.

Fig. 5.7

The load versus crack mouth opening displacement (CMOD) response simulations for orthogonal and parallel lamination by VPF

MEX 0-1b (UFZ) will be also provided as an OGS benchmark case at: https://www.opengeosys.org/docs/benchmarks/phase-field/pf_tpb_ani/.

Meta Data Overview (According to Dublin Core)

See (Tables 5.7 and 5.8)

Table 5.7 MEX 0-1b (CAU)

Table 5.8 MEX 0-1b (UFZ)

5.3.3 MEX 1-1: Swelling Process, Red Salt Clay

Participating institutions of MEX 1-1 (see Sect. 4.2): CAU, IfG (Table 5.9).

Table 5.9 MEX 1-1: Data overview

CAU Kiel

The required LEM code and the input variables for simulating the swelling process of the salt clay are uploaded to the IfG (Kiel) NextCloud server. The data is accessible through the following link: https://nextcloud.ifg.uni-kiel.de/index.php/s/JmZseQqrsbgWNqC.

The uploaded protected MATLAB file in a *.p format requires a MATLAB version with a built-in Voronoi Tessellation and Delaunay Triangulation functions.Fig. 5.8 shows the change of hydraulic conductivity with applied linear strains as described in Sect. 4.4.

Fig. 5.8

The change of hydraulic conductivity with applied linear strains

Meta Data Overview (According to Dublin Core)

See (Table 5.10).

Table 5.10 MEX 1-1 (CAU)

5.3.4 MEX 1-2: Drying and Wetting Paths of the Opalinus Clay

Participating institutions of MEX 1-2 (see Sect. 4.5): CAU, UFZ (Table 5.11).

Table 5.11 MEX 1-2: Data overview

CAU Kiel

The experimental results of the drying and wetting paths of the sandy Opalinus Clay are uploaded to the IfG (Kiel) NextCloud server. The data is accessible through the following link: https://nextcloud.ifg.uni-kiel.de/index.php/s/q6g25nWyWJKqzNB.

The experimental data (*.xlsx) of drying and wetting paths are uploaded to the server. The data includes the reading number, time (day), stain values in perpendicular and parallel orientations, weight of the sample and measured water content values. Fig. 5.9 shows the change of the strains under the applied suction values.

Fig. 5.9

The suction versus linear strains

The required LEM code and the input variables for simulating the drying and wetting paths of the sandy Opalinus Clay are uploaded to the IfG (Kiel) NextCloud server. The data is accessible through the following link: https://nextcloud.ifg.uni-kiel.de/index.php/s/fDNoPoXpXMqeAsK.

The uploaded protected MATLAB file in a *.p format requires a MATLAB version with a built-in Voronoi Tessellation and Delaunay Triangulation functions. Fig. 5.10 shows the change of hydraulic conductivity with applied linear strains as described in Sect. 4.5.

Fig. 5.10

The change of hydraulic conductivity with applied linear strains, Opalinus Clay

Meta Data Overview (According to Dublin Core)

See (Table 5.12).

Table 5.12 MEX 1-2 (CAU)

5.3.5 MEX 1-4: CD/LP Experiment (BGR)

Participating institutions of MEX 1-4 (see Sect. 4.7): BGR (Table 5.13).

Table 5.13 MEX 1-4: Data overview

Link to the data set at UFZ data investigation portal (Download only for project members): www.ufz.de/record/dmp/archive/7471/.

The data set contains all project files that are relevant to reproduce the simulation results described in Sect. 4.7 and its corresponding subsections. For every simulation run with OGS-6, the data set comprises a project file (*.prj), a geometry file (*.gml) and the computational mesh (*.vtu). For simulations run with OGS-5, the project is split into files for boundary conditions (*.bc), initial conditions (*.ic), fluid properties (*.mfp), material properties (*.mmp), solid properties (*.msp), numerical parameters (*.num), output parameters (*.out), process definitions (*.pcs), tabulated functional dependencies (*.rfd), source terms (*.sc), and output parameters (*.out). In addition, separate files are provided for the geometry (*.gli) and the computational mesh (*.msh).

Meta Data Overview (According to Dublin Core)

See (Table 5.14).

5.3.6 MEX 2-1a: Pressure Driven Percolation in Salt

Participating institutions of MEX 2-1a (see Sect. 4.8): CAU, IfG, UFZ (Table 5.15).

Table 5.14 MEX 1-4 (BGR)

Table 5.15 MEX 2-1a: Data overview

UFZ

Link to the data set at UFZ data investigation portal (Download only for project members): https://www.ufz.de/record/dmp/archive/7706/.

The link contains two input decks for OGS-6 in which pressure driven percolation as described in MEX2 is simulated under different configurations of boundary loading. The first case applies the boundary loading of 12 MPa, 21 MPa, and 8 MPa in x-, y-, and z-direction respectively. It is called “case 1” and the corresponding OGS-6 input file is “me2_insitu_case1.prj”. The second case is loaded with 4 MPa, 15 MPa, and 19 MPa in x-, y-, and z-direction respectively and the input file is named “me2_insitu_case2.prj”. The remaining files are vtu files that describe the computing domain and the boundaries as shown in Fig. 4.58.

MEX 2-1a (UFZ) will be also provided as an OGS benchmark case at: https://www.opengeosys.org/docs/benchmarks/phase-field/pf_perc/.

CAU Kiel

The required LEM code and the input variables of the percolation test on saltstone samples are uploaded to the IfG (Kiel) NextCloud server. The data is accessible through the following link: https://nextcloud.ifg.uni-kiel.de/index.php/s/9JZZcpS4S3JJT9S.

The uploaded protected MATLAB file in a *.p format requires a MATLAB version with a built-in Voronoi Tessellation and Delaunay Triangulation functions. The input variables are prepared in two files for two different stress configurations. Fig. 5.11 shows the frack surfaces under the percolation test as described in Sect. 4.8.

Fig. 5.11

The frack surfaces (red) under the percolation test, Saltstone

Meta Data Overview (According to Dublin Core)

See (Tables 5.16 and 5.17).

Table 5.16 MEX 2-1a (UFZ)

Table 5.17 MEX 2-1a (CAU)

5.3.7 MEX 2-1b: Pressure Driven Percolation, Opalinus Claystone

Participating institutions of MEX 2-1b (see Sect. 4.9): CAU, UFZ (Table 5.18).

Table 5.18 MEX 2-1b: Data overview

CAU (LEM)

The experimental results of the pressure driven percolation test on the cubic Opalinus Clay samples are uploaded to the IfG (Kiel) NextCloud server. The data is accessible through the following link: https://nextcloud.ifg.uni-kiel.de/index.php/s/EMdNkdF4PRKWCqa.

The experimental data (*.ASCII) of two different stress configurations (Sect. 2.4.2) are uploaded to the server. The data includes the time ($\Delta T=1s$), pump volume (mL), given oil pressure (Bar) and actual oil pressure in the system (Bar). Fig. 5.12 illustrates an example of the plotted borehole pressure versus flow volume for the 1st stress configuration discussed in the Sect. 4.9.

Fig. 5.12

The recorded results depicting the evolution of the borehole pressure versus flow volume for the 1st stress configuration

The required LEM code and the input variables of the percolation test on Opalinus Clay samples are uploaded to the IfG (Kiel) NextCloud server. The data is accessible through the following link: https://nextcloud.ifg.uni-kiel.de/index.php/s/tFKKjxnSpgNG25b.

The uploaded protected MATLAB file in a *.p format requires a MATLAB version with a built-in Voronoi Tessellation and Delaunay Triangulation functions. The input variables are prepared in two files for two different stress configurations. Fig. 5.13 illustrates an example of the evolved frack surfaces for the 2nd stress configuration discussed in the Sect. 4.9).

Fig. 5.13

The frack surfaces (red) for the 2nd stress configuration

UFZ (FEM-VPF)

MEX 2-1b (UFZ) will be also provided as an OGS benchmark case.

Meta Data Overview (According to Dublin Core)

See (Tables 5.19 and 5.20).

Table 5.19 MEX 2-1b (CAU)

Table 5.20 MEX 2-1b (UFZ)

5.3.8 MEX 2-2: Closure and Healing of Cracks (IfG)

Participating institutions of MEX 2-2 (see Sect. 4.10): IfG, UFZ (Table 5.21).

Table 5.21 MEX 2-2: Data overview

The measured gas flow is converted into permeabilities, which are stored as time series in an Excel file. For each of the three experiment there are two columns. The first contains the time in hours since the start of the experiment, the second contains the permeability (Fig. 5.14).

Meta Data Overview (According to Dublin Core)

See (Table 5.22).

Table 5.22 MEX 2-2 (IfG)

Fig. 5.14

Graphical representation of data

5.3.9 MEX 2-3: Effect of Compressibility on Pressure Driven Percolation

Participating institutions of MEX 2-3 (see Sect. 4.11): CAU, IfG, UFZ (Table 5.23).

Table 5.23 MEX 2-3: Data overview

CAU Kiel

The required LEM code and the input variables for simulating the effect of compressibility are uploaded to the IfG (Kiel) NextCloud server. The data is accessible through the following link: https://nextcloud.ifg.uni-kiel.de/index.php/s/6Mfg3P4PyKNN6By.

The uploaded protected MATLAB file in a *.p format requires a MATLAB version with a built-in Voronoi Tessellation and Delaunay Triangulation functions. The input variables are prepared in two different files for the simulation of the pressure drop in gas and brine reservoirs. Fig. 5.15 shows the differences between the pressure drop in gas and brine reservoirs.

Fig. 5.15

The comparison between the pressure drop in gas and brine reservoirs

Meta Data Overview (According to Dublin Core)

See (Table 5.24).

Table 5.24 MEX 2-3: Meta Data according to Dublin Core

5.3.10 MEX 2-4: Large Wellbore Test (Springen)

Participating institutions of MEX 2-4 (see Sect. 4.12): IfG (Table 5.25).

Table 5.25 MEX 2-4: Data overview

Meta Data Overview (According to Dublin Core)

See (Table 5.26).

Table 5.26 MEX 2-4 (IfG)

Fig. 5.16

Measured pressure curve data: p(t)

The data set contains a measured pressure curve over time (50 days) during an injection test in a large wellbore (Springen URL). The injection regime was conducted in 5 bar increasing pressure steps followed by shut-in periods for observing the related pressure diffusion. Corresponding pumping rates were between 1–5.5 liters per second. Peak pressures of more than 60 bars were required to create in-situ discontinuities by overcoming the tensile/adhesive rock strength of salt. Additionally, acoustic emission (AE) data were recorded (Fig. 5.16).

5.3.11 MEX 3-1: CNL Direct Shear Test Data (TUBAF)

Participating institutions of MEX 3-1 (see Sect. 4.13): TUBAF (Table 5.27).

Table 5.27 MEX 3-1: Data overview

Link to the data set at UFZ data portal (DMP): www.ufz.de/record/dmp/archive/7925/.

The CNL data set contains four text files. One text file with the rock properties of the used granite (see Table 2.1). Two files with the scan data of the two surfaces. One point cloud can be seen in Fig. 5.17. The last file contains the laboratory data. In Fig. 5.18 the results for the four shear stress levels can be seen.

Fig. 5.17

Point cloud representing the surface of a granite sample from Saxony. The size is 65 mm by 170 mm and the cloud contains approx. 98000 points

Fig. 5.18

CNL test results

Meta Data Overview (According to Dublin Core)

See (Table 5.28).

Table 5.28 MEX 3-1 (TUBAF)

5.3.12 MEX 3-2: CNS Test

 

Participating institutions of MEX 3-2 (see Sect. 4.14): TUBAF (Table 5.29).

Table 5.29 MEX 3-2: Data overview

Link to the data set at UFZ data management portal (DMP): https://www.ufz.de/record/dmp/archive/7924/.

The data set of the CNS test contains a file with the rock properties of the used basalt (see Table 2.1), two files with the scan data of the two surfaces. One point cloud can be seen in Fig. 5.19. The results of the laboratory tests are available as ASCII files and the shear curves and the dilatation are visualized in Fig. 5.20. Additionally three photos of the basalt surface before, after the first and after the fourth shear test are included.

Fig. 5.19

Point cloud representing the surface of a basalt sample from Thuringia. The size is 196 mm by 149 mm and the cloud contains approx. 252000 points

Fig. 5.20

CNS test results

Meta Data Overview (According to Dublin Core)

See (Table 5.30).

Table 5.30 MEX 3-2 (TUBAF)

5.3.13 MEX 3-3: Inverse Analysis of Reiche Zeche Data and Harmonic Testing of a Single Fracture

Participating institutions of MEX 3-3 (see Sect. 4.15): UoS, UFZ (Table 5.31).

Table 5.31 MEX 3-3: Data overview

Link to the data set stored at the Gitlab server located at the Institute of Applied Mechanics, University of Stuttgart:

https://galilei.isr.uni-stuttgart.de/gitlab/schmidt/geomint (Fig. 5.21).

Fig. 5.21

Visualisation of data obtained from the pre-compiled executable used for inverse analysis computations of pumping tests performed at Reiche Zeche at a depth of $40.6\,$m

Table 5.32 MEX 3-3: Reiche Zeche Data

The uploaded data set contains two compiled executables for simulations to fit experimental data for the Reiche Zeche fracture characterization tests and to reproduce the non-linear flow response throughout harmonic testing of a single fracture on the laboratory scale like described in Sect. 4.15. The folder includes executables, input files and the used discretization in terms of meshing files, required to perform the simulations. README.txt files provide further information how to start the simulations.

Meta Data Overview (According to Dublin Core)

See (Table 5.32).