Simulation-based assessment of data center waste heat utilization using aquifer thermal energy storage of a university campus

The global energy consumption of data centers (DCs) has experienced exponential growth over the last decade, that is expected to continue in the near future. Reasonable utilization of DC waste heat, which is dissipated during the computational process, can potentially be an effective solution to mitigate the environmental impact. However, the practical implementation of waste heat utilization in the DC environment is a very challenging task. The possible benefits of waste heat utilization are uncertain and difficult to quantify with the methods that are common in practice. This paper introduces a feasibility study in which dynamic simulation tools were used to predict the energy performance of a university campus resulting from the integration of a proposed DC system with an existing aquifer thermal energy storage (ATES). The presented study utilizes building energy simulation (BES) to evaluate uncertainty of the waste heat potential associated to various thermal management strategies of the proposed DC. Further in the feasibility study, the carbon footprint of the integrated approach is assessed for both the current and future situation based on measured data from the existing university campus and its district ATES system. Keywords


Introduction
Recent energy statistics indicate that the DC industry is responsible for 1.7% to 2.2% of the world's electricity consumption (Masanet and Koomey 2013;Shehabi et al. 2016). End-users of DC services comprise a broad group of individuals, companies, banks, government entities and, of course, academic institutions (Joshi and Kumar 2012, Chap. 1). Nowadays, no university, research laboratory or other educational institution can be operated without the support of IT. Since, all electrical power required by IT is dissipated in the form of heat, this heat must be mechanically removed by a cooling system. In fact, almost half of the entire DC energy use is assigned to the cooling system to ensure required operational conditions for the IT and other housed electronics. This dissipated heat is generally dispersed into the ambient air, and its potential is wasted (Harrison et al. 2012;Ebrahimi et al. 2014).
The premise of this research is that DC waste heat can be reasonably reused by neighborhood buildings as heat consumers, which would ultimately lead to a reduction of total carbon emissions at the district level (Davies et al. 2016). Indeed, a better opportunity for effective utilization of DC waste heat can be expected in the near future considering the current roadmaps and strategies for low-carbon heating and cooling systems in EU promoting district thermal grids as most promising solutions (Connolly et al. 2014;Lund et al. 2017).
Recent studies (Wahlroos et al. 2017(Wahlroos et al. , 2018Goumba et al. 2017) demonstrated theoretical environmental benefits of synergy between modern district thermal systems and DC infrastructure, these studies also identified the main barriers such as (i) DC low-quality heat, (ii) unbalanced energy demand of DC infrastructure (lack of heating demand), (iii) high investment costs to district infrastructure, (iv) differing financial outcome expectation of DC and district heating operators.
Traditionally, IT hardware has been cooled down by air, which is generally conditioned and supplied by computer room air-conditioning (CRAC) units into a dedicated space in front of the IT hardware inlet in the DC space. Air-based cooling is popular especially since it allows easy access and physical manipulation of racks, in which IT hardware is situated. However, air as a cooling medium has physical limitations such as its low heat capacity that leads to the inlet temperature in range of 18 to 27 °C recommended by ASHRAE Thermal Guidelines for Data Processing Environments in order to prevent the IT hardware from overheating or even failure. Due to the low temperature level, the harvesting of the waste heat from air-cooled DC is very challenging (ASHRAE 2012). A typical outlet air temperature level from IT hardware in an air-cooled DC is in the range of 35 to 40 °C. Adding possible inefficiencies of the DC heat removal system, water temperature supplied to a thermal grid could be estimated to be only up to 25 °C (ASHRAE 2010).
Alternatively, the liquid-based cooling integrated in the IT hardware may offer much higher heat removal efficiency and also higher operational temperatures of the cooling system. A water glycol mix, refrigerant or oil is usually used as a coolant for individual electronic parts (Chipset, RAM, etc.) (Zimmermann et al. 2012;Brunschwiler et al. 2009). In fact, the liquid-based cooling enables to harvest the waste heat at levels of 50 to 60°C that are already common for nowadays district heating applications (Lund et al. 2018;Guo and Goumba 2018).
Nevertheless, liquid-based cooling has been mainly used for special high-performance IT hardware in research due to its technical complexity. Although its application can already be seen in a limited number of industry experiments and demonstration sites, the market penetration of this solution is only around 14% according to the current Uptime institute survey (Uptime Institute 2018). For this reason, this feasibility study limits its analysis to air-based DC cooling.
Despite relatively low air temperature levels, the amount of dissipated heat produced by conventional air-based cooling system is still enormous. A typical mid-sized DC, equivalent in size to a small office building can dissipate around 1 MW of heat, which is equivalent to the heating demand of a small district comprising hundreds of households (ASHRAE 2008;Riahi 2015). This waste heat can be utilized in low-temperature thermal grids such as ATES systems (Paksoy et al. 2017). However, adding the DC system with its vast waste heat into a low-temperature thermal grid serving typical residential buildings may lead to uncertain energy and environmental outcomes due to the different requirements of these two actors, where there are often different expectations regarding the operation of the integrated district system. This feasibility study uses BES tools for assessing and predicting the impact and energy performance of a DC when added to an existing university campus district heating and cooling system with ATES. The benefits and drawbacks of the integrated operation between DC and district system are analyzed. BES tools allow uncertainty analyses of the DC waste heat potential concerning yet unknown DC thermal management and further elaborate the integrated operation with the district ATES system under different DC thermal management strategies.
Thus, the objective is to quantify the energy performance of the DC waste heat utilization system allowing a comprehensive understanding of its energy behavior. The energy and environmental implications are evaluated using a numerical model representing a typical DC infrastructure that might be added to the campus district thermal system of the Eindhoven University of Technology (Meulen 2016).

DC waste heat utilization system
The performance of the waste heat utilization system of the planned DC is the focus of the feasibility analysis, as an indicator of the potential benefits arising from the addition of a DC into the existing district-energy system in the university campus. Examples of these pioneering systems, which are able to reasonably exploit low-grade DC waste heat, are documented in the literature (Drenkelfort et al. 2015;Agentschap NL 2012). The district ATES system of the Eindhoven University Technology campus, in Netherlands, is illustrated in Fig. 1.

ATES district system
This ATES district system includes 32 wells divided over three warm and three cold clusters. Groundwater is pumped from or to these wells, which are 20-80 m under the surface. The mentioned clusters are connected to warm and cold rings, each about 2 km long under the university campus. The system uses two temperature levels: a "cold" well with Fig. 1 Illustration of the case-study ATES system (Spruijt 2015) Dvorak et al. / Building Simulation 3 water of 6-9 °C and a "warm" well with water of 14-17 °C. The denotation of "cold" and "warm" is relative to the natural temperature of the subsurface of 10-13 °C (Becvar et al. 2004). The district system provides heating and cooling to the majority of campus buildings. The temperature level of the cold ring is usually sufficient for cooling purposes and the cooling water can be used directly. However, for heating purposes, the temperature of the hot ring is raised by heat pumps to end-use temperatures that is circa 50 °C. Additionally, there are gas-boilers used for achieving peak temperatures above the reach of heat-pumps (i.e. for the use of hot water as well as for older buildings that require higher temperatures for space-heating) The campus district system is equipped with one of the largest ATES in the Netherlands with a designed import/ export rate of 25 MW. As the campus energy report (Meulen and Marks 2012) depicts, the load of the connected campus buildings is in the range of 1.7 MW in winter to 8.8 MW in summer season. Currently the district system operates throughout the year at aproximetly one third of its total thermal load potential (i.e. MW). Thus, it can be assumed that the size of the district ATES system is sufficient to accommodate a mid-sized DC as long as their combined operation satisfies a balanced energy storage over the year.

DC system
The integration of the DC into the campus district would imply that its waste heat would be supplied to campus buildings through the existing district ATES system. The mid-sized data center with air-based cooling specified in ASHRAE Case Studies and Best Practices (ASHRAE 2008) was selected as reference DC infrastructure able to satisfy the computational demand of the Eindhoven University of Technology in the future. In the specifications of this reference DC, its IT equipment is located on a raised-access floor with an area of 23 m × 30 m which houses approximately one hundred racks. The racks within this area dissipate 10.7 kW per rack in average. The total IT power load is equal to 1090 kW, which dissipates in form of heat. The dissipated heat must be removed from the DC space and can, in principle, be reused.
Typically, dissipated heat is removed by computer room air-conditioning (CRAC) units, where the indoor air is cooled down mechanically by chilled water prepared in chillers (using vapor-compression refrigeration cycle). The chiller rejects the waste heat to the ambient air either via cooling towers or dry-coolers located outside of the DC room. However, in the feasibility study presented analyses the performance of integrating the DC cooling system within the district ATES system described above. This would reduce significantly the use of an energy-hungry cooling system, typical of DCs as well as the reutilization of DC waste in campus buildings within the district system.
The cooling system shown in Fig. 2 represents the typical DC cooling configuration. This system can work in two modes depending on the external conditions: (a) chiller mode as a primary cooling source using a cooling circuit composed of a compressor, condenser, expanse valve and an evaporator and (b) water-side economizer mode using the DC's cooling towers.
The series of valves (2a and 2b in Fig. 2) distribute water according to the desired mode. When the ambient conditions do not allow the use of economization, the energy-hungry chiller must be operated. If ambient conditions permit (the outside wet-bulb temperature is below the defined setpoint for economization), the cooling system is switched to economization mode and the DC's cooling tower (CT) is utilized (highlighted red and blue lines).
The cooling system shown in Fig. 3 represents the system integrated into an ATES system. Similarly, this system operates in two modes according to ambient conditions: (a) connection to ATES as primary cooling source and (b) water-side economizer mode using the DC's cooling towers.

Current utilization scenario of the district ATES system
Based on design conditions, the ATES system could absorb and reuse all rejected heat from the DC considering its capacity. However, based on the latest measurements (Meulen 2016), the district ATES system has mainly supplied cooling to the other campus buildings. As such, the amount of imported heat is higher than the exported heat from the ATES system, which leads to the energy imbalance of cold and hot storage clusters. This imbalance, if maintained in time (i.e. years), could cause permanent temperature changes in the soil. This may not only reduce the overall storage potential of the ATES system but lead to severe environmental damage.
In order to achieve yearly energy balance, the district ATES system includes two cooling towers to, if outdoor conditions permit, reject the required surplus heat from the system. This balance between heat import and export can be depicted from Fig. 4, which presents historic data from 2005 to 2016. In this figure, heat exported (heating) and imported (cooling) as well as heat rejected by the cooling towers are shown.
Keeping yearly balance of heat importing/exporting to the wells is the potential barrier to the connection of a new DC to the district system. This is mainly due to the continuous cooling demand required by DCs, that may lead to an increase of the storage imbalance over the year.
In order to manage and operate the district ATES system, the campus real-estate department, through a facility management (FM) team that:  oversees and regulates the amount of imported/exported heat from campus buildings to the ATES;  limits the temperature level of returning water from each campus building. These define an operation policy referred as "ATES policy" in this paper.
Firstly, this ATES policy states an allowable range of water return temperatures that can be exported to the district ATES system from connected buildings. The desired temperature difference between hot and cold cluster is 7 °C, while the maximal temperature that can be delivered to the hot cluster is set at 15 °C and the minimal temperature related with the cold cluster is set at 8 °C. The management of each attached building is obliged to operate its energy systems within these limits in order to assure the smooth operation of the entire Secondly, the ATES policy also prescribes limits to importing and exporting heat from and to each individual building to the central ATES district. These limits are generally agreed between building and district network operators. For the assessment presented in this paper, the following agreement is assumed: cooling of DC from the ATES is only available when economizer-based operation is not feasible. This agreement facilitates significant reduction of consumption from the energy-hungry chiller and allows a reduction of the heat exported into the ATES maximizing the use of the economizer.

Future utilization scenario of the district ATES system
The amount of exported and imported heat may change dramatically in the near future due to the progressive connection of more buildings into this district system. For instance, since 2016, two new residential buildings have been connected to the ATES, which is not reflected in Fig. 4. Continuing with this trend, new additional residential buildings are planned to be connected to the ATES system after their refurbishment in 2020. These additions are included as part of a "future scenario" in the analysis.
The ATES capacity was specified in its design stage accounting for the development and growth of the university campus. In accordance with the environmental policy of the university, it is planned that the ATES system will gradually replace the use of gas for heating. Campus buildings, which were primarily supplied by gas boilers in the past, will satisfy their heating demand by a combination of the ATES system and heat pumps. This transition where with refurbished and newly built residential buildings will significantly increase the heat demanded to the ATES, likely transitioning into a future situation where heating demand is much closer or even surpasses cooling demand in the campus. The cooling towers that currently keep ATES balance (under greater cooling demand) would not help in reaching this balance anymore. The solution to this new situation would be to either not satisfy completely the heating demand or consider adding a cooling dominated building type (e.g. laboratory, DC etc.) to the district ATES system.

Modelling method and assumptions
A detailed physics-based model of the DC system has been developed to investigate waste heat potential under various DC thermal management strategies. The contribution of the predicted waste-heat to the existing district system is then analyzed. Finally, the integrated operation of the proposed DC and the existing campus district ATES system is evaluated. The DC environment is a multi-domain environment, where the operation of IT, cooling and power delivery systems are dependent on each other. For this reason, it must be represented to truly capture the nature of its energy behavior. In addition, the modelling method must lead to the generation of reliable outputs with limited technical specification data.
Building Energy Simulation (BES) tools, developed over the last 50 years, are widely considered as trustworthy physics-based modelling environments, enabling modelling of multiple domains These tools offer sophisticated computational engines for modelling building physics and building services systems. They allow simulation of transient behaviors of building envelopes, heating, ventilation, and air-conditioning (HVAC) systems, as well as advanced systems such as onsite renewable energy systems (RES) (Hensen and Lamberts 2011).
Since a BES approach satisfies the multi-domain modelling aspect, the BES tools have been recently adapted for the purpose of DC energy assessment. This can be demonstrated by the addition of special energy models for data centres in Energyplus (version 8.3, released 2015) (NREL 2015) or the DC library now available for Transient System Simulation Tool (TRNSYS, renewIT library, 2016) (RenewIT 2016). Concerning the application of BES for DC modelling in the past, Phan and Lin (2014) investigated a multi-zone representation of a DC with cold and hot aisle arrangement in EnergyPlus. Also, the modelling of airflow in DC space using Modelica was recently addressed (Leva et al. 2014). Similarly, Salom et al. (2015), introduced simplified dynamic modelling of DC space. In addition, the same team (Oró et al. 2015a) also investigated DC modelling to support the integration of renewables into the DC using TRNSYS (TRNSYS 2012b).
As shown, there are several BES tools that can be used for the defined purpose. The assessment of the DC environment requires a multi-domain approach to address various energy systems and building physics phenomena. The integrated modelling approach for DC environment was described in literature (Zavřel et al. 2015) and later demonstrated in (Zavřel et al. 2017;Pesch et al. 2017). More specifically, the method adopted to simulate the transient energy behavior of the entire DC is the co-simulation of Transient System Simulation program (TRNSYS), including the TESS and "Green Data Center" Libraries, that allows the modelling of various type of energy systems including building physics, HVAC, electrical and control systems (TRNSYS 2009(TRNSYS , 2012RenewIT 2016), CONTAM, which provides airflow prediction to address the two thermal management strategies with and without cold-aisle containment (Walton and Dols 2005).
The model of the DC environment consists of three main parts: a model of the DC room, a model of the DC cooling system and a power delivery system model. All these models follow the specifications of the case study mentioned in the previous section. Individual components of the model were configured based on the available technical specifications.
Firstly, the multi-zonal airflow network method was used to model the DC space, which is divided into several zones. These zones represent underfloor plenum, cold aisle, IT cabinets, hot aisle and upper plenum. The model considers the phenomena of used air recirculation or supply air bypass. The zones are linked together with an airflow network, where the CRAC supplies air to the underfloor zone and receives it (return air) from the upper plenum. This represents the conditioned air circulation. The additional pressure drop caused by the IT hardware fans is modelled in the zone of IT cabinets, with the inlet from the cold aisle and outlet to the hot aisle. The definition of this airflow network enables the assessment of air distribution efficiency. Secondly, two model variants of the two cooling system alternatives (schematically depicted in Figs. 2 and 3) serving the DC space were developed using the available components in the TRNSYS simulation environment. The component technical specification of the two models are described in detail in the study (Dvořák 2017). The building energy model was divided into zones similarly to the airflow network model of the reference DC geometry (ASHRAE 2008). The envelope consists of typical lightweight thermally-insulated sandwich panels with an average U-value of 0.34 W m −2 K −1 . The model also considers the effective thermal mass of external walls and housed racks. However, it is worth to note that the heat transfer through the external envelope is minor in comparison to the massive heat removal rate of the DC cooling system. Therefore, the uncertainty associated to the thermal properties of the envelope has relatively small impact on the overall outcome. Lastly, to model the power delivery, the components from the "Green Data Center" Libraries component (RenewIT 2016) were applied. The performance of power delivery components is usually modelled either by a quadratic dependence on load factor characterizing the part-load efficiency, or by values in manufacturers data-sheets.
Several assumptions were made for the development of the DC model. These are:  In the DC space, only the temperature conditions are controlled. The (de)humidification of the air is neglected in this study, because its influence on the overall energy demand and waste heat potential is small. Indoor humidity is set constant in the model (RH = 50%).  Weather conditions are represented by the relevant weather dataset for a characteristic year defined by a data format of a typical metrological year TMY2 (Marion and Urban 1995)  ATES system temperatures in warm and cold rings are assumed to remain constant according to ATES operational strategy. This assumption is based on the latest energy report of campus facility management Energie Efficiency Plan 2013-2016, where the water temperature in rings fluctuates in the range ±1 °C (Meulen and Marks 2012). Water temperatures are considered as constant boundary condition for the simulation of the DC performance.
Water temperature was assumed 7 °C in the cold ring. To note, the return temperature from the DC model should not exceed the temperature of the hot ring of 15 °C.  The storage size is approximately three times larger than the required load (including the DC addition). As such it can be assumed that the storage capacity is unlimited for the current district configuration as long as the yearly energy balance of imported and exported heat from/to ATES is guaranteed. The dynamic behavior of the ATES system has not been taken into account since falls out of the scope of this study. Further, the modeling techniques and model configuration are described in detail in these studies (Dvořák 2017;Zavrel 2018).

Uncertainty study of the DC thermal management
Since the desired DC operational strategy is unknown at this stage of the project, an uncertainty analysis of the DC thermal management was performed to provide the possible range of cooling power demand and waste heat potential for an integrated operation with the district ATES. Multiple simulations for various combinations of DC thermal management strategies were executed. The range of inputs representing the DC management variation are presented below:

IT workload profiles
 A high-Performance Computation (HPC) workload profile -this type of workload is typically CPUintensive and performs a large amount of floatingpoint operations, which leads to a relatively high and constant IT power profile ranging between 15% to 100% of the IT power capacity. An example of this workload is scientific oriented computation (Carbó et al. 2016).  A Web workload profile -this type of workload usually has periodical periods of peaks and pits based on end-user demand. This workload fluctuation leads to a variable IT power profile with daily and weekly patterns. The fluctuations were set ranging between 5% and 100% of the IT power capacity. An example of this workload could be web-based services such as search engines and social networks (Carbó et al. 2016).

Setpoints of supply air temperature
The supply air temperature and related setpoints of economization can significantly influence the energy consumption of the overall cooling system. Therefore, the variation of these setpoints is considered in the uncertainty analysis of the DC management. As such, four different setpoints were simulated. These supply temperature setpoints were set in the range of ASHRAE thermal guidelines at 13, 17, 21 and 25 °C. The lower temperature represents a conservative approach aiming to avoid hotspots. The higher temperatures correspond to advanced cooling approaches with an emphasis on energy savings. In practice, raising the temperature setpoint is usually related to complementary thermal management strategies such as cold and hot aisle containment (Oró et al. 2015b).

Thermal management: cold aisle with or without containment
This study considers two air distribution management scenarios, (i) where the cold aisle is not physically separated from the rest of the DC room (without containment) and (ii) the cold aisle is physically separated e.g. by plasterboard from the rest of the DC room (with containment). The arrangement and physical separation of the DC room has a significant influence on the efficiency of the air distribution. Efficient air distribution can reduce the phenomena of air recirculation within the DC room, causing hot spots and bypasses of the supply air, thereby lowering the energy efficiency of DC cooling (Capozzoli and Primiceri 2015).
First, the influence of the supply air temperature setpoint settings is analyzed. Fig. 5 shows the range of annual cooling electricity demand. The results are sorted according to the setpoint settings. The results are presented for all scenarios; the conventional cooling system, the district ATES with policy, and the district ATES without policy. The results for conventional cooling system show that by increasing the setpoint, the annual cooling electricity demand is reduced from 2820 MWh to 2260 MWh on average. It also shows a decreasing trend of the range of uncertainty for higher temperature setpoints. This effect is caused by the previously mentioned empirical relation between supply air setpoint and economization setpoint. Thus, for lower temperature setpoints, the energy-hungry chiller must cover higher cooling load and also be operated more often. For example, to satisfy the lowest temperature setpoint of 13 °C, the chiller was operated 80% of the time. For the higher temperature setpoints, the economizer with relatively constant power demand could be operated more often. For the case with the highest temperature setpoint (25 °C), the chiller had to be operated only 11% of the time.
The results for the cooling system with the ATES using the policy show an opposite trend of the annual cooling electricity demand for the same temperature setpoints. The average cooling demand was increased from 1040 MWh to 1950 MWh on average. In comparison to conventional cooling system, a lower range of uncertainty was observed in general. To reiterate, the ATES policy is set to only substitute the energy-hungry chiller with the ATES. In this case, the primary cooling source is the ATES, however, the less efficient DC's economizer is utilized if necessary. In this case, the DC cooling system with the lowest setpoint (13 °C) used the ATES instead of the chiller 80% of the time. Indeed, higher temperature setpoints allowed longer operational hours of the DC's economizer, avoiding the unnecessary import of waste heat, which lead to an increase in the cooling electricity demand. For instance, the cooling operation with the highest temperature setpoint (25 °C) used the less efficient economizer 89% of the time. The results for the cooperation with the ATES without any policy show no difference in annual cooling electricity demand. The demand is constant at approximately 955 MWh. In this case, all rejected heat is imported to the ATES. From the perspective of the DC management, only several pumps with constant electricity demand are required to reach the desired setpoints.
Further, the influence of the aisle containment on annual cooling electricity demand was investigated. Fig. 6 shows the comparison of all scenarios with and without physical aisle separation for both design variants: the conventional cooling system and the system connected to ATES regardless of the policy (in the case with policy it is 955 MWh, as in the previous case).
The results show a reduction of the annual cooling electricity demand for both design variants. The reduction was about 430 MWh on average for conventional cooling system and about 170 MWh on average for the case of cooperation with the district ATES. The spread of the box plot is mainly caused by the variation of the temperature setpoints.
Finally, DC efficiency is indicated by the PUE metric (Oró et al. 2015b) for each operational scenario and each design variant. These efficiency results are presented as boxplots. The PUE metric, considering the range of DC management uncertainties, is shown in Fig. 7. The average PUE metric was calculated for conventional cooling system, the cooling system connected to the district-ATES with policy and connected to the district ATES without policy resulting in values of 1.78, 1.56 and 1.45 respectively.

Assessment of the integrated operation of the district ATES and DC systems
The outcomes of the uncertainty study together with the reported energy use of the existing campus are the main resources to assess the integrated operation of the district ATES system and DC. First, the DC waste heat potential is analyzed and compared with the energy use of other campus buildings that are connected to the district ATES system. Second, the annual energy demand, which is imported to or exported from the ATES-district system, is estimated for the actual and the future utilization scenario. Finally, the prediction of the CO 2 emissions reduction is presented and discussed.
As mentioned above, the main outcome of the campusscale analysis is the amount of heat imported to the district ATES system obtained in the uncertainty study. Fig. 8 shows the effect of ATES policy considering all operational scenarios. The average amount of imported heat results in 3330 MWh th for the variant with ATES policy and in 8120 MWh th for the variant with no limitations. The spread  of the box plots indicates the uncertainty of DC thermal management, where the higher bound of the results is related to a conservative management (low temperature setpoints without aisle containments), while the lower bound of the results is related to the best practice thermal management (high temperature setpoints with aisle containments). Fig. 8 demonstrates the energy use that is imported to or exported from the district ATES system by the connected buildings including the DC. It can be observed that the imported heat from the campus-DC dominates over all other campus buildings. Especially if the DC is not limited by any policy, the imported heat from the DC accounted on average for 32% of the overall heat stored in the district ATES system. In order to demonstrate the possible environmental benefits of the integrated operation, the annual thermal energy demand of the campus is assessed. Fig. 9 shows the heating and cooling demands of the campus by primary source. In addition, Fig. 9 also depicts the related amount of heat imported to and exported from the district ATES system. This assessment compares three possible cases. The first baseline case shows the situation without integrated operation for the actual utilization scenario. The second case shows the integrated operation for the actual utilization scenario considering the ATES policy. The third case shows the integrated operation for the future utilization scenario without the ATES policy. To reiterate, the actual and future scenario is defined based on the energy report for 2015 and energy plan for 2013-2016 (Meulen and Marks 2012;Meulen 2016).
As mentioned above, in the actual utilization scenario, the district ATES is imbalanced over a year, which means that imported heat from the other campus buildings exceeds the exported heat, and hence a significant amount of heat has to be artificially released to the ambient environment by the additional cooling tower integrated to the district ATES system. By connecting the DC into the district ATES system, the current imbalance significantly increases, as shown in the middle part of the chart. It is predicted that the cooling towers integrated into the district ATES system must release an additional 3500 MWh th on average.
To reiterate, the university intends to significantly reduce the gas consumption for heating to reach their environmental goals in the near future. Currently, 23000 MWh of heat is  produced by gas combustion. The assumption is that at least half of this amount is going to be covered by heat pumps connected to the district ATES, which increases the amount of exported heat from the district ATES system. The assumed balance of the integrated operation for the future utilization scenario is addressed in the last part of the chart, where the benefits of the DC waste heat utilization are demonstrated.
To illustrate the potential carbon reduction, Table 1 shows the emission intensity of electricity against natural gas. It is assumed that 1 m 3 of natural gas provides 10 kWh th using heat pumps with a coefficient of performance (COP) = 4. The carbon emission factor and intensity mentioned in Table 1 are used for conversion of the DC energy demand to the equivalent carbon emissions.
The estimate of carbon emissions considering the DC thermal management uncertainties is shown in Fig. 10.
Again, the same three cases were investigated and compared with each other. The difference between average values of baseline case without integrated operation and the situation with the integrated operation considering ATES policy is 678 tons of CO 2 . Even higher emissions reductions can be assumed in the future utilization scenario, where the DC waste heat utilization is not limited in order to balance the additional heating demand. In that case, the entire system can save 1003 tons of CO 2 on average. It is worth noting that this value corresponds only to energy savings of the DC cooling system.
From the broader perspective, Fig. 10 shows the breakdown of emissions per building on campus, including the DC case-study. While the energy report from the year 2015 (Meulen 2016) shows that the overall campus emitted the 25500 tons of CO 2 , the results demonstrate that the addition of the DC increases the total carbon emission of the overall campus by about 20% to 30760 tons of CO 2 on average.
The integrated operation of the DC and the district ATES policy mitigates the produced emissions by approx. 2.2% of the total carbon emissions, leading to a total of 30080 tons of CO 2 per year. From the perspective of the overall campus, these savings are comparable to the carbon emissions produced by a large administrative building (e.g. B9 N-Laag in Fig. 10) Higher savings can be assumed in the future utilization scenario, where the integrated operation has no limitations resulting from the storage balance requirement. The reduction of CO 2 emissions related with the integrated operation is approx. 3.3% of the total carbon emissions, which translates to a total of 29757 tons of CO 2 per year for the overall campus.
In addition, the future utilization scenario assumed that the DC waste heat will balance the district ATES system thereby supporting the reduction of gas consumption for space heating of other campus buildings. To also consider the emission savings of the other campus buildings, the total CO 2 emission of the integrated operation for the future utilization scenario results in 29140 tons of CO 2 per year on average (saving approx. 5.2%).

Recommendation for the DC management
The uncertainty study demonstrated a large influence of the DC thermal management on DC cooling performance.
The study mainly focused on the supply temperature setpoints and air management in the DC. The study revealed that a campus-DC can greatly benefit from the cooperation with ATES. For instance, the PUE efficiency improves from 1.78 to 1.45 on average. Considering the reported PUE in literature (Yuventi and Mehdizadeh 2013), the improvement might not seem so significant. However, it should be noted that the PUE usually stated in literature is for design conditions. In this study, the PUE is calculated based on dynamic simulation representing several operational scenarios considering also part-load efficiency of all parts of systems. Therefore, the absolute value of the PUE is not truly comparable with the one stated in most of the literature. The uncertainty study also revealed that the district ATES as a substitution for conventional chillers can reduce the uncertainty of the cooling system efficiency because the cooling source and related energy consumption is more stable over the whole year.
Moreover, the study shows that the level of rejected heat from the DC can significantly vary based on the applied DC thermal management, as shown in Fig. 8. This means that there is a certain flexibility of the DC as a heat producer, which may support optimization of the integrated operation of the district ATES system. Based on the obtained results, the integrated operation of DC and district ATES systems is in principle feasible and as such can be recommended as an energy efficient method of cooling DCs.

Recommendation for ATES management
In the presented case-study, the existing ATES system has enough storage capacity to accommodate the mid-sized DC and balance its seasonal mismatch. However, the DC as an industrial building type introduces into the campus district an enormous amount of waste heat and it requires a significant amount of energy. Hereunder, the results are discussed from the perspective of ATES management for the current and future utilization scenarios.

Current utilization scenario: Reduction of DC electricity demand and carbon emissions due to the integrated operation
In fact, the simulated amount of waste heat generates an imbalance in the cold and hot wells of the district ATES system for the current utilization scenario. For the current situation, the DC thermal management can benefit from the integrated operation because the DC can avoid the operation of the energy-hungry chillers at the expense of increasing operational hours of the cooling towers integrated to the district ATES system. It worth noting that the waste heat surplus is stored in the ATES until the weather permits free cooling. As such, the DC waste heat enters the district ATES system, but it is not truly recovered by campus buildings.
The DC addition is generally still recommended if the district ATES system is physically capable to accommodate the DC system. However, the facility management should carefully state the policy, which considers the additional requirements leading the integrated operation. Also, the policy must ensure the long-term (yearly) balance of the ATES system. However, the statement of the policy is beyond the scope of this work.
Due to the large capacity of the studied district ATES system, the policy was defined to be relatively benevolent for DC thermal management. The applied policy for all operational scenarios allows ATES to be used to completely substitute the operational hours of the DC chillers to fully demonstrate the environmental impact. For instance, the policy may require applying the best-practice DC thermal management. Such a thermal management approach allows the amount of rejected heat to be reduced to approx. 4790 MWh (still considering no operation of DC chillers). This amount of heat is comparable with the utilization of other regular campus buildings, and thus the thermal management approach is applicable even for smaller ATES systems or with a stricter policy.
From the environmental point of view, the integrated operation can reduce the DC carbon footprint by 678 tons of CO 2 per year on average, which roughly corresponds to the annual emission production of the university sports center or the educational building, Matrix, described in Fig. 10. It can be concluded that the integrated operation brings environmental benefits even though the waste heat is not truly recovered.

Future utilization scenario: Benefits of using DC waste heat for space heating in university campus buildings due to the integrated operation
The situation may change significantly in the near future. In accordance with the university carbon policies (Meulen and Marks 2012) and national policies of the Netherlands (European Environment Agency 2015), a reduction of natural gas consumption is expected. Applied to the studied university campus, these plans likely promote and accelerate a higher utilization of the district ATES system for heating purposes. Moreover, considering the additional heating demand of planned residential buildings in the campus, the balance of the ATES system will likely reverse from the current situation.
Considering the future utilization scenario, the DC as a cooling-dominated building type can help to reach the necessary balance of the wells. In this case the DC waste heat is truly recovered by other campus buildings. The results proved that the integrated operation brings environmental benefits to both participants. In this case, the DC thermal management is no longer limited, and it can fully use the energy efficient cooling provided by the district ATES system. From the other side, the district ATES system can store and supply the DC waste heat to all campus buildings, which leads to reduction of their gas consumption for space heating.

Conclusion
In this paper, an analysis of the waste heat utilization of a mid-sized campus data center was carried out. The integration of the DC into the existing district ATES system of the Eindhoven University of Technology (NL) campus is a feasibility study, which is part of the future development plans of the university. Three scenarios where investigated: (a) the proposed DC with a conventional cooling system (isolated from the district ATES), (b) the DC with waste heat utilization via ATES in the current campus (c) and the DC with waste heat utilization via ATES in the future campus.
The performance of the cooperation between DC and ATES was assessed and compared with the reference case of DC with a conventional cooling system. To assess these three variants, a numerical model of the DC was developed using building energy simulation. Specifically, the numerical model was created in TRNSYS and CONTAM. The model can represent the dynamic behavior of the DC energy performance.
Then, the numerical model of the DC was used to assess the waste heat potential of the mid-sized DC. First, the uncertainty of the DC performance, caused by the undecided DC thermal management, was evaluated. Various operational scenarios were defined and tested. The simulation results from the uncertainty study provided comprehensive information for the next stage. Second, the simulation results from the uncertainty study were combined with the statistical report of the university campus and existing ATES system. It was predicted that by the integrated operation, the DC could reduce its CO 2 emissions by 680 tons per year on average for the current situation considering the ATES policy. The expected increase of exported heat from the ATES may lead to changes in the policy. In the future scenario, the DC could import even more waste heat. Thus, the DC could reduce its CO 2 emissions by 1000 tons per year on average. In conclusion, this study proved the environmental benefits of the integrated operation of the DC and the district ATES systems.
Open Access: This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/