Evaluation of Fire Models by Using Local and Global Metrics and Experimental Uncertainty Estimates: Application to OECD/NEA Prisme Door Tests

The use of numerical methods in fire safety investigations for civil buildings and nuclear facilities has received enormous attention in recent years. To evaluate quantities—such as gas temperatures—in fire models, local metrics using single points (e.g. comparing maximum or minimum peak value of two time series) are well-established. Experimental (measurement and model input) uncertainty estimates can be used to quantify the model uncertainty. Although the peak value is a relevant and well-defined quantity, global metrics comparing the entire course of two time series can often provide additional information for the validation of fire models. A comparative methodology COMET for evaluating the predictive power of fire models is developed and presented in this paper. In the methodology, both local and global metrics are combined to incorporate the explanatory power of both quantities in the validation process. While uncertainty analysis is well established for peak values, to the best of our knowledge, there are no analytic results on quantifying the uncertainty of the global metric in the literature. We address the latter based on experimental measurements and derive confidence regions for both metrics. Finally, this paper summarizes the results using COMET to validate the Fire Dynamics Simulator (FDS) version 6 for a room fire scenario. Validation examples are tests 3, 4 and 5 of the DOOR series of the international OECD/NEA PRISME project, in which the transport of heat and flue gases through a door between two rooms was examined. Using COMET, we can easily identify sensors with high level of agreement between model and experimental results with respect to the local and/or the global metric.


Introduction
In fire safety engineering and many other fields, a variety of issues requires a comparison of different outcome time series of relevant quantities. These can be design issues as well as validation purposes. Both have in common that there is a demand of comparing not only a single time point of a time series but mostly the whole course of two curves (in a cumulative sense). In spite of this common and widespread problem, no standard procedure has been established as a solution yet, even though appropriate mathematical methods are available. An overview of the activities in this field is given in [1]. Documents to assess the predictive capability of fire models, e.g. from ASTM (American Society for Testing and Materials) and ISO (International Standards Organization), can be found in [2,3], among others. The mathematical foundation is given in [4]. In the nuclear sector in particular, there are extensive approaches towards quantification of the uncertainty in the evaluation process, e.g. in [5][6][7].
As a solution strategy for the described issue the comparison methodology COMET is developed and presented in this paper. As already intensively discussed in [7], the analysis of single points and the investigation of the overall distance of experimental and simulated curves are of parallel interest. COMET evaluates both criteria simultaneously. Exemplarily, Fig. 1 shows the course of the gas temperature measured in a specific experiment and simulated with a computational fluid dynamics (CFD) model.
Obviously, the peak values of both curves are almost identical (but not at the same time), while the overall behaviour differs significantly. As a metric for local effects the peak distance (PEAK), and for global effects the normalized Euclidean Figure 1. Gas temperature (TG), experiment (EXP) and simulation (SIM) with a CFD fire model, time series representing sensor ''TG_FP_240'' from OECD/NEA PRISME DOOR test 5 [7]. distance-based measure (NED) is used in COMET. The latter global evaluation parameter is considered amongst others by Peacock et al. in [4]. As [7] points out, PEAK (of temperature) is the relevant parameter when e.g. determining the maximum level of stress on building materials during a closed compartment fire. Even if two temperature curves are generally rather close to each other, their PEAK values may differ which may have a significant effect on material stability.
Although for almost all relevant quantities the PEAK criterion is an important information, the course of a quantity over time is an equally important aspect of the impact on the building and for human life in the event of a fire. Hence, the comparison of the overall courses of the entire time series should be considered as well [7]. Conclude that a parallel evaluation of the PEAK and NED metrics is generally advisable.
We introduce the two-dimensional graphical tool COMET for the comparison of model prediction and experiment based on uncertainty quantification of PEAK and NED criteria. While confidence regions for PEAK have already been established by the United States Nuclear Regulatory Commission (NUREG 1824) [5,6] and the National Institute of Standards and Technology [8], additional effort is required to derive confidence intervals for NED in a similar manner.
In the literature, approximate ranges of uncertainty for NED are therefore sometimes constructed by using the overall experimental uncertainty; see for example [7]. However, as those authors point out, this approach does not take the temporal dependence of time series observations into account (which generally leads to significantly larger confidence regions). The contribution in this paper proposes an improved estimation of the NED uncertainty, which takes the temporal dependencies into account, for the purpose of confidence interval construction. These intervals then serve as ranges of uncertainty (RoU) in the model evaluation process. It is worth noting that the result is a formula that depends exclusively on experimental observations and on the quantities commonly used for PEAK in the literature, and is therefore easily applicable.
Previous studies with respect to local and global effects have already been used to validate simulations concerning results coming from the international OECD 1 / NEA 2 PRISME 3 Project [7,9,10]. The acronym PRISME comes from the French phrase propagation d'un incendie pour des sce´narios multi-locaux e´le´mentaires, which in English can be translated as ''fire propagation in elementary, multi-room scenarios''. In particular, the PRISME LEAK [11] and PRISME DOOR [12,13] test series have been already investigated to validate the CFD fire simulation code FDS Version 6 within the OECD/NEA PRISME project. As an example, the PRISME DOOR test 3 (PRS_D3), PRISME DOOR test 4 (PRS_D4), and PRISME DOOR test 5 (PRS_D5) have been chosen for this publication in order to demonstrate the methodology and the results gained by its application for FDS Version 6.7.0 [14].
The remainder of this paper is structured as follows. In Sect. 2 the two metrics PEAK and NED used in our novel tool COMET are reviewed and discussed. COMET is then introduced in Sect. 3. As an example for the application of COMET we use results from the PRISME DOOR experiment series described in Sect. 4. Section 5 describes the CFD simulations to be validated. The application of COMET in the setting of Sects. 4 and 5 is described in Sect. 6. Section 7 summarizes our results. Mathematical derivations for the uncertainty estimates are deferred to the Appendix.

Evaluation Criteria
The methodology proposed in this paper for the analysis of differences between two time series-one from the experiment and one from a simulated model-includes two criteria. We combine the well-established PEAK method according to [3] with the normalized Euclidean distance (NED) introduced in [2].
The local comparison criterion PEAK describes the relative difference of model peak and experimental peak, and is given by where DM p is the difference between the peak value M p of the model prediction and its baseline value M 0 , and DE p is the difference between the peak value E p of the time series of experimental measurements and its baseline value E 0 . This method allows a very fast and easy evaluation of the deviation-with regard to the extreme values-of the time series from each other. However, it does not give information about global similarities of the paths of the two time series. In particular, it lacks information about the time points of the extreme events DM p and DE p . Moreover, the distance of model prediction and experimental measurements at any other time point is not incorporated. This may be a challenge for model evaluation in practice, cf. Figure 1 for an example with small PEAK value where the curves differ significantly on a global scale. As an additional criterion for a global comparison, the normalized sum of squared differences-the so-called normalized Euclidian distance NED (also known as standardized L2 norm) is used in COMET which is defined via where t runs from 1 until T and represents the time points at which each quantity is measured (E t ) or modelled (M t ). As before,DM t ¼ M t ÀM 0 and DE t ¼ E t ÀE 0 . It was initially presented by Peacock in [2] and is easily interpretable as highlighted in [7]. In the present context, NED describes a quantification for the deviation of the experimental and the model prediction curve during the whole temporal course of the time series and has some key benefits. Note that squaring of the deviation at the single data points ensures that positive and negative deviations cannot compensate each other. Moreover, the consideration of squared distances implies a stronger penalization of large distances compared to small distances. Larger values (e.g. the temperatures in the hot gas layer) are expected to have larger absolute deviations compared to those with smaller values (e.g. the temperatures in the lower, cooler gas layer). A comparison based on absolute values and without normalization is generally feasible, but might assign large NED values to certain time series although their relative differences may be small. This can be avoided by normalisation with respect to the measurements. On this basis, it is also possible to compare the metrics of different physical quantities; see Fig. 2 for an illustration example. There, for a large number of different sensors in a particular PRISME DOOR experiment, both the PEAK and the NED deviation between model simulation and experimental measurement are calculated for each sensor, and the (NED, PEAK) results of the different sensors are depicted as points in a two-dimensional scatterplot. In Fig. 2 r; l and m denote empirical standard deviations, means and medians of PEAK and NED, respectively.
Comparing PEAK and NED, first note that PEAK has neither an upper nor a lower bound and yields the value 0 as an optimal result, in terms of an exact congruence of the extreme values in experiment and simulation. The value -1 describes a significant limit. For values smaller than -1, the signs of the extrema  Table 5 below), PRISME DOOR, test 3, test 4 and test 5, model FDS 6.7.0. of the experiment and simulation are different. A detailed analysis of the time series is advisable in this case. The NED value in Eq. (2) merely has a lower bound, the value 0, which, as for PEAK, describes an optimal result. Finally, it can be seen from Fig. 2 that there is a significant difference between (absolute) PEAK and NED values for the majority of the experiments. Thus, in order to decide whether the model is suitable for the experiment at hand, it is indispensable to incorporate confidence regions for PEAK and NED in the validation process.

The Tool COMET
COMET is a two-dimensional graphical tool for the comparison of model prediction and experiment based on uncertainty quantification of NED and PEAK criteria. Figure 2 shows the basis for this tool: A scatterplot of the observations from sensors from one experiment, where for each sensor the NED value between model and experiment is depicted on the abscissa, and the PEAK value on the ordinate. To get a benchmark for the magnitude of these values, COMET also contains approximate 95% confidence ranges for both quantities. The result can be seen in Fig. 3; the red area gives the confidence range for NED, the green one the confidence range for PEAK. At the intersection of the red and green area both metrics are within their respective confidence regions. To assess the boundary of these ranges, we rely on [5] (Sect. 1.4.2) for the PEAK criterion. There, the 95% confidence interval is approximated by [-U PEAK, + U PEAK ], where Here,Ũ E denotes a measure of relative experimental (measurement) uncertainty andŨ M denotes a measure of relative numerical (model input) uncertainty, see [5] (Sect. 1.4.2) for details. For the present study the relative uncertaintiesŨ E ¼ 2ũ E andŨ M ¼ 2ũ M to determine U PEAK are taken from [6] (see Sect. 3.3.3) and are shown in Table 1. This approach will be used as the basis for evaluating the PEAK results of PRISME DOOR.
To the best of our knowledge, there are no results on uncertainty quantification based on NED in the literature. For that reason [7] usedŨ E as a rough approximation. However, it turns out that one can follow the ideas presented in [5] (Sect. 1.4.2) to derive an analogue to U PEAK which is in general substantially larger thanŨ E : Still, the design of the underlying time series model is much more delicate since we have to consider temporal dependencies within the experimental and within the simulated curves. A detailed description of our approach is given in the Appendix. Finally, we will obtain an approximation for the variance of NED: In line with the PEAK approach, the latter quantity is a function of the uncer- Table 1. As the NED values are always positive, the accuracy of numerical predictions concerning the NED criteria is given with the confidence interval [0, + U NED ] with Table 1 Measurement (E) and Model Input (M) UncertaintyŨ ¼ 2ũ According to [6] Measured variable in [5]Ũ EŨ M Reference to measured variables for this study (sensor group), see Table 5 Hot gas layer temperature  Table 5 for abbreviations and Fig. 6 and Fig. 7 for sensor locations In Fig. 3 an illustration of the two-dimensional graphical tool (COMET) for the comparison of model prediction and experiment based on uncertainty quantification of NED and PEAK criteria is given. Every sensor is marked by a point in the plane with its NED value on the abscissa and its PEAK value on the ordinate. The RoUs for PEAK (P) and for NED (N) resulting from the confidence intervals [-U PEAK, + U PEAK ] and [0, + U NED ], respectively, are highlighted as green and red bands. The uncertainty measures for PEAK and NED are denoted in percent and are averaged values for the compared sensors. In Fig. 3 PEAKs (P), NEDs (N) and PEAK/NEDs (P/N) denote the proportions of sensors lying in the green RoU of PEAK, in the red RoU of NED and the intersection of the green and red RoUs, respectively. Additionally, for statistical evaluation, standard deviation (r), mean (l), and median (m) for PEAK and NED have been computed for all data and for individual sensors. In this sense, Fig. 3 delivers a rather compact but yet informative presentation of our model evaluation. It can be summarized as follows: For all sensors (points) in this graphic lying in the intersection of the green and red area, model and experiment fit well with respect to both criteria; NED and PEAK. For all sensors (points) in this graphic lying within the green but outside red area, model and experiment fit well with respect their PEAK values but differ significantly with respect to their overall structure (and vice versa for points within the red but outside the green area).
If the sensor points are located in the white area, there are doubts on the validity of the model. For sensors behaving as described in the latter two bullet points, a deeper analysis is advisable, e.g. using the actual plots of modeled and measured results as displayed in our Fig. 1.
In contrast to PEAK, NED is capable of detecting deviations between the curves. As an example, the (NED, PEAK) point resulting from sensor TG_FP_240 illustrated in Fig. 1 is highlighted (black filled). While PEAK is nearly zero (in particular, lying within the green RoU of PEAK), NED is remarkably large and, hence, outside the red RoU of NED. This can be interpreted as follows: While the maximum values of the experiment are reasonably simulated by the model, this is not the case for the overall gas temperature curves over time. In Sect. 6.2, this behavior is discussed in further detail.

Experimental Setup
The tests of the first test series PRISME DOOR within the OECD/NEA PRISME project [12,13] were carried out by the IRSN (Institute de Radioprotection et de Suˆrete´Nucle´aire) in the test facility DIVA in Cadarache (France). The experi-ments on the DOOR series (1 to 5) of the OECD/NEA PRISME project were carried out in the rooms ''Local 1'' (L1 or Room 1) and ''Local 2'' (L2 or Room 2) of DIVA, cf. Figure 4 for an overview of the spatial conditions. The DIVA compartments are located in the JUPITER facility, which has a volume of total 3600 m 3 and a net volume of approximately 2700 m 3 considering the DIVA internals.

Room Geometry and Ventilation
Each of the lower cuboid-shaped rooms (room 1 to 3) has a volume of 120 m 3 with a clear dimension of 6 m 9 5 m 9 4 m and is connected to a complex ventilation system, which controls the rooms' air exchange via inlet and outlet ducts (see Fig. 5). The walls of these rooms are made of 30 cm thick concrete. During the tests, the ceiling and the walls of the fire room (room 1) were provided with a 5 cm thick insulation layer of rock wool, in order to avoid spalling of the concrete and thus damage to the test facility. For the DOOR series, the door between the two rooms (L1 and L2) was open. The door opening has a dimension of 0.8 m 9 2 m and is located in the middle of the partition (see Fig. 5).
The air exchange rate in the tests PRS_D3 and PRS_D5 was 4.7 l/h or 560 m 3 / h for both rooms (fire and target room); in test PRS_D4 the air exchange rate was 8.4 l/h or 1000 m 3 /h, see Table 2.
The inlet and outlet ports during these tests were located at the top of the compartments, which means 75 cm below the ceiling.

Fire Source
The fire source (liquid pool, see Fig. 5) was modelled as a rectangular area roughly corresponding to the pool size used in the experiment, which was 0.4 m 2 for PRS_D3 and PRS_D4 and 1 m 2 for PRS_D5, respectively. The required reaction parameters for the fuel (n-dodecane) used in all experiments (C12H26) can be specified directly in the CFD program and are given in Table 3. Soot yield (SOO-T_YIELD), carbon monoxide yield (CO_YIELD) and the heat released per unit mass O 2 (EPUMO2) were taken from the SFPE Handbook [15].   Table 4 contains the properties of the materials for walls, ceilings, fuel pan and ventilation ducts used in the experiments. The properties for PVC analytical and real cable can be found as well. In FDS a simple 1D heat conduction calculation is carried out across the thickness d of the material and, as a result, the temperatures and the gradient inside are determined. In addition, as Table 4 shows, the thermal conductivity k and the specific heat capacity c p can be set as a function of the temperature.

Instrumentation
In Fig. 6 a schematic of the two rooms, noted L1 and L2, and selected instrumentation are given. For more information concerning the legend see Table 5.

Target Objects
In order to investigate the effects of the fire on safety-related objects (so called targets) in the experiments on the PRISME DOOR series two types of objects are used: On the one hand, PVC rods, so-called ''analytical cables'', and on the other real PVC cables (used in the tests PRS_D4 and PRS_D5). These objects were placed on horizontal steel ladders on the walls at the top and bottom of both rooms Li (i = 1, 2), as shown in Fig. 7 in detail and later in Fig. 9. In addition, the total heat flux density (FLT) at two locations (_UPW and _DWN) and gas temperatures at the steel ladders (TG) nearby the target (_TA) were measured, and analytical (TCA) and real cable temperatures (TCR) at different positions of the cables (_SURF, _INTER, and _CENTER) each at three sections (not shown) in each target object.

Simulations with Numerical Model
We illustrate the COMET approach with simulations based on the Fire Dynamics Simulator (FDS) which is well-established in the fire community and frequently used internationally. However, COMET is not restricted to this particular model but can be applied to other numerical fire models as well. Extensive information about the FDS model is given in [14]. FDS was used without changing the default  In open simulations, there is essentially a specification of the scenario with regard to the geometry of the room, the physical parameters of the enclosure components and the course of the experimentally determined heat release rate of the fire source, which is known as the input parameter. The specification of the heat release rate corresponds to the state of the art in fire modelling purposes.
The simulations for PRS_D3, PRS_D4 and PRS_D5 were performed with FDS, version 6.7.0 [14]. The results are presented and discussed in the following section.
For the simulation, time courses of the heat release rate (HRR) must be specified. For this study, open simulations for validation were used as a basis, that is, the course of the HRR which was measured (see Fig. 8) during the individual experiments in the test facility DIVA. As heat of combustion a value of 46 MJ/kg was used for n-dodecane. Volume flows (inlet/outlet) were specified as boundary conditions. A 10 cm grid was used for the simulation of the gas phase. Finer grating checks have not revealed any significant change in the calculated sizes. The calculation of heat conduction in solids was carried out independently of the gas phase on a much finer grid.
Modelling suppression of a fire due to the exhaustion of oxygen within a closed compartment is challenging because the relevant physical mechanisms typically occur at subgrid-scale. Flames are extinguished due to lowered temperatures and dilution of the fuel or oxygen supply [14]. FDS with default settings uses a simple suppression model. There must be sufficient energy released to raise the cell temperature above the critical flame temperature for combustion to occur. This is the case when SUPPRESSION = TRUE is set in the FDS input file. To illustrate the advantages of using COMET, FDS version 6.7.0 was also used without any flame suppression (i.e. SUPPRESSION = FALSE in the FDS input file). For results concerning different settings from FDS SUPPRESSION parameter consult Sect. 6.2.  The input model contains about 300 measuring points/sensors, which were also used in the experiments. In this way, a comparison of almost all measured quantities between simulation calculation and experiment is possible.   Table 5 lists the analysed and evaluated quantities. Since the measurements were carried out at different heights, the ''original'' designations may be additionally provided with height information. Although pressure measurements were also carried out, the results were distorted or unrealistic on both the experimental and the simulation side, so this quantity was not analysed.
Objects and housing components were considered thermally and for a one-dimensional heat transfer calculation the thermal conductivities and the specific heat, if present, were considered temperature-dependent.

Application of Methodology
The computation results of the open simulations of the test are contrasted with the data measured during the test and analysed by an application of COMET for the time series as described above. Table 6 summarizes the uncertainties of the analysed sensor groups and the number of evaluated and analysed sensors for this investigation. For the evaluation, a limitation of the data with respect to the evaluation criteria PEAK and NED was assumed: Data that were outside the interval [-1; 1] for PEAK and outside the interval [0; 1] for NED were excluded from further evaluation. For PEAK or NED values, which lie within these intervals, it is assumed that there were no irregularities in the experimental execution or in the CFD fire simulation model. 6.1. PEAK/NED Analysis for All Sensor Groups (Weight D3, D4, D5) Figure 10 shows the PEAK/NED analysis for all sensor groups named PEAK/ NED-CO, CO 2 , FLT, O 2 , TCA, TCR, TG, and TP. The results are sampled for PRISME DOOR test 3, test 4 and test 5 (D3, D4, and D5 in the following) and illustrated with COMET. Table 6 Uncertainties U NED and U PEAK for PRISME DOOR and Number of Used Sensors (using FDS 6.7.0) PEAK values close to zero exist for all sensor groups. It can be seen, however, that NED values close to zero can be observed for certain sensor groups, only   Table 7 summarizes the results concerning this validation study (see Fig. 10 for details) from FDS version 6.7.0. For each sensor group, the table lists the standard deviation, the mean value and the proportion of PEAKs and NEDs and of PEAKs and NEDs (PEAK/NEDs) in percent [%] that are within the range of uncertainty. Figure 11 summarizes the results concerning this validation study. For each sensor group the mean (NED, PEAK) value l is illustrated. The figure also contains horizontal and vertical whiskers for each sensor group. The horizontal whiskers of length U NED are used in a one-sided manner (to the left) since all NED values are larger than the optimal value zero by construction of this quantity. If these whiskers cross the ordinate, the model fits the data well (on average) for the corresponding sensor class in terms of NED value. Since PEAK values of sensors can be smaller or larger than the optimal value zero, we use two-sided whiskers of length U PEAK here. If they cross the abscissa, the model fits the data well (on average) for the corresponding sensor class in terms of PEAK value.
The results illustrated in Fig. 11

Impact of FDS SUPPRESSION Parameter on Gas Temperature (TG) Modelling
In  Fig. 13 (for more details see [10] and Sect. 5).
RoU for PEAK (meaning fraction of data lying within RoU of PEAK) is about 47.1% versus 49.5%, RoU for NED is 52.9% versus 39.1% and for PEAK/NEDs the RoU is 39.3% versus 33.3%. Switching the model parameter SUPPRESSION from its default value (TRUE) to FALSE strongly improves the model adequacy with respect to NED while the PEAK values do not change significantly which confirms the benefit of the evaluation of the NED criterion in addition to the PEAK value. In detail the forecast capability for sensors NW (TG sensor north west) and SE (TG sensor south east) strongly improves with respect to NED, in case of using SUPPRESSION = FALSE in the model (and in connection with the investigated experiment).

PEAK/NED Analysis for Different Parameters for All Sensor Groups
In order to obtain information on how much of the observed uncertainty is due to the individual test and how much is due to the respective room, two groups are analyzed separately in the following. The groups are PRISME DOOR test (D3, D4 and D5, Fig. 14) and room (L1 and L2, Fig. 15). Since all sensor groups were evaluated together according to Table 7, a representation of the RoUs (green and red areas) for all physical quantities in one diagram is not feasible.
The results illustrated in Fig. 14 are summarized in the following. Standard deviation (r) mean (l) and median (m) for PEAK and NED are significantly higher for D5 than for D3 and D4 Mean (l), and median (m) for PEAK are positive, which indicates over-estimation of the relevant sensors Some sensors show higher NED values (NED > 0.4), but PEAK values close to zero, coming from PRISME DOOR test 5, while the contrary situation does never occur The results illustrated in Fig. 15 are summarized in the following.
Standard deviation (r) mean (l) and median (m) for PEAK and NED are significantly higher for the door between room L1 and L2 (L1_L2) and room L2 than for room L1 Mean (l) and median (m) for PEAK are nearby zero for room L1 which indicates good forecast capability for PEAK values in room L1; in contrast, from higher NED values we can conjecture that the simulation forecast for the overall curve does not reach a convincing level. A deeper investigation using COMET for individual sensors or specific sensor groups as in Sect. 6.1 will provide more insights.

Conclusion and Outlook
For many validation purposes, the local PEAK criterion provides far-reaching information about the prognostic capability of CFD models. Nevertheless, the global NED method can provide additional information about the performance of sub-models used in a CFD model for fire simulation. We introduced the novel two-dimensional tool COMET for uncertainty quantification with respect to both of these frequently used local and global metrics in parallel. To obtain the boundaries of the RoU for NED we applied similar techniques as described in NUREG [5,6] for the corresponding uncertainty quantification for PEAK. A set of three experiments coming from OECD/NEA PRISME DOOR test 3, test 4 and test 5 was used to demonstrate the new approach. The results emphasize that an investigation of both metrics is inevitable.
In particular, it is shown that the gas temperatures calculated with the model are often higher than those determined experimentally, i.e. they show predominantly positive PEAK values. In comparison to the first room, larger deviations of statistical values such as standard deviations, means and medians of the (NED, PEAK) values are obtained for the studied sensors in the adjacent room. It can be assumed that the transport of the physical quantities from the first room through the door into the second room cannot be reproduced with the same prediction accuracy in the model with the default values set here.

3110
Fire Technology 2022  Carbon dioxide and temperatures of real cables shows very low mean PEAK, but higher NED values, which indicates good forecast capability of local aspects (peak of time series course), but lower forecast capability of global aspects (course at every time point).
Summing up, using COMET instead of simply PEAK can provide substantial additional information about model performances. Still, to check the performance of a model, it may not be useful to evaluate all possible sensors with the COMET method. For example, for the evaluation of temperatures in the plume, it is certainly useful to check the ability of the model for the prediction in the close range of the flames. Moreover, several additional issues have to be tackled before the new methodology can serve as a guide for regulatory compliance. For instance, it might be useful to restrict the time interval for which NED is applied since the ramp-up period of fires is known to be rather volatile, and the influence of this ramp-up phase on intervention measures is rather negligible. All these kinds of considerations go far beyond the scope of the paper and are left for further research.
To this end, we interpret NED as a function of random quantities, to be precise. as T ! 1. On the other hand, the first summand on the right-hand side of (8) can be written as