On the application of neural networks for temperature field measurements using thermochromic liquid crystals

Abstract

This study presents an investigation regarding the applicability of neural networks for temperature measurements using thermochromic liquid crystals (TLCs) and discusses advantages as well as disadvantages of common calibration approaches. For the characterization of the measurement technique, the dependency of the color of the TLCs on the temperature as well as on the observation angle and, therefore, on the position within the field of view of a color camera is analyzed in detail. In order to consider the influence of the position within the field of view on the color, neural networks are applied for the calibration of the temperature measurements. In particular, the focus of this study is on analysis of the error of temperature measurement for different network configurations as well as training methods, yielding a mean absolute deviation and a mean standard deviation in the range of 0.1 K for instantaneous measurements. On the basis of a comparison of this standard deviation to that of two further calibration approaches, it is shown that neural networks are suited for temperature measurements via the color of TLCs. Finally, the applicability of this measurement technique is illustrated at an exemplary temperature measurement in a horizontal plane of a Rayleigh–Bénard cell with large aspect ratio, which clearly shows the emergence of convective flow patterns by means of the temperature field.

Graphic abstract

1 Introduction

For the detailed analysis of many natural and technical systems, the simultaneous determination of several physical quantities is necessary. In many cases, simultaneous measurements of the velocity and temperature are of interest, as these quantities allow for investigations of the transport of momentum and heat, e.g., in natural convection (Schmeling et al. 2014; Tummers and Steunebrink 2019), process engineering (Massing et al. 2018) and general heat transfer problems (Cafiero et al. 2014; Irwansyah et al. 2016). For this purpose, many different measurement techniques with each having its own specific advantages and disadvantages have proven to be successful, enabling the choice of an appropriate measurement technique based on the application. Among those techniques for simultaneous measurements of the velocity and temperature field, the use of TLCs is distinguished by offering the possibility to measure temperatures in macroscopic (Fujisawa et al. 2004; Moller et al. 2019) and microscopic (Basson and Pottebaum 2012; Segura et al. 2013) applications over a relatively small temperature range very precisely. Therefore, TLCs are predestinated for applications, which require only a small temperature measuring range. On the contrary, the well-established temperature measurement technique laser-induced fluorescence (LIF) is frequently used to determine temperature fields covering a larger temperature range (Sakakibara and Adrian 1999; Banks et al. 2019), however, at the expense of an increased uncertainty of temperature measurement.

In most cases, TLCs are illuminated with white light for the temperature measurements and only reflect a certain range of the incident wavelength spectrum of illumination in dependency of the temperature, thereby showing different colors which can be used for optical temperature measurements (Hiller and Kowalewski 1987; Stasiek 1997; Dabiri 2008). In particular, using a so-called supercontinuum laser for the illumination, which emits a broad spectrum of wavelengths within the visible range, holds a lot of promise, since the possibility of generating a thin white light sheet opens up new options for measurements with a high spatial resolution (König et al. 2019). However, the color signal of TLCs does not only depend on their temperature, but also on the angle between illumination and observation, which affects the temperature measuring range as well as the uncertainty of temperature measurement, shown by Nasarek (2010) and quantified by Moller et al. (2019). As this angle, denoted by observation angle from now on, also varies within the field of view of the recording device, which might be a color camera for instance, its effect must be taken into account in the calibration of the temperature measurements. Within the scope of this study it is investigated, to which extent the dependency of the color of TLCs on the position within a camera’s field of view can be corrected using neural networks for the calibration of the temperature measurements.

The development of neural networks, which are able to learn the relations of a system in analogy to the human brain based on a set of training data (Aggarwal 2018), goes back to the mid of the twentieth century. Already from that time on, many different fields of application opened up, which was reviewed comprehensively by Widrow et al. (1994). However, due to the enormous rise in computing power in the last decades, neural networks can nowadays be applied to a much wider variety of problems and, therefore, have gained an increasing importance. In this context, numerous algorithms for the training of neural networks were developed, improved and implemented in software, in order to enhance the learning ability of neural networks (Burney et al. 2007). Furthermore, besides the simple feedforward neural network many types of neural networks with different interaction possibilities between the single neurons and various mathematical operations within their layers have been created, e.g., recurrent neural networks and convolutional neural networks (Haykin 2008). Hence, depending on the problem the most appropriate type of neural network and training algorithm can be chosen for the optimization of the results, such that neural networks are used in a wide field of applications. For example, neural networks are very popular for pattern classification, clustering, function approximation, forecasting, optimization and control engineering (Jain et al. 1996). Unfortunately, in the field of neural networks neither a precise notation nor a fundamental rigorous mathematical understanding about how the networks are working is available (Bronstein et al. 2017; Genzel and Kutyniok 2019). However, these questions are currently addressed by mathematicians as well as computer scientists and go beyond the scope of this work.

In the present study, a neural network is used for function approximation, meaning to find a relation between the color and temperature of TLCs, based on a set of training data which contains many images of the TLCs with known temperatures. Grewal et al. (2006) were to the authors’ knowledge the first to apply neural networks for the calibration of temperature measurements using TLCs and obtained a mean absolute error of about 0.1 K, averaged over a temperature range of 8 K. However, they investigated the color appearance of a thin sheet of TLCs with respect to surface temperature measurements, where the observation angle was set to $15^\circ$. Here, the investigations are performed with regard to simultaneous measurements of the velocity and temperature field inside an extended fluid volume, which is accompanied by several restrictions, e.g., the observation angle and the exposure time of the camera for the recording of the TLC images. Therefore, especially the uncertainty of temperature measurements has to be analyzed for this kind of application when using neural networks for the calibration, which is compared to those obtained by two further established calibration approaches. The aim of this study is to evaluate whether neural networks can be applied for temperature measurements via the color appearance of TLCs without writing complex program codes that have to be adapted to the specific problem of interest. For this reason, the Deep Learning Toolbox of MATLAB R2019a (Beale et al. 2019) is used, since it allows to easily adjust the structure of neural networks in terms of the number of neurons and layers as well as the algorithm for training the neural network. In order to demonstrate the applicability of this method, a neural network designed with this Deep Learning Toolbox is exemplarily used for determining the temperature field in a large aspect ratio Rayleigh–Bénard cell, which is directly opposed to the results obtained by another established calibration method for discussion.

2 The experimental setup

In the present case, TLCs are applied to determine the temperature field in Rayleigh–Bénard convection. This type of flow, induced by density gradients in an enclosed fluid volume which is heated from below and cooled from above, is suited for the characterization of a temperature measurement technique, as the temperature range of the medium inside the Rayleigh–Bénard cell can be adapted with the temperature of the heating plate at the bottom and cooling plate at the top, according to the temperature range of the color play of the TLCs. For this study water is used as the working fluid in the Rayleigh–Bénard cell. The Rayleigh–Bénard cell has dimensions of $l \times b \times h = 700\,{\mathrm {mm}} \times 700\,{\mathrm {mm}} \times 28\,{\mathrm {mm}}$ and accordingly an aspect ratio of $\Gamma =l/h=25$. In such large aspect ratio cells, the so-called turbulent superstructures (Pandey et al. 2018) can be observed clearly and their spatial and temporal scaling can be analyzed in detail, which was the main motivation for building up the experimental setup. In order to study the turbulent superstructures, optical measurement techniques for the determination of the velocity and temperature field are applied, such as liquid crystal thermography for the present study. However, besides the transparency of the sidewalls the investigation of the turbulent superstructures especially requires optical access to the horizontal cross section of the Rayleigh–Bénard cell. In the case of large aspect ratio cells, this must be realized by either a transparent heating plate (Kästner et al. 2018) or cooling plate. Here, the sidewalls and the whole cooling plate made of glass are transparent, which is why a large field of view can be analyzed when investigating the TLCs in horizontal planes of the cell. However, due to the widely varying observation angle within the large camera’s field of view, which causes a considerable change in the temperature dependency of the color of TLCs for different positions, special care has to be taken for the calibration (Moller et al. 2019).

The cooling plate at the top of the Rayleigh–Bénard cell is cooled by water flowing above it in a separate cooling circuit. As illustrated in Fig. 1, this circuit is covered by a second glass plate at the top, so that the optical accessibility is not impeded. The temperature of the heating plate at the bottom is controlled by water flowing through meander channels in the heating plate within a second external temperature-controlled water circuit. The heating plate is made of aluminum, which compensates for temperature gradients on its surface due to a high thermal conductivity. Furthermore, the cell is equipped with several small temperature sensors at the heating and cooling plate as well as at the sidewall, in order to measure the temperature during operation. As an option, punctual temperature measurements inside the Rayleigh–Bénard cell can be performed with a temperature measuring lance.

Fig. 1

Sketch of the experiment (left) and a photograph of the real experimental setup (right) for simultaneous velocity and temperature field measurements in horizontal planes of the Rayleigh–Bénard (RB) cell. The photograph shows the Rayleigh–Bénard cell with aspect ratio $\Gamma =25$. It should be noted that the coordinate system in the left corner of the sketch is valid for all of the following investigations

For measuring the temperature field in a horizontal plane of the Rayleigh–Bénard cell, microencapsulated TLCs of type R20C20W (LCR Hallcrest) with an average diameter of about $10\,\upmu$m based on a Gaussian distribution are used in this case. The TLCs have a density in the range from $1.00\,{\mathrm {g/cm}}^3$ to $1.02\,{\mathrm {g/cm}}^3$, which nearly matches that of water in the given temperature range. Therefore, the TLCs are nearly neutral with respect to buoyancy and their velocity of sedimentation is several orders of magnitude lower than the typical velocities occurring in the presented Rayleigh–Bénard experiment. According to the manufacturer’s data, these TLCs start to get red at a temperature of $T=20\,^\circ \hbox {C}$ and the color passes the visible spectrum with increasing temperature until they appear blue at $T=40\,^\circ \hbox {C}$. However, these specifications only apply to the case that illumination and observation are done from the same direction, which is not possible when observing the TLCs within a light sheet. If the camera is not orientated parallel to the light sheet and, therefore, the observation angle according to Fig. 1 deviates from $\varphi =0^\circ$, the temperature range with a low uncertainty of temperature measurement decreases with increasing observation angle up to $\varphi =90^\circ$. For this reason, the camera is not arranged perpendicularly, but at a nominal angle of $\varphi =70^\circ$ to the measurement plane covered by the light sheet. However, considering the image angle of the camera’s objective and the refraction of the light at the cooling plate, the observation angle varies from about $\varphi =70^\circ$ up to $\varphi =80^\circ$ within the field of view, as indicated in Fig. 1. This arrangement represents a suitable compromise to perform temperature measurements with a low uncertainty of measurement over a certain temperature range as well as with a high spatial resolution over an extended field of view, which is investigated in the following section. Details regarding the uncertainty of temperature measurement in dependency of the observation angle can be found in Moller et al. (2019).

For the temperature measurements in horizontal planes of the Rayleigh–Bénard cell via the color signal of TLCs, those are recorded with a color camera (sCMOS pco edge 5.5), which is equipped with a Bayer filter for color recording and a wide-angle lens (Zeiss Otus 1.4/28), in order to observe a large field of view. Furthermore, the lens is connected to the color camera via a tilt adapter, so that the whole measurement plane can be projected onto the camera’s sensor as sharply as possible according to the Scheimpflug condition.

In order to provide the white light, which is necessary for the illumination of TLCs, in form of a white light sheet for two-dimensional measurements within horizontal planes, 90 white high-power LEDs (Platinum Dragon, Osram GmbH) are arranged along a line at the height of the measurement plane. For the generation of a white light sheet out of the light emitted by the LEDs, which is strongly divergent, a special light sheet optic was designed, see Fig. 2 (Schmeling et al. 2014). This enables to shape a light sheet with adjustable thickness over the Rayleigh–Bénard cell’s whole cross-sectional area of $700\,{\mathrm {mm}} \times 700\,{\mathrm {mm}}$. For the measurements of this study, the thickness of the light sheet was adapted to about $t_{{\mathrm {ls}}}=3\,{\mathrm {mm}}$ in its focus in the center of the cell and increases to about $t_{{\mathrm {ls}}}=4\,{\mathrm {mm}}$ toward the sidewalls. On the left side of Fig. 3, the light sheet at the back sidewall is shown at mid-height of the Rayleigh–Bénard cell for better visibility. However, for the measurements presented in this study the light sheet reaches from $z=23$ mm to $z=27$ mm at the sidewalls, when measured from the surface of the heating plate at $z=0$ mm. Therefore, the light sheet illuminates the area close to the cooling plate of the cell with its height of $h=28$ mm. Accordingly, the benefit of using TLCs for temperature measurements can be illustrated clearly, as the most distinctive flow structures, which can be retraced by the color of the TLCs due to different temperatures, occur in close vicinity to the isothermal plates.

Fig. 2

Sketch of the light sheet optics (left) and a photograph showing the interior of the light source for the generation of a white light sheet (right)

Fig. 3

Photograph of the setup showing the light sheet, which was positioned in a horizontal plane close to the center of the Rayleigh–Bénard cell for better visibility (left) and the field of view for the measurements of this study indicated in a view from the top (right)

For inducing thermal convection inside the cell, the temperature of the heating and cooling plate was adjusted to $T_{\rm h}=21.5\,^\circ \hbox {C}$ and $T_{\rm c}=18.2\,^\circ \hbox {C}$, respectively, yielding a Rayleigh number $Ra=\alpha \,g\,\Delta T\,h^3 / (\nu \,\kappa )$ of about $Ra=10^6$ with the thermal expansion coefficient $\alpha$, acceleration due to gravity g, kinematic viscosity $\nu$, thermal diffusivity $\kappa$ and the temperature difference between heating and cooling plate $\Delta T$. Thus, the temperatures inside the Rayleigh–Bénard cell are in the range $T \in [T_{{\mathrm {c}}},T_{{\mathrm {h}}}]$. The temperature range is chosen in accordance to the characteristic of the TLCs, which should show a strong color variation with temperature for accurate temperature measurements (Moller et al. 2019). The typical convective timescale in terms of the free-fall time $t_f=\sqrt{h\,/\,(\alpha \,g\,\Delta T)}$ amounts to about $t_f=2.1$ s in this case and is therefore much larger than the thermal response time of the TLCs, which is in the order of milliseconds. Hence, the TLCs are well suited to determine the temperature field for this type of flow.

For the temperature measurements, the color of the TLCs must be related to their temperature, which is why a calibration measurement had to be performed initially. During the calibration measurements, 100 images of the TLCs in the measurement plane were recorded with the color camera at a frequency of $f=5$ Hz and an exposure time of $t_{{\mathrm {exp}}}=100$ ms for 19 different temperature levels in the range from $T=17.9\,^\circ$C up to $T=22.3\,^\circ$C in the cell. Therefore, all of the temperatures occurring in the cell during the convection measurement with $T_{\rm h}=21.5\,^\circ \hbox {C}$ and $T_{\rm c}=18.2\,^\circ \hbox {C}$ are covered in the calibration measurement. While recording the images, the TLCs in the whole cell were exposed to isothermal conditions by matching the temperatures of the heating and cooling plate, respectively. In order to ensure the uniform temperature distribution in the whole cell for each recording of the calibration, at least 30 min was waited after adjusting each temperature of the isothermal plates. Furthermore, the uniformity of temperature was strongly abetted by forced convection, which was induced by a handpump connected to a small hole in one of the sidewalls over a tube that is used for filling the water into the cell, too. The pressure applied to the handpump was equalized through a second hole in another sidewall, thereby enabling to mix the water in the cell. For each temperature step of the calibration, the temperature was measured with the temperature lance inside the Rayleigh–Bénard cell and compared to the temperatures measured with the sensors on the heating and cooling plate as well as on the inside of cell’s sidewall. In each case the deviations were less than 0.1 K considering all of the sensors, which confirms the uniform temperature distribution of the water inside the Rayleigh–Bénard cell during the calibration measurements. But if a temperature difference is adjusted to induce Rayleigh–Bénard convection and thus a heat flux from the heating to the cooling plate, it is a much greater challenge to keep the temperature of the plates constant. However, due to the large flow rate through the heating and cooling plate of about $\dot{V}_{\rm h}=\dot{V}_{\rm c}=30\,{\mathrm {l/min}}$ it was possible to limit the temperature variation of the heating and cooling water between the inlet and outlet of each plate to about 0.2 K. Since those temperature variations are further reduced by heat conduction across the plates, the isothermal boundary conditions can be considered as fulfilled in good approximation. Moreover, as indicated on the right side of Fig. 3, the measurements were not performed over the whole cross-sectional area, but over a central section of the cell, where the temperature distribution is most uniform due to the vanishing effect of the sidewalls. The central section for the measurements in this study is slightly trapezoidal due to the observation angle and the correction according to the Scheimpflug condition. Furthermore, the investigated section covers dimensions of about $l_x \times l_y = 350\,{\mathrm {mm}} \times 300$ mm, which corresponds to about $l_x \times l_y = 12.5\,{h} \times 10.7\,{h}$, yielding a suitable spatial resolution of about 0.13 mm/pixel for the study of the flow structures evolving in the experiment presented in Sect. 3.4.

Fig. 4

Snapshots of the TLCs R20C20W for three different temperatures. The yellow rectangles indicate the interrogation windows with a size of $32 \times 32$ pixels, which are considered regarding the color signal of the TLCs in dependency of the temperature. The TLCs are illuminated from the left side, which is indicated by the coordinate system corresponding to Fig. 1

3 Evaluation and results

3.1 Characterization of the color signal of TLCs

Fig. 5

Color appearance of the TLCs R20C20W in terms of the hue value H in dependency of the temperature for three interrogation windows in the field of view, which are marked in Fig. 4. For a better illustration, the colorbar on the right side shows the change of color appearance with hue. The red markers at the minimum of each curve indicate the onset of the investigated temperature range with unambiguous correlation to the color appearance of the TLCs, respectively

In order to perform the temperature calibration, the color signal of the TLCs in dependency of the temperature was investigated in detail. Based on this, it has been found that the color of TLCs is also considerably affected by the observation angle and accordingly by the position of the TLCs within the camera’s field of view. This can be seen in Fig. 4, which shows the TLCs over the whole field of view for three different temperatures. Besides the color change of the TLCs with temperature, a distinctive color trend from the left to the right end within each of the images can be seen despite the isothermal states, which has to be taken into account in the calibration of the temperature measurements. Here, the local color characteristic of the TLCs in dependency of the temperature is considered by splitting the whole field of view into different interrogation windows with each window having its own calibration curve. The single interrogation windows have dimensions of $32 \times 32$ pixel for all investigations in this study and are overlapped by 50%, resulting in $172 \times 150$ interrogation windows and as many calibration curves over the whole field of view. In order to demonstrate their differences, three exemplary interrogation windows, which are marked in Fig. 4, will be investigated with regard to their correlation between color and temperature of the TLCs, respectively.

For the characterization of the color of the TLCs in the single interrogation windows, the red (R), green (G) and blue (B) intensities of the pixels are used. In this case, each intensity is given for all pixels by an internal algorithm of the color camera with a Bayer filter. In order to remove reflections from the heating plate in the background, the minimum intensity of each pixel over the 100 recorded images at each temperature step is subtracted, respectively. Furthermore, intensity thresholds were used to exclude pixels which represent the dark background or agglomerated, very large and brightly scattering particles. The remaining intensities are averaged over the whole interrogation window, respectively. In the next step the mean values of the red, green and blue intensities are used to characterize the color of the TLCs in each interrogation window by means of the HSV-colorspace (Loesdau et al. 2014). In this colorspace the color shade of the TLCs is represented by the value of hue (H) in terms of an angle $H \in [0^\circ , 360^\circ ]$, which is normalized here, so that $H \in [0, 1]$. While $H=0$ stands for a red color shade, the color shade varies continuously over green ($H=1/3$) and blue ($H=2/3$) back to red again, which is also represented by $H=1$, resulting in a closed circuit (Schmeling et al. 2014). Therefore, hue values close to $H=0$ are obtained at the red start temperature of the TLCs, while the value of hue H increases to about $H=2/3$ toward the upper end of the active temperature range, where the TLCs are blue. However, it shall be noted again that the red start as well as the size of the active temperature range strongly depends on the position within the field of view as will be shown in the following. Furthermore, the saturation (S) indicates, if the color is rather pure ($S=1$) or has a high content of white light ($S=0$), while the value V quantifies the brightness of the signal by $V \in [0,1]$.

The HSV-colorspace and other similar models for the characterization of color, e.g., the HSI-colorspace (hue–saturation–intensity) and HSL-colorspace (hue–saturation–lightness), are well-established for liquid crystal thermography (Hiller et al. 1993; Dabiri 2008; Schmeling et al. 2014). Applying those colorspaces is of advantage, as the temperature of the TLCs can be determined by a single scalar in terms of hue $H \in [0, 1]$. Since the value of hue does neither depend on the seeding concentration of the TLCs nor on the intensity of their illumination, only hue is considered for the temperature calibration at first, yielding a very robust calibration method. However, if the intensity of illumination and the seeding concentration do not vary during the measurements, also S and V could be taken into account to increase the measurement range beyond the region with unambiguous correlation between hue and temperature. Even though also other combinations of the red, green and blue intensities could be considered to further improve the results, the authors mainly stick to the calibration using only the value of hue, due to its robustness, but also show some advanced calibration approaches in Sect. 3.3. For all of the interrogation windows and each temperature step adjusted in the calibration, the values of hue are averaged over the 100 recorded images in time t, resulting in a calibration curve $\langle H \rangle _t (T)$ for each interrogation window.

The exemplary calibration curves for the interrogation windows marked in Fig. 4 can be seen in Fig. 5. From these calibration curves it is obvious that the range of hue reaches from nearly $H=0$ for lower temperatures up to about $H=0.6$ for higher temperatures, corresponding to a color change from red to blue across the visible wavelength spectrum. However, the main color change of the TLCs from red to blue passes within a considerably smaller temperature range at the left side of the field of view compared to the right side. Therefore, larger angles between light sheet and the direction of observation by the camera according to Fig. 1 come along with a faster change of color with temperature. Furthermore, the red marked minimums of the curves, which represent the red start temperature of the TLCs, are shifted to higher temperatures when approaching the right side of the field of view, where the TLCs are recorded from smaller observation angles. In order to ensure an unambiguous correlation between the color and temperature of the TLCs, the red start temperature also determines the lower limit of the measurable temperature range for each interrogation window separately. Accordingly, the temperature measurements over the whole field of view are limited by the right end of the field of view in this case, where only temperatures $T>18.8\,^\circ$C can be measured due to unambiguity. Hence, the temperature of the heating and cooling plate in the Rayleigh–Bénard experiment presented in Sect. 3.4 was iteratively adapted such that the temperatures occurring within the measurement plane of the Rayleigh–Bénard cell start from about $T=18.8\,^\circ \hbox {C}$. The color of the TLCs at the right side of the field of view served as an indicator for that, since those become nearly invisible due to the drastically decreasing intensity of the light reflected by the TLCs when the temperature is lower than the local red start temperature. Moreover, as already addressed in the previous section, the observation angle was adjusted such that the TLCs show a sufficient color play over the whole temperature range of interest in the main part of the field of view, which is necessary for a low uncertainty of the temperature measurements. Although the color of the TLCs at the left side of the field of view changes very fast with temperature from red to blue, the local calibration curves within this area are still suitable for the temperature measurements presented in Sect. 3.4, since the temperatures mainly range from about $T=18.8\,^\circ \hbox {C}$ up to $T=20.8\,^\circ \hbox {C}$. In Sect. 3.2.2 it will be shown that the temperatures can be measured with a low uncertainty of measurement within this temperature range.

3.2 Calibration of temperature measurements

3.2.1 Calibration methods

Fig. 6

Schematic illustration of three different methods for the calculation of the temperature of TLCs based on the value of hue by means of a neural network (a), a system matrix (b) and linear interpolation of hue in the local calibration curves (c)

For the determination of the temperature of the TLCs via their color signal and their position within the field of view, three methods, which are schematically depicted in Fig. 6, are considered in the following. These methods have in common that the temperatures are calculated on the basis of the time-averaged calibration curves of the corresponding interrogation window, as exemplarily shown in Fig. 5 for three different interrogation windows, enabling a direct comparison of the results.

The first method shown in Fig. 6a is based on a neural network, which enables to determine the temperature T(x, y) via the hue value H of the TLCs and the position $\mathbf {X}=(x,y)$ of the center of the corresponding interrogation window within the field of view given in pixels. Therefore, this neural network transforms three input parameters, namely x, y and H into one output parameter, which is the temperature T. In general, neural networks incorporate different neurons arranged in so-called hidden layers, which transform the input parameters of the input layer to the output parameters of the output layer. In this case, the transformations are performed via the Hyperbolic tangent sigmoid transfer function, as this allows the network to learn nonlinear relationships between the input and output of each neuron. However, a pure linear transfer function is used for the output layer itself, as commonly for function approximation applications such as in the present case (Beale et al. 2019). The structure depicted in Fig. 6a consisting of 10 neurons in six hidden layers, respectively, enables accurate temperature measurements, while keeping the computation time still within reasonable limits, shown in the following section. Therefore, this structure was used for the following exemplary temperature measurements in Rayleigh–Bénard convection. However, several neural networks with different structures regarding the number of neurons and hidden layers have been designed and trained using the Deep Learning Toolbox of MATLAB R2019a. In this context the network structures and also various training algorithms were tested with respect to the resulting uncertainty of temperature measurement and the computation time, which will be presented in this study. For each combination of the network structures and training algorithms five training runs were performed. In each of the runs, 80% of the input data were used for training, while the remaining 20% were used for testing the trained neural network with respect to the target values, which are the known temperatures of the calibration measurements. In order to prevent the effect of overfitting, which will be discussed in Sect. 3.2.3, the data for the training and testing were chosen randomly by the algorithm, respectively.

Another method which can be used for the calibration of temperature measurements is shown in Fig. 6b. When applying this method, a separate vector $\mathbf {C\,(X)} = (1,C_H,C_{H^2},C_{H^3},C_{H^4})$ for each interrogation window is determined to transform a given system matrix based on the measured hue values to the known temperatures of the calibration as accurately as possible. The system matrix includes the hue values from the first up to the forth order and an offset, so that the shape of the calibration curves can be described mathematically with a low remaining deviation.

Furthermore, a third and last calibration method according to Fig. 6c is investigated in the scope of this study. This method uses the local calibration curves, i.e., one calibration curve for each interrogation window, in order to calculate the respective temperature on the basis of the measured hue value within the corresponding interrogation window by means of linear interpolation.

3.2.2 Quantitative assessment

Fig. 7

The mean absolute deviation (MAD) and standard deviation ($\sigma _{T_{{\mathrm {calc}}}}$) in dependency of the temperature for a single hidden layer (a, b) with a varying number of neurons and multiple hidden layers (c, d)

In order to assess these methods with respect to the error of temperature measurement, each method is applied to calculate the temperatures of the calibration measurements for all interrogation windows of the 100 images recorded during calibration for the different temperature levels. However, the following investigations are all restricted to the temperature range from $T=18.8\,^\circ \hbox {C}$ upward due to unambiguity of the temperature measurements, as outlined in Sect. 3.1. Since the temperature is a known input parameter in the calibration, the deviation of the calculated temperatures $T_{{\mathrm {calc}}}$ to the set values can be estimated to determine the mean absolute deviation (MAD), which is ${\langle |T_{{\mathrm {calc}}}-T |\rangle |_{T_i}}$, while the random error can be characterized by the standard deviation $\sigma _{T_{{\mathrm {calc}}}}|_{T_i}$. In order to compare the MAD and $\sigma _{T_{{\mathrm {calc}}}}$ for different network structures only one training algorithm was used to ensure the comparability. For this, the Levenberg–Marquardt algorithm (Levenberg 1944; Marquardt 1963) was chosen, since this is one of the most frequently used algorithms for numerical optimizations, which holds a lot of promise to provide good results due to its stable convergence (Yu and Wilamowski 2011). The MAD and $\sigma _{T_{{\mathrm {calc}}}}$ as a function of the temperature level were calculated for all of the five training runs for each network structure, respectively. For the comparison of the results obtained by using the neural networks with the results of the other two calibration approaches shown in Fig. 6, the best of the five training runs with regard to the mean absolute deviation over all temperature levels of the calibration ($\overline{{\mathrm {MAD}}}$) was chosen for each of the network structures, respectively. The type of the structure is denoted as a listing of $n_i$ in the following, with n representing the number of neurons in the corresponding hidden layer with index i. Thus, for example a structure with three hidden layers containing 10 neurons, respectively, is specified as “$n_1 - 10,n_2 - 10,n_3 - 10$.” However, the simple case of one single hidden layer for a varying number of neurons $n_1$ was tested at first. The resulting values for the MAD and $\sigma _{T_{{\mathrm {calc}}}}$ for six different numbers of neurons can be seen in Fig. 7a, b. When using only one neuron, these values reach up to about 0.5 K in maximum, which does not allow for accurate temperature measurements. But when increasing the number of neurons to $n_1 \ge 5$, the MAD and $\sigma _{T_{{\mathrm {calc}}}}$ decrease considerably. In particular for the lower part of the shown temperature range of $T \le 20\,^\circ$C the mean absolute deviation is in the range of about 0.1 K or even less, while the standard deviation does not exceed $\sigma _{T_{{\mathrm {calc}}}}=0.2\,{\text K}$ with exception from the lowest temperature of $T=18.8\,^\circ \hbox {C}$. However, for larger temperatures the measurements are in general more erroneous again, as the mean absolute deviation and the standard deviation reach up to about ${\mathrm {MAD}}=0.2\,{\text{K}}$ and $\sigma _{T_{{\mathrm {calc}}}}=0.3\,{\text{K}}$. This fall-off in quality of the temperature measurements is a result of the color characteristic of the TLCs according to Fig. 5, which shows that the change of color decreases with increasing temperature for each position within the field of view. Therefore, the temperatures cannot be measured as precisely anymore when approaching the upper end of the active temperature range, also shown by Moller et al. (2019).

For estimating the effect of the number of hidden layers, the case of $n_1=10$ was considered and a second, third as well as a fourth, fifth and sixth hidden layer with 10 neurons was added successively. The results for the MAD and $\sigma _{T_{{\mathrm {calc}}}}$ in Fig. 7c, d show that adding further layers leads to better results in each case, especially for the lowest temperatures. However, Fig. 7a–d shows that with increasing number of neurons or hidden layers the improvements become progressively smaller. Locally even worsening can be found, for example when comparing the results for $n_1-10{,}\,n_2-10$ and $n_1-10{,}\,n_2-10{,}\,n_3-10$. Furthermore, the structure of the neural networks with three and four hidden layers was also modified by increasing the number of neurons in single layers, thereby testing many different combinations, like for example $n_1-30{,}\,n_2-20{,}\,n_3-10$ or $n_1-40{,}\,n_2-30{,}\,n_3-20{,}\,n_4-10$. For the sake of clarity, the results for those additional variations of the network structure are not depicted, as considerable improvements could not be achieved anymore. In summary, the network structure with 6 hidden layers containing 10 neurons, respectively, provides the best results of all the investigated network structures for using the Levenberg–Marquardt algorithm. Hence, this network structure was chosen for the comparison of the results for all the different training algorithms, which are available in the Deep Learning Toolbox of MATLAB R2019a.

However, in order to maintain the overview of those results, the MAD and $\sigma _{T_{{\mathrm {calc}}}}$ cannot be discussed in dependency of the temperature level of the calibration measurements for each training algorithm. Therefore, the average of the mean absolute deviation ($\overline{{\mathrm {MAD}}}$) and standard deviation ($\overline{\sigma _{T_{{\mathrm {calc}}}}}$) over the whole temperature range are given in Table 1 for the best training run, respectively. At this point it should be noted that the differences between the results of the five training runs for each combination of training algorithm and network structure are negligible, so that the listed results represent all the five training runs in good approximation. Furthermore, the computation time summed up over all of the five training runs can be seen in the last column of Table 1. From this table it is obvious that the results for most of the training algorithms are comparable to those obtained with the Levenberg–Marquardt algorithm, if the same network structure is used with 10 neurons in 6 hidden layers, respectively. For these network structures, the computation time over all five training runs ranges between $20\,\mathrm {min}< \sum \,t_{{\mathrm {comp}}} < 01\,\mathrm {h}\,\,30\,\mathrm {min}$. Three training algorithms, namely Gradient Descent, Gradient Descent with Momentum and Variable Learning Rate Gradient Descent show an outstandingly short computation time of only a few minutes or even seconds for this network structure, however, at the cost of considerably higher values for the mean absolute deviation and standard deviation. Furthermore, the results for the Levenberg–Marquardt algorithm applied to different network structures are also listed in Table 1. Based on the results shown in Fig. 7, it could already be expected that the values for $\overline{{\mathrm {MAD}}}$ and $\overline{\sigma _{T_{{\mathrm {calc}}}}}$ tend to decrease, if the number of neurons and hidden layers is increased, which is proven in the table. However, the fact that the computation time for the depicted net configurations ranges from a few seconds up to several hours shows that the number of neurons and hidden layers should in general not be enhanced arbitrary, but it should be taken into account, if there is a considerable effect on the error of temperature measurement justifying a large rise in computation time. Concluding it can be stated that similar results can be achieved when using different combinations of training algorithms and network structures, but the network with 6 hidden layers containing 10 neurons, respectively, trained with the Levenberg–Marquardt algorithm yields the best results. Therefore, it was chosen for all the following investigations.

Moreover, the results for the other two calibration methods according to Fig. 6 are also listed in the table. It can be seen that using the linear interpolation of the hue values in local calibration curves results in similar values for $\overline{{\mathrm {MAD}}}$ and $\overline{\sigma _{T_{{\mathrm {calc}}}}}$ compared to the chosen, best performing neural network. If the system matrix is applied, the $\overline{{\mathrm {MAD}}}$ and $\overline{\sigma _{T_{{\mathrm {calc}}}}}$ are larger, showing that this calibration method is not as suitable as the other two methods. However, when using the system matrix approach, the time interval required to compute the correlation between the hue values and temperature for different positions within the field of view is about 0.1 s, which is very small compared to the computation time for the training of the chosen neural network approach. In the case of applying linear interpolation of the hue values, this correlation does not even have to be computed, as the calibration curves are given by the measurement a priori.

Table 1 Mean absolute deviation ($\overline{{\mathrm {MAD}}}$) and standard deviation ($\overline{\sigma _{T_{{\mathrm {calc}}}}}$) of the calculated temperatures over the whole temperature range from $T=18.8\,^\circ \hbox {C}$ to $T=22.3\,^\circ$C for the best training run applying different training algorithms and network structures for the neural network. The notation of the network structure in the second column specifies the number of neurons in each of the hidden layers starting from the first one. The entire computation time for the training summarized over all the five runs is also registered in the right column, respectively. In addition, the results for the other two calibration approaches are listed as well for comparison

For a better imagination of the results shown in Fig. 7 and Table 1, exemplary instantaneous temperature fields of the calibration for $T=19.0\,^\circ$C and $T=19.6\,^\circ$C, which were calculated by using the neural network with the chosen structure, are depicted in Fig. 8a, b. With regard to a reliable calibration of the temperature measurements, the calculated temperatures should be as narrowly distributed around the corresponding temperature level of the calibration as possible, respectively. In Fig. 8a it can be seen that the isothermal state with a temperature of $T=19.0\,^\circ$C can be reproduced reliably with except from a few distinctive outliers at the left end of the field of view. These outliers arise due to the fact that the corresponding local calibration curves do not show a sufficient slope at this temperature anymore, which leads to a significant increase of the uncertainty of temperature measurement. Hence, also the temperature level of $T=19.6\,^\circ$C was checked, since the calibration curves at the left end of the field of view are almost very flat at this temperature level, compare Fig. 5. Even though the number of outliers increases for $T=19.6\,^\circ$C in Fig. 8b, the performance of the calibration is still good in the main part of the field of view, as nearly all of the distinctive outliers are located at its very left end. However, those outliers downgrade the quality of the temperature measurements regarding the whole field of view in terms of the MAD and $\sigma _{T_{{\mathrm {calc}}}}$ delusively. Therefore, these outliers were removed from each temperature field of the calibration by applying a global outlier filter. The global outlier filter eliminates the values that do not comply with the following condition (1)

$$ \tilde{T}_{x,y,t}-2\cdot \sigma _{T_{x,y,t}}< T_{x,y,t} < \tilde{T}_{x,y,t}+2\cdot \sigma _{T_{x,y,t}}, $$

(1)

where $\tilde{T}_{x,y,t}$ represents the median temperature of each temperature step of the calibration over x, y as well as the 100 steps in time t, while the standard deviation of the temperature at the corresponding temperature step is denoted by $\sigma _{T_{x,y,t}}$.

Fig. 8

Raw temperature field of the calibration measurement for $T=19.0\,^\circ$C and $T=19.6\,^\circ$C (a, b) when using the neural network according to Fig. 6a and the corresponding temperature fields after outlier removal (c, d)

Here, the most distinctive outliers can be detected by this kind of filter reliably and, therefore, no other approaches such as universal outlier detection (Westerweel and Scarano 2005) are necessary. Applying the global outlier filter to the exemplary calculated temperature field of the calibration for $T=19.0\,^\circ$C and $T=19.6\,^\circ \hbox {C}$ in Fig. 8a, b results in the temperature fields shown in Fig. 8c, d, in which the outliers are replaced by linearly interpolated temperatures of the adjacent interrogation windows. In order to compare the results of the three calibration methods outlined above, the respective distributions of the mean absolute deviation are depicted in Fig. 9 in dependency of the x-coordinate and the temperature level. In order to distinguish this local mean absolute deviation from MAD, which incorporates each absolute deviation over the coordinates x, y and time t for the averaging, it is denoted as ${\mathrm {MAD}}_x$. For the determination of ${\mathrm {MAD}}_x$ the averaging of the absolute deviations was only performed in y-direction of the calculated outlier-cleaned temperature fields and over the 100 recordings in time for each temperature level, since this enables to evaluate the local variation of the mean absolute deviation, which mainly occurs in x-direction due to the varying angle between illumination and observation. It can be seen that using the neural network or the linear interpolation of the hue values in the local calibration curves results in similar distributions of the mean absolute deviation ${\mathrm {MAD}}_x$. Based on Fig. 8 it could already be expected that the deviations are the largest for higher temperatures in the left part of the field of view due to the diminishing color change of the TLCs, which is proven in Fig. 9. However, in the temperature range $18.8\,^\circ \,{\mathrm {C}} \le T \le 20.8\,^\circ {\mathrm {C}}$, which is of interest for the temperature measurements presented in this study, the local mean absolute deviation does never considerably exceed ${\mathrm {MAD}}_x = 0.2\,{\text K}$ when the neural network or the linear interpolation approach is applied, thereby confirming the suitability of the measuring technique for temperature measurements over the whole field of view. From Fig. 9 it is also obvious that the isothermal states of the calibration measurements cannot be reproduced as accurately by the system matrix in most cases. When applying the system matrix, larger deviations already occur in the lower temperature range, which is why the other two calibration methods are better suited for this application.

Fig. 9

The local mean absolute deviation ${\mathrm {MAD}}_x$ of the temperature measurements in dependency of the x-coordinate and of the temperature level for the threee different calibration methods according to Fig. 6

Fig. 10

The mean absolute deviation (MAD) and standard deviation ($\sigma _{T_{{\mathrm {calc}}}}$) in dependency of the temperature for using different calibration methods according to Fig. 6

The results for the absolute deviations averaged over the whole field of view and over all time steps of the recordings based on the data including outlier removal and the standard deviations are depicted in Fig. 10 in dependency of the temperature levels of the calibration for the three calibration methods. Furthermore, the results based on the raw data of the neural network are also plotted for comparison. Therefore, the effect of the outlier filter can be evaluated for the whole temperature range. As it is already obvious from Fig. 9, it also becomes apparent in Fig. 10 that the best results can mostly be achieved when using the neural network or the method based on linear interpolation. In particular for the interval $18.8\,^\circ {\mathrm {C}} \le T \le 20.8\,^\circ {\mathrm {C}}$, in which the temperatures of the measurements presented in Sect. 3.4 mainly range, the MAD and $\sigma _{T_{{\mathrm {calc}}}}$ do never exceed 0.17 K when the neural network or linear interpolation of the hue values is used for the calculation of the temperature field. Even if the outliers are not removed, both the MAD and $\sigma _{T_{{\mathrm {calc}}}}$ do never considerably exceed 0.2 K within this temperature interval, which also confirms the suitability of both of these methods for precise temperature measurements via the color of TLCs. The advantage of using those two methods is proven when comparing the average of the mean absolute deviation $\overline{{\mathrm {MAD}}}$ and of the standard deviation $\overline{\sigma _{T_{{\mathrm {calc}}}}}\,$, respectively. In consideration of the outliers at $T=18.8\,^\circ \hbox {C}$, those values amount to about $\overline{{\mathrm {MAD}}}|_{{\mathrm {sm}}}=0.166\,{\text{K}}$ and $\overline{\sigma _{T_{{\mathrm {calc}}}}}|_{{\mathrm {sm}}}=0.234\, {\text K}$ for the system matrix, while the use of the neural network results in $\overline{{\mathrm {MAD}}}|_{{\mathrm {nn}}}=0.086\,{\text{K}}$ and $\overline{\sigma _{T_{{\mathrm {calc}}}}}|_{{\mathrm {nn}}}=0.110\,{\text{K}},$ which are very close to $\overline{{\mathrm {MAD}}}|_{{\mathrm {li}}}=0.090\,{\text{K}}$ and $\overline{\sigma _{T_{{\mathrm {calc}}}}}|_{{\mathrm {li}}}=0.110\,\hbox {K}$ for the linear interpolation of the hue values.

3.2.3 Discussion of the calibration methods

As the quantitative assessment of the calibration methods has shown, there are some differences regarding the resulting error of the temperature measurement. It should be noted that those results concerning the mean absolute deviation and the standard deviation originate from the specific application presented in this study and, therefore, cannot be transferred one-to-one to any other application. However, the methods themselves are universal, such that some general conclusions about their applicability for temperature measurements with TLCs can be drawn.

The previous investigations have shown that using the system matrix leads to the largest error of temperature measurement in this case. This is due to the fact that the functional correlation between hue and temperature is not known a priori, but it must be defined somehow to determine the coefficients for the calculation of the temperature based on the hue values in the system matrix. If an inappropriate type of function is chosen, the coefficients cannot be determined such that the temperatures are calculated correctly over the whole temperature range. Here, some polynomial functions with different orders were tested and the fourth-order polynomial has proven to be the most suitable one in this case. However, in general a compromise must be found to prevent overfitting caused by too high polynomial orders, while the order must still be high enough to represent complex functions. A big advantage of using the system matrix is that also other functional approaches could be checked as well as additional variables like for example the saturation S and the value V could be implemented easily, which might improve the results considerably (Segura et al. 2015). For the current application hue was chosen to be the only temperature-dependent variable in the system matrix to circumvent problems caused by a varying particle image density during the measurements. Furthermore, when observing an extended field of view with a large range of observation angles, different functional approaches should be used to cope with the variety of the correlation between hue and temperature, which complicates the calibration using a system matrix. However, if only a small field of view must be investigated and only one functional approach is necessary, different types of functions aiming for a minimum error of temperature measurement can be tested effortlessly, so that this calibration method can also be applied successfully.

The investigations in Sect. 3.2.2 have already shown that the other two calibration methods are well suited for temperature measurements via the color of TLCs over a large field of view. Even though the method based on linear interpolation of hue in local calibration curves is very straightforward, it is a very robust and reliable one, as no deviations are caused by numerical optimization or curve fitting. However, a sufficient number of sampling points for each calibration curve is necessary to ensure that a linear approximation between two neighboring points is justified and the temperatures are determined correctly. Moreover, using this method it must be taken care of local outliers within the calibration curves very accurately, as these outliers directly result in large measurement errors for the corresponding interrogation window. In comparison, the other methods can handle local outliers to a certain degree, since their effect is reduced by the curve fitting.

Concerning the central method of investigation, which is the neural network, it can be stated that it holds a lot of promise, as this method provides good results regarding the MAD and $\sigma _{T_{{\mathrm {calc}}}}$ without the necessity to specify functional correlations between hue and temperature, contrary to the system matrix. But precisely for this reason, this method carries the risk of overfitting, meaning that too many variables are internally defined to adapt the calculation model to the set of the training data, which might make the model prone to errors for the real measurements. In general, this problem of overfitting is more likely to appear if only a low amount of training data is available and there are strong outliers in that set of data. One way to detect overfitting is to train several nets based on the training data and to compare the MAD and standard deviation when applied to another set of test data. If the different nets result in considerably varying values for the MAD and standard deviation, there might be the problem of overfitting. Therefore, a common way to circumvent overfitting is to apply a method called “dropout” (Srivastava et al. 2014). Using this method, some randomly chosen neurons are ignored in each iteration of the training, thereby preventing any neuron from relying excessively on the output of any other neuron (Baldi and Sadowski 2014) and making the neural network more robust. In addition, also the effect of applying different methods of training on the result can be evaluated. Even though there are further methods to improve the results obtained by neural networks, no specific methods were used here, in order to demonstrate that neural networks can easily be applied to achieve reliable results. However, several network structures were trained with different training algorithms over five training runs, yielding similar results in each case, which confirms that overfitting is not present in this case. Therefore, based on the previous investigations the neural network and the method that uses linear interpolation of hue in the local calibration curves have been proven to work robustly for temperature measurements over a large field of view.

In addition to the computation time for the training of the calibration methods given in Table 1, it is finally referred to the rapid computation of the temperature fields when applying those methods. Averaged over a set of 100 exemplary temperature fields with each having $172 \times 150$ interrogation windows, for the computation of one single temperature field on the basis of the given hue field the linear interpolation of the hue values takes 0.015 s, while the neural network takes 0.041 s and the system matrix is the fastest approach with only about $350\, {\upmu}{\text s}$, which further confirms the applicability of those methods.

3.3 Advanced calibration with neural networks

The previous results obtained with the neural networks are all based on a network training, which incorporates the location of the interrogation windows and the hue values as input parameters, while the corresponding known temperature levels of the calibration measurements are the target values. As already outlined above, solely the hue value was chosen as information about the color appearance of the TLCs so far, since this parameter shows the most distinctive and robust dependency of the temperature. However, the saturation S and value V of the HSV-colorspace also vary with temperature and, therefore, can be used for the temperature measurements. The dependency of hue, saturation and value of the temperature is depicted in Fig. 11a–c for the three interrogations windows, which were already chosen for the investigations in Sect. 3.1 and are marked in Fig. 4. In order to calculate H, S and V based on the red, green and blue values of the RGB-colorspace (Loesdau et al. 2014), those values were normalized by the maximum occurring intensity $max\,(R,G,B)$ of the recordings of the calibration measurement, such that $R, G, B \in [0,1]$. The normalized temperature-dependent red, green and blue values can be seen in Fig. 11d–f for comparison.

Fig. 11

The time-averaged values of H, S, V (a, b, c) and R, G, B (d, e, f) in dependency of the temperature for the three interrogation windows marked in Fig. 4, respectively

From Fig. 11b, c it is obvious that the saturation and value do not show a clear unambiguous trend contrary to hue, but also hold some valuable information about the temperature. For example, the variation of the value V on the left side of the field of view in the temperature range from about $19.2\,^\circ \mathrm {C} \le T \le 21.4\,^\circ$C, where hue does only vary to a small extent, results in an improvement of the temperature measurements. This can be seen in Fig. 12a, which shows the differences of the mean absolute deviation when considering H, S and V for the training in comparison with the reference case represented by the outlier-cleaned results of the neural network, which is only trained with the hue value. In particular for temperatures $T > 20\,^\circ$C the mean absolute deviation decreases up to nearly 0.05 K corresponding to a reduction of about 30%. Moreover, a neural network training was performed with H and S besides the locations of the interrogation windows as input data, which also led to an improvement compared to the calibration only with hue. However, as expected the reduction of the mean absolute deviation is smaller in this case, due to the excluding of the information about the temperature contained in the value V.

Furthermore, using the red, green and blue values for the training of the neural networks was tested. While the use of the absolute values of R, G and B yields similar results compared to H, S and V, a further approach incorporating the ratios of R/G and R/B is not as good, but also better than the calibration only with hue. Hence, the calibration approaches, which directly incorporate the intensity of the light reflected by the TLCs, i.e., the red, green and blue values of the RGB-colorspace or the value V of the HSV-colorspace, are most successful. Applying those approaches the strength of neural networks to make use of all the valuable information of the input data takes effect and results in a lower measurement error. However, when such a calibration approach is applied, it must be taken care that the intensity of the recorded light in the application of this measuring technique matches that of the calibration measurements. If considerable differences regarding the intensity are present, e.g., due to a varying seeding concentration, only hue, saturation and ratios of the red, green and blue values should be taken into account. With regard to the application presented in this study, this point is of importance, since the flow structures evolving in Rayleigh–Bénard convection have to be investigated over time spans of many hours to understand their reorganization in greater detail. Although the TLC particles are nearly neutral with respect to buoyancy, the absolute intensities within those extended time spans are affected by sedimentation and, therefore, should not be considered in this case.

Fig. 12

The variation of the mean absolute deviation (MAD) and of the local mean absolute deviation averaged over the temperature levels of the calibration ($\overline{{\mathrm {MAD}}_x}$) when applying the advanced calibration methods in comparison to the reference case represented by the outlier-cleaned results of the neural network, which only uses hue for the calibration

From the previous investigations the question arises, in which part of the field of view the main improvements of the temperature measurements are located. In order to clarify this point, the local mean absolute deviations shown in Fig. 9a, which are based on the outlier-cleaned data obtained with the neural network trained only with hue, have to be considered. Those deviations have to be averaged over the axis of ordinate of Fig. 9a, i.e., over the different temperature levels of the calibration, and the resulting average values $\overline{{\mathrm {MAD}}_x}|_{{\mathrm {ref}}}$ must be subtracted from the analogously determined average values $\overline{{\mathrm {MAD}}_x}$ for using the advanced calibration approaches. The results in Fig. 12b indicate that the main improvements are located on the left side of the field of view, which is reasonable, since the hue value does not suffice to precisely measure the highest temperatures of the investigated temperature range in this part of the field of view.

3.4 Temperature field measurements in Rayleigh–Bénard convection

The applicability of TLCs for temperature field measurements is demonstrated by means of an exemplary measurement in the presented Rayleigh–Bénard cell with a heating and cooling plate temperature of ${T_{\rm h}=21.5\,^\circ }\hbox {C}$ and ${T_{\rm c}=18.2\,^\circ }\hbox {C}$, respectively. An instantaneous image of the TLCs in a horizontal plane in close vicinity to the cooling plate of the cell, which clearly shows the evolving flow structures, is depicted in Fig. 13a. These flow structures organize themselves in patterns, which evolve over the whole cross-sectional area with different size and shape. The color of the TLCs already reveals their emergence without knowing the temperature and velocity field, since there is only one reasonable explanation from the physical point of view, which matches those fields and also complies with the principle of continuity. Warmer fluid with decreased density streams upward in the center of each pattern, flows outward and, therefore, parallel to the cooling plate in its close vicinity, hits the fluid of an adjacent pattern and flows down again with decreased temperature resulting from the heat loss to the cooling plate. However, for further investigations on the nature of Rayleigh–Bénard convection the temperature field must be analyzed quantitatively, especially with regard to the local heat flux.

Fig. 13

Instantaneous image of the TLCs (a) and the corresponding hue field (b) as well as the temperature field calculated with the neural network (c) and linear interpolation of the hue values in the local calibration curves (d). The image of the TLCs was recorded in close vicinity to the cooling plate with a temperature of $T_{\rm c}=18.2\,^\circ \hbox {C}$, while the temperature of the heating plate was adjusted to $T_{\rm h}=21.5\,^\circ \hbox {C}$. The position of the field of view within the Rayleigh–Bénard cell can be seen in Fig. 3

In order to determine the temperature field based on the color of the TLCs, the image is initially split into interrogation windows of $32\times 32$ pixels corresponding to about 4 mm $\times$ 4 mm in size. For each interrogation window, the color is characterized in terms of hue, leading to the hue field H(x, y) shown in Fig. 13b. This field clearly demonstrates the trend of hue from the left to the right end, which is not only caused by temperature differences, but also by the variation of the observation angle within the camera’s field of view. Thus, the need for a calibration approach taking this effect into account is confirmed. As shown above, the neural networks and the method that uses linear interpolation of the hue values in the local calibration curves can cope with this effect. Hence, both of those methods are applied to determine the corresponding temperature field depicted in Fig. 13c, d. In order to show that the measuring technique works despite a varying seeding concentration, the seeding was increased for the convection measurement. Therefore, the intensity of the light reflected by the TLCs in terms of the value V could not be taken into account for the temperature measurements as discussed in the previous section and the network trained with the hue value and saturation was chosen for the calculation of the temperature field.

In the temperature fields the small-scale convective flow patterns can clearly be retraced again in accordance with the image of the TLCs in Fig. 13a and the hue field in Fig. 13b. Furthermore, it can be seen that the temperature is in average warmer in the left half of the field of view, enabling to draw conclusions about larger, superimposed flow structures. It can be stated that the left half is rather a source of fluid, since the fluid flows up due to higher temperatures, while the right half in summary represents a sink of colder, downwelling fluid. These larger structures, which are called turbulent superstructures, relocate over time and strongly affect the heat flux between the heating and cooling plate, which is why they are in general of great interest. However, it is referred to Pandey et al. (2018) and Fonda et al. (2019) for further details regarding turbulent superstructures in Rayleigh–Bénard convection and to Bodenschatz et al. (2000), du Puits et al. (2007) as well as Chillà and Schumacher (2012) for general insights into Rayleigh–Bénard convection, as the present study is focused on the temperature measurement technique itself. Moreover, a close match can be found from the temperature fields in Fig. 13c, d. In particular on the right side of the field of view, differences between those two temperature fields can hardly be seen by eye. On the left side some differences become apparent, especially in the regions with the highest temperatures, at which the corresponding local calibration function of the hue value is already flat and the highest uncertainty of measurement is expected. However, overall the absolute deviations of the temperature fields are small and amount to about 0.1 K in average, confirming the achieved results and the applicability of those two calibration techniques for temperature field measurements using TLCs.

4 Conclusion

In this study the use of TLCs for temperature field measurements was investigated. In this context it was shown that the color signal of TLCs used for the determination of the temperature strongly depends on the position within the field of view of a color camera. Three different calibration techniques, which take this effect into account, were analyzed with respect to the resulting error of temperature measurement. Here, the focus was on testing neural networks. For this purpose, different structures of the neural network as well as different training algorithms were investigated and the computational time required for the training of the networks was compared. A neural network consisting of 10 neurons in 6 hidden layers, respectively, trained with the Levenberg–Marquardt algorithm has proven to be most successful in this case. Using this structure of the neural network, the mean absolute deviation as well as the mean standard deviation over a temperature range of $3.5\,\hbox {K}$ amount to about $0.1\,\hbox {K}$ for instantaneous measurements with a spatial resolution of about $4\,\hbox {mm}$ over a large field of view with dimensions of about $350\,\mathrm {mm} \times 300\,\mathrm {mm}$. Thus, the dynamic temperature range (DTR) and dynamic spatial range (DSR) are ${\mathrm {DTR}} = 3.5\,\mathrm {K} / 0.1\,\mathrm {K} = 35$ and ${\mathrm {DSR}} = 300\,\mathrm {mm} / 4\,\mathrm {mm} = 75$. The comparison to the results obtained with the other two calibration techniques shows that using neural networks holds a lot of promise. The results are comparable to those of the method, which is based on linear interpolation of the hue values in local calibration curves and has already been used successfully for previous measurements (Moller et al. 2019).

The present study also shows that larger uncertainties appear for all calibration methods in the temperature range where the calibration curves get flatter, i.e., for higher temperatures. If a lower uncertainty is also necessary for higher temperatures, the observation angle could be changed, such that the color of the TLCs varies to a considerable extent over the whole required temperature range. However, if the observation angle cannot be changed, using another type of TLCs with differing specifications could be considered, too.

Furthermore, the investigations have shown that the computational time for the training of neural networks varies significantly depending on the structure of the network and its training algorithm. Therefore, a suitable trade-off between the measurement error, which is affected by the structure of the neural network as well as by its training algorithm, and the computational time must be found for each application. In the present study it was demonstrated that also a very simple neural network, e.g., with 10 neurons in a single hidden layer, might be useful, since the mean absolute deviation and mean standard deviation of the temperature measurement are smaller than 0.2 K in this case. The training of this simple neural network, which was performed with the Levenberg–Marquardt algorithm, took about 1 min, confirming the potential of neural networks to be trained very efficiently. Moreover, the application of various training algorithms has led to similar results regarding the error of temperature measurement, which also confirms the roughness of this calibration method. At this point it shall be noted that neural networks particularly demonstrate their strengths when applied to very complex problems based on a large amount of training data. Therefore, neural networks are also well suited for problems with many input parameters, whereas another simple calculation model may reach its limits. For this reason, advanced calibration approaches taking into account not only hue, but also the saturation and the value or the red, green and blue values were tested. Especially for the largest temperatures of the calibration measurement, the mean absolute deviations have been reduced by up to 30%, proving the strength of neural networks to extract all the valuable information of the input data.

For all the investigations concerning neural networks, the Deep Learning Toolbox of MATLAB R2019a was used. This toolbox allows to adjust the settings for neural networks comfortably, enabling to achieve reliable results by checking different combinations of the settings. Therefore, without advanced knowledge in the field of neural networks those can be designed and applied easily, which indicates an increased use of neural networks in the future. In order to demonstrate the applicability of neural networks for temperature measurements via the evaluation of the color of TLCs, this technique was finally used for some exemplary measurements in a horizontal plane of a Rayleigh–Bénard cell. The results clearly show that the corresponding temperature field can be determined precisely with high spatial resolution by means of the color of TLCs within a large field of view when applying neural networks. Furthermore, the results are also confirmed by opposing the temperature field calculated with the proven calibration technique based on linear interpolation of the hue values in local calibration curves.

5 Outlook

In the near future this measurement technique will be applied for simultaneous measurements of the velocity and temperature field in the presented Rayleigh–Bénard cell, in order to study the turbulent superstructures in dependency of the Rayleigh number for different aspect ratios. The velocity field will be determined by particle image velocimetry (Westerweel 1997; Raffel et al. 2018), also using the TLCs as tracer particles for the evaluation of the velocity field. In order to be able to investigate the vertical component of the velocity as well, stereoscopic PIV measurements will be performed by applying two additional monochrome cameras. Thus, the perspective error affecting the estimation of the horizontal velocity components in the case of particle motion in the vertical direction is corrected, too (Kästner et al. 2018; Cierpka et al. 2019). Furthermore, due to the versatile applicability of neural networks it is planned to use them in combination with other measurement techniques, e.g., with two-color laser-induced fluorescence (Sakakibara and Adrian 2004) and luminescent two-color tracer particles (Massing et al. 2016), enabling a comparison of the obtained results.