1 Introduction

Experimental studies of failure of energy intensive objects such as lithium-ion batteries are becoming more widely used to understand the consequences of failure which can lead to combustion events [1,2,3]. These experiments provide an effective method of measuring temperature, pressure, off-gassing, chemical composition, and the use of visual imaging to attempt to study flame flares is becoming more widespread [4, 5]. However, as with most combustion phenomena, visualization of a flame is helpful to try and interpret its behaviour and possible quantification of heat release rate, if the images obtained from the experimental setup can be processed and calibrated correctly [6,7,8,9].

Digital images can be used to make dimensional measurements and when recorded in video format these can be made as a function of time. This can be extremely beneficial when measuring spatially or temporally rapidly changing events, where there is insufficient time to make measurements or the position where the intended measurement is being taken changes, e.g.: unsteady flames, hazardous events where it is not safe to get close enough to make measurements, or where instrumentation modifies the experimental measurements or behaviour of the system such as thermocouples locally quenching a flame [10]. Another advantage over other physical instrumentation is that as the burning velocities of the different gases vary, there has been extensive work in modelling such velocities [11, 12] and depending on conditions, is that a camera can make measurements over a larger field of view. Recently, image processing tools have been implemented in estimating heat release rates for façade fires, with the use of two cameras [13] where the relationship between volume and heat release rate from the images was calibrated against known heat release rates.

There are challenges associated with the application of these digital images as a viable option to match the measurement sensitivity of more invasive options, such as heat flux gauges and thermocouples. First, it is important to define a flame, in this paper a flame is the zone in space where fuel and oxygen react together in a process called combustion to produce visible light. The inside of the flame zone contains unburnt fuel, the outside of the zone is air, with exhaust gases emitted from the uppermost tip of the flame. The flame zone is assumed to be thin (e.g., 0.4–1.4 mm (methane), 0.6–1.6 mm (60% methane 40% hydrogen mix) [14]) as combustion reactions are faster than the diffusion of fuel gases (5.1 mm/s methane in air) and oxygen. A cross section of a candle flame with the flame zone is shown in Figure 1.

Figure 1
figure 1

(a) A diagram of a candle and flame and (b) a cross section of the flame, showing the flame zone, unburnt fuel and oxygen moving towards the flame zone, not to scale

Full size image

In the flame, oxygen molecules migrate towards the flame zone from the surrounding air, whilst unburnt fuel molecules migrate towards the flame zone from within. For a fire to start it needs oxygen, fuel and an ignition source, but once a fire has started the ignition source is the flame itself, creating a self-sustaining flame zone [15]. The flame zone can appear to be stationary, as with a candle flame in a motionless atmosphere, or a rapidly changing shape, for example, in a draft. To measure the external dimensions of a stationary candle flame is relatively simple, it is regularly shaped, i.e., it is the same from all angles at the same altitude if the viewer is normal to the vertical direction of the wick, does not change with time, allowing a ruler or other metric device to be placed behind it at a known distance and the observer only needs to use a single equation to calculate the size of the flame, see Figure 1b, it is small and does not produce large quantities of gases or heat, i.e., it does not pose a significant safety risk to a person or measuring equipment in close proximity to the flame.

In Figure 2, the distance from the camera/observer to the flame is ‘A’ and the distance from the camera to the measuring device is ‘B’. The dotted lines show how much of the measuring device is blocked by the flame, 5 units, but the flame is not 5 units high. The actual height of the flame (H) is given by:

$${\text{H}} = \frac{{{\text{number\, of\, units}} \times A}}{A + B}$$
(1)
Figure 2
figure 2

A schematic showing how to measure the height of a candle flame

Full size image

So, if B equals A, the height if the flame in Figure 2 will be 5 units × A/2 × A = 2.5 units. The width can be measured in the same way by rotating the measuring device 90° whilst keeping it in the line of sight, behind the flame from the camera/observer, see Figure 3.

Figure 3
figure 3

The camera/observer view of measuring the width of a candle flame

Full size image

To measure a rapidly changing, irregularly shaped and large flame, a method is needed which keeps the surveyor and equipment away from the maximum reach of the flame and any fumes and hot gases being generated. Cameras combined with a measuring device offer a solution to these problems. A camera replaces the person, saves images at small intervals of time, and when the measuring device is in the camera's field of view, Eq. (1) is employed. Cameras do, however, present an additional problem in obtaining accurate measurements because the view can be distorted by the lens, in what is commonly referred to as the “fish-eye effect”. Current literature does not quantify the errors in measurements of flame lengths, distortion effects, as well as depth consideration that arise from using 2Ddigital imaging, as well as how these images could be accurately used for the calculation of total flame areas or heat release rates for battery fires. Therefore, this paper presents a novel methodology for capturing and analysing flame data in complex phenomena such as lithium-ion battery failures, and then tests the methodology with real experimental setups and validates the results with reference measurements. The experiments were conducted at the DNV site at RAF Spadeadam in April and May 2021 using Envision-AESC modules comprising 8 × 56.3Ah lithium nickel manganese cobalt (NMC532) pouch cells. Fire was initiated by nail penetration using a purpose-built steel rig.

2 Image Processing

This section highlights the different options for processing digital images for the purposes of correctly measuring flame lengths, flame areas, and the location of flames. A digital image is a two-dimensional recording of a scene from a set point. It is stored in matrix configuration, with each cell within the matrix representing a single pixel. The number of rows of pixels gives the height of the image and the number of columns gives the width. These numbers give the resolution of an image, for example, a 1080p HD image (1920p × 1080p) is 1920 pixels in width (columns) and 1080 pixels in height (rows). These are numbered from the top left corner of the image, with the top left pixel given a location of (1,1), as shown in Figure 4 and in the case of a 1080p HD image the bottom right pixel location is (1080, 1920), as the row number is quoted before the column number. In the rest of this manuscript and supplementary material, we will use this standard way of defining pixels for digital images.

Figure 4
figure 4

Representation of pixels for a given location in an image

Full size image

The light sensor in a digital camera is called a charge coupled device (CCD). If this is scrutinised at the microscopic level, it can be seen that it is made of many small squares, also called pixels, arranged in a grid formation. These are numbered in the same manner as the pixels in the image and correspond to each other. A CCD takes advantage of the photoelectric effect, where an electron is ejected from an atom if a photon of light hits it at a high enough energy, however a CCD only emits electrons when photons between certain wavelengths strike it, as shorter wavelengths (higher energy photons) are blocked by the lens [16], and longer wavelengths do not contain the required energy to eject an electron, leaving the CCD sensitive only to the visible part of the electromagnetic spectrum. If the CCD is connected to an electrical circuit, an electrical current is produced which is proportional to the number of photons that are incident on the pixel in a unit of time: thus, measuring the current measures the light intensity. In the next sections we present the possible recording methods available from digital imaging, with the concomitant advantages and disadvantages.

2.1 Black and White Recording Method

The simplest method to record an image is as ‘black and white’. This is where each pixel in the CCD either receives light with both a wavelength in the working range of the CCD, and a high enough intensity of these to create a current above a minimum threshold to record a ‘1’ in the corresponding pixel in the digital image matrix; or, if neither of these events occur, a ‘0’ is recorded. When this image is displayed the pixels that contain a ‘1’ are white and the pixels that contain a ‘0’ are black. The resulting image from using this method can be hard to decipher, and vary depending on the minimum threshold, due to the quantity of information that is lost in recording, however, this method can be useful for identifying objects, with a high contrast, such as barcodes and QR codes. An example of this can be seen by comparing Figure 5, which is a colour photo comparable to what the human eye sees, with Figure 6a–f which are black and white images of the same scene but have different minimum thresholds. The images were collected during experiments at the DNV site at RAF Spadeadam during April and May 2021.

Figure 5
figure 5

A colour still photo taken from an observation camera, scene used in the following section to identify thresholds

Full size image
Figure 6
figure 6

Figures in black and white images with different thresholds for light detection used to identify objects with a threshold of: (a) 0, (b) 200, (c) 400, (d) 600, (e) 800 and (f) 1000 (Color figure online)

Full size image

Comparing the images in Figures 5 and 6, at a minimum threshold of 0, there are still a few black pixels in the centre of the image with a value of ‘0’ as ‘0’ is not above the minimum threshold these pixels. The black appears grey, because each pixel is small, so to the observer they converge with the surrounding white pixels and appear lighter. When the threshold is increased to 200, the rig becomes visible in the centre of the image, along with other dark areas of Figure 5. The sky, clouds and turret, are however, still indistinguishable from each other. When the threshold is increased to 400, more of the concrete pad comes over the threshold and the rough surface can now be seen more clearly. The rig arm can now be seen, but the sky and clouds are still indistinguishable from one another. When the threshold is increased to 600, the concrete has become largely black, the turret is clearly distinguishable against the sky and clouds, but these are still indistinguishable from one another. When the threshold is increased to 800, the sky and clouds are now distinguishable, but at the expense of almost every other part of the image becoming black, the rig and turret can no longer be distinguished from the concrete. When the threshold is increased to 1000, almost all of the image is black, the maximum value is 1024, so only the brightest pixels in Figure 5 are now ‘1’s’ or white.

Figure 7 shows there is a nonlinear dependency between the minimum light intensity threshold and the number of pixels that record a ‘1’. This is because an image has bright and dark areas, and it is possible that a whole area of the picture can change from a ‘0’ to a ‘1’ by changing the minimum threshold by 1. Figure 7 was created using Figure 5, changing the minimum threshold from 0 to 1000 and counting the number of ‘1’s’ each time, the image has 2160 rows and 3840 columns, producing a maximum number of pixels of 2160 × 3840 = 8,294,400 pixels.

Figure 7
figure 7

A plot of the effect of minimum threshold on the outputted number of pixels with thresholds from 0 to 1000 (Color figure online)

Full size image

2.2 Greyscale Recording

The next simplest method is to record an image as ‘monochrome’, often called ‘greyscale’. This is where each sensor grades the light intensity that falls onto it using a scale between 0 and 255, with 0 being black, i.e., no light, then getting brighter at each increment until 255, which is displayed as white. The image obtained using this method requires more digital storage (128 times more than black and white), which gives a much clearer representation of the original scene, as shown in Figure 8. In Figure 8, there are 3 methods shown for recording an image. All three use the general equation for calculating each pixels intensity:

$${\text{Pixel\, intensity}} = {\text{rR}} + {\text{gG}} + {\text{bB}}$$
(2)

with the condition r + g + b = 1, and where r is the coefficient for the red light intensity (R), g is the coefficient for the green light intensity (G) and b is the coefficient for the blue light intensity (B).

Figure 8
figure 8

Different methods of accounting for intensity of red, green and blue light intensity with (a) treating all colour equally, I.e., average (b) giving green twice the importance as blue and red light intensity and (c) a weighted scaling between red, blue and green light intensity (Color figure online)

Full size image

Figure 8a treats each colour equally, r = g = b = 0.3333. Figure 8b, gives G twice the importance of R and B, r = b = 0.25, g = 0.5. Figure 8c uses r = 0.2126, g = 0.7152, b = 0.0722. The coefficients can be changed to any that the user desires, but as can be seen in Figure 8, all three methods show an image with greater detail than using black and white, with little difference between the three methods.

2.3 RGB Recording

Images can be recorded in colour, using a red, green and blue method (RGB). This method uses a colour filter as shown in Figure 9, which is placed over the CCD. There are twice as many green pixels as red or blue, because the human eye has evolved to recognise green light the most, this makes the resulting display appear a better quality. Only red light can pass through the red filter, green light through the green filter and blue light through the blue filter, so the CCD now measures the intensity of a single colour on each pixel. However, digital screen pixels are made up of three parts, a red, green and blue, so a value for each of the three colours must be stored in each pixel. So, when saving the image, a value for the two missing colours is estimated from the surrounding pixels of the same colour, for example pixel (4,4) in Figure 9 will only record a value for green light intensity, to get a value for blue light intensity, for example, pixels (2,3), (2,5), (4,3), (4,5), (6,3) and (6,5) may be used, similarly for red light intensity, pixels (3,2), (3,4), (3,6), (5,2), (5,4), and (5,6) may be used. The resulting image from using this method requires more digital storage (3 times more than greyscale) but produces a colour representation of the original scene, as shown in Figure 5.

Figure 9
figure 9

An example of a colour filter used with a CCD to produce colour digital images (Color figure online)

Full size image

3 Image Processing of Lithium-Ion Battery Fires

Using the different digital imaging techniques presented in Sect. 2, the objectives of the processing of flame images are to: (i) determine the impact of image resolution and flame capturing speeds on quality of outputted flame measurement (Sect. 3.1), (ii) remove the ‘fisheye’ effect from images (Sect. 3.2), (iii) measure the surface area of a rapidly changing, irregularly shaped flame (Sect. 3.3.) and (iv) calculate the heat release rate as a function of time in a battery fire (Sect. 4).

3.1 Comparing Different Camera Resolutions and Camera Speeds

If the change in shape and size of an object of interest (m s−1 or m2 s−1) is much faster than the camera speed (frames s−1), there will be a large jump in measurements between images (m frame−1 or m2 frame−1), meaning approximations of these dimensions will have to be made, causing a loss in accuracy. If a low shutter speed (s−1) is used, the parts of the object of interest that move will be recorded by multiple pixels, generating a blurry image: for accurate measurements of an object of interest, a clear image is required. Each pixel in an image should represent a small area in the plane of interest, this is achieved with a high pixel density (pixels per m2), the smaller the pixel area the greater the precision. If a pixel represents a large area, edges of the object of interest, e.g., a flame, may not be recorded, as the pixel it is recorded in, by the CCD, will take an average of everything in that pixel, including the background.

An experimental setup was assembled with imaging for lithium-ion module failures caused by mechanical abuse by nail penetration, based on the experiments of Christensen et al. [17]. The setup included GoPro Hero 9 Black digital cameras with SanDisk Ultra 256 GB A1 micro SD memory cards. The cameras have settings for resolution, camera speed and shutter speed. To test the effect of different resolutions and flame capture speeds, the cameras were setup with different settings: three cameras were positioned as shown in Figure 10, with two different settings, camera ‘x’ had a resolution of 3840p (pixels) × 2160p with a camera speed of 60FPS (frames s−1), cameras ‘y’ and ‘z’ had resolutions of 1920p × 1080p with a camera speed of 240FPS. All three cameras had their shutter speeds set to ‘AUTO’ as this setting allowed the camera to adjust the shutter speed to allow for changing light conditions, caused by the cloud coverage on the day.

Figure 10
figure 10

The arrangement of the cameras employed to record capturing flame flares generated from the failing Envision-AESC lithium-ion module. The images used in the rest of this text are from the cameras labelled ‘X’ ‘Y’ and ‘Z’

Full size image

Figures 11 and 12 show images taken from camera ‘x’ during the experiment shown in Figure 10, where a single Envision-AESC module, held on the metal frame using steel straps and bolts, was penetrated by a steel nail (see Ref. [17] for all details). Figure 13 show images taken from camera ‘y’. Images in Figure 11a and b were taken the moment before and after ignition, respectively. The images were taken by camera ‘x’, 1/60 = 0.0167 s apart. Comparing them, the moving parts of the images are the vapour cloud and ignition. The vapour cloud is a mixture of gases, predominantly hydrogen, CO, CO2 and fine droplets of organic solvent, which is vented by lithium-ion batteries in thermal runaway caused by abuse such as nail penetration [17]. The vapour cloud is moving too slowly for there to be an appreciable difference in the images, and the ignition shows only a small flame. Figure 11c is the magnified ignition from Figure 11b, it is 64p × 36p and the high resolution used clearly shows the flame edges. However, even though the flame that appears in Figure 11b is small it cannot be presumed to show that the camera speed is high enough to produce images close enough together not to cause a large jump in measurements, as the flame could have been produced from any time after Figure 11a was taken until just before Figure 11b was taken. To check the camera speed image, Figure 12a and b were also recorded 0.0167 s apart but, show images with different flame edges. Figure 12c shows the difference between these images, the black colour represents no change in that pixel, the red colour represents a new flame in that pixel, i.e., there is no flame in Figure 12a but there is in Figure 12b, the green colour represents a flame disappearing in that pixel, i.e., there is a flame in Figure 12a but not in b. Therefore, Figure 12c shows that the flame has mainly grown at the top edge and shrunk in the bottom left corner, but the amount is small compared to the body of the flame, showing that the camera speed is at a high enough rate for this experiment.

Figure 11
figure 11

(a) camera ‘x’ before ignition; (b) camera ‘x’ at ignition, 1 frame (16.7 ms) after (a); (c) zoomed in version of (b) where ignition commenced

Full size image
Figure 12
figure 12

(a) camera ‘x’ during thermal runaway; (b) camera ‘x’ during thermal runaway, 1 frame (16.7 ms) after (a); (c) difference between (a) and (b)

Full size image
Figure 13
figure 13

(a) camera ‘y’ during thermal runaway; (b) camera ‘y’ during thermal runaway, 1 frame (4.17 ms) after (a). (c) camera ‘y’ during thermal runaway, 4 frames (16.7 ms) after(a); (d) the difference between (a) and (b); (e) the difference between (a) and (c); (f) zoomed in version of (a) where the flame edge meets the horizon

Full size image

Figure 13a and b are two frames taken by camera ‘y’ at a moment in time where there is a large flame present in the frame and were taken 0.004167 s apart. Comparing them is more difficult than Figures 11a and b as less movement of the flame edges happens in the shorter time period, but from Figure 13d, the red pixels, showing the new flame, and the green pixels, showing disappearing flames, makes the comparison easier. The main difference is the top edge to the left hand side, with red and green mixed, it shows the fluttering nature of the flame. Now looking at Figure 13e, which is the difference between images from Figures (a) and (c) and were taken 0.0167 s apart, the same time interval as in Figure 12c, the difference in camera speed becomes apparent, however, this is still a small change when compared to the body of the flame, so even though a higher camera speed is desirable, it is not essential in this experiment to obtain the required degree of accuracy.

Comparing Figures 11c and 13f, both contain an equal number of pixels, therefore the magnification is different, with Figure 11c having a magnification of c.a. 60 times and Figure 13f having a magnification of ca. 30 times, which is one of the reasons it looks clearer, with different parts of the flame visible, and the horizon can be distinguished in the right hand side, overall a higher resolution is desirable but the lower resolution still provides good precision.

3.2 Removing the Fish-Eye Effect from Recorded Images

Close scrutiny of Figure 5 shows that the yellow turret on the right hand side appears to be curved, and also, although not immediately obvious, the right hand side of the rig appears to be further from the camera than the left; however, from Figure 10 it can be seen that the rig was set perpendicular to the camera. These are examples of the fisheye effect on images, which occurs when the camera lens distorts the image, usually to obtain a wide field of view. To remove this effect, a calibration box was used to provide exact reference points, as shown in Figure 14. This is a method previously used in other works [18, 19]

Figure 14
figure 14

(a) The calibration image showing the full calibration box; (b) zoomed in version showing that the box has stretched in image and (c) calibration line to account for image distortion

Full size image

The calibration box was 2200 cm × 600 cm × 600 cm and comprised 200 cm × 200 cm black and white squares. The box was made of an internal steel frame made from 25 mm box section steel, covered in plastic chequered panelling on each of the 6 faces, the four large faces were black and white and the two smaller (end) faces were green and red plastic chequered panelling. The use of the box allowed the distortion of the image to be seen more easily. Thus, from Figure 14a, given the fact that the box is perpendicular to the camera, and all the squares are the same size, the distortion of the lens becomes apparent: the box appears to be shorter and further away from the camera on the left hand side compared to the right.

To alleviate the distortion, the image was loaded into MATLAB for processing. The coordinates of the corners of the squares were found by the user marking them with a box, as shown in Figure 14a.

Figure 14b is the same image as Figure 14a but zoomed in around the top left corner of the calibration box, with a square outline superimposed on the image. The curvature of the turret becomes more apparent, the squares on the calibration box appear to be rectangles, i.e., they are longer than they are wide, and the right edge of the superimposed square outline shows how the calibration squares appear to be curved. The superimposed square is used to find the co-ordinates of the chequers of the calibration box, in this case the top left corner of the calibration box, as that is the point where the top left corner of the superimposed square is over. The top left corner has coordinates of (870, 508). Once all of the coordinates have been found they can be plotted, as shown in Figure 14c.

Figure 14c is a scale plot of where the calibration box sits in Figure 14a, it is a useful redundancy, built in as a check that the data has been entered correctly, i.e., there are no erroneous points in the plot. The points have been joined horizontally and vertically to make it easier for the user to see if there is a pattern in any distortions of the image. In Figure 14c, the left hand side of the plot looks shorter than the right, as does the calibration box in Figure 14a and the squares do not look of equal width along a horizontal line, i.e., they are narrower on the left hand side than the right.

The next step is to stretch the image in one direction, any direction can be first, as long as it is stretched in a direction perpendicular to this afterwards. In this case, the image is stretched vertically first, then horizontally. With the squares being shorter on the left, the vertical stretch for each pixel is greater on the left side than the right, to make the squares of equal height.

MATLAB has an inbuilt function that uses the least squares method for finding trend lines, for example, a horizontal line fits the equation y = c, where c is a constant. A straight line with a gradient fits the equation y = ax + c, where a and c are constants, with the coefficient of x being a, which is the gradient of the line. A curved line has a term with a power of at least 2, for example, y = ax2 + bx + c, where a, b and c are constants. MATLAB can generate equations up to and including a power of 9, for example, y = ax9 + bx8 + cx7 + dx6 + ex5 + fx4 + gx3 + hx2 + ix + j, where a, b, c, d, e, f, g, h, i and j are all constants, but this requires a minimum of 10 points, limiting the vertical trend lines to a power of 3, as there are only 4 vertical points. The number of points could be quadrupled by adding imaginary lines of symmetry (mirrors), horizontally and vertically, see supplementary material section D running through the centre of the image, or doubled by using one ‘mirror’, this assumes that the image is equally distorted by the lens around the centre of the image.

Once horizontal trend lines have been found, they can be overlaid on the horizontal lines in Figure 14c, to check that they are a good fit, and to obtain an insight as to how the image is distorted on the right hand side. The plots for each power considered is found in the Supplementary Material, Section A, for the nine options for fitting trend lines to the points of the calibration box. In these results, it is shown later that it is not necessarily the case that a higher power fit increases accuracy. The next step is to find how much each pixel needs to be stretched so that each pixel represents the same height. It was decided that each pixel should represent an area of 1 mm × 1 mm on the plane where the calibration box lies, as this size is acceptable for both precision and accuracy, as 1mm2 is much smaller than the flame area. To achieve this a factor was calculated between each horizontal trend line on every column of pixels. As the calibration squares on the calibration box were 200 mm in height, 200 can be divided by the number of pixels in a column between the trend lines to give a vertical expansion factor in the middle of the trend lines. For example, if at column 50, two trend lines are at row 600 and row 750, then a vertical expansion factor of:

$$\frac{200}{{750 - 600}} = \frac{4}{3}$$
(3)

would be needed at row:

$$600 + \frac{750 - 600}{2} = 675$$
(4)

This was done for various powers and full detail of all possibilities analysed can be found in Supplementary Material Section B, with the outcome being that a vertical expansion above a power of two does not give realistic results. This analysis was expanded to all directions, for all cameras, and the fisheye effect was removed from all imaging, and again full details are provided in Supplementary Material sections C and D.

3.3 Surface Area of a Rapidly Changing, Irregularly Shaped Flame

To calculate precision measurements compared to referencing, a score was assigned to each image compared to what the actual size was in the reference images. A score of ‘1’ meant that the measurements were perfect, above ‘1’ means that the measurement was too large and below ‘1’ it was too small. For example, a score of 1.1 means that if the actual measurement was 1000 mm the measurement after the analysis was 1100 mm. This was done for all physical reference points, namely: (i) on the calibration box after the horizontal expansion when only using one mirror; (ii) on the calibration box after the vertical expansion when only using one mirror, (iii) on the calibration box after the horizontal and vertical expansions when using both mirrors; (iv) on the rig widths measuring how accurately the expansions have expanded the width of the rig, at various heights, after both expansions, when using both mirrors, and averages these into a single score; (v) on the rig heights measuring how accurately the expansions have expanded the height of the rig, at various widths, after both expansions, when using both mirrors, and averages these into a single score; (vi) is a weighted average of the xy result and both rig scores, and is the score used to determine the best combination of powers. The results are shown in detail in Supplementary Material section E, but the best accuracy is when using a horizontal power of two and a vertical power of three, the weighted average score was 1.007357, meaning there was an average overestimation of 0.736% in the measurements.

Finally, the aspect of error not yet addressed is the numerical error that can arise from the code. For example, the code may not always correctly recognise pixels which are flames and may falsely mark pixels which are not flames as flames, i.e., generate false positives. This is of particular concern on the perimeter of a flame body, as this is where the flames become thinner and from the viewpoint of the camera, merge with the background, which is made worse by the fluctuating nature of these flames. Recognising that no measurement is perfect a maximum and minimum range is also measured in the analysis, the maximum adds a set number of pixels around the flame perimeter, the minimum only records a flame if that pixel is surrounded by flames of the set number of pixels. The set number of pixels can be changed by the user, in this analysis it was set at ‘2’, which represents 2 mm in length or 4 mm2 in area. Figures 15a–d show the measured, minimum, and maximum flames from Figure 12a.

Figure 15
figure 15

(a) image of flame, with off-gassing and rig in background, (b) flame image in black and white, (c) threshold accounting for light from pixels considering the maximum amount of light, i.e., the maximum flame body, (d) higher threshold accounting for a minimum amount of pixels per area counting as flame, (e) maximum difference between the flame with the two thresholds considered, (f) minimum difference between the flame with the two thresholds considered, (g) red box drawn around the area being measured which changes based on where the flame is, as can be seen at a later time when a smaller flame is present, as shown in (h) (Color figure online)

Full size image

The differences in the perimeter measurements are that when the maximum perimeter is used (c), more interference becomes recorded, for example the reflection of the flames from the wet concrete floor, also the body of the flame has small areas where no flame was recorded, whereas the minimum (d) is the opposite. These are shown in Figures 15e and f, where the red in (e) shows new pixels that have recorded a flame in (c) compared to the original (b), and the green in (f) shows pixels that have not recorded a flame in (d) compared to the original (b).

Another source of error is interference in the background, e.g., when a rough surface is wet with a thin film of water, the film can reflect sunlight to the camera, direct sunlight has R, G and B parameters similar to those used to identify flames, therefore, Matlab marks these pixels as flames. To reduce this effect an assumption can be made that the flame edge is connected to a flame body and only summing pixels that are within a body that is above minimum dimensions. This is complicated by the irregular shape of a flame, so a box cannot be drawn around an area at the beginning of the analysis, but a box that constantly changes shape and size with the changing flame is required. To achieve this a minimum pixel number which contained flames was required for the horizontal and vertical axis: only if these were above the minimum would a pixel be counted as a flame.

Figures 15g and h show a box (in red) drawn around the area which was being measured, it changed size and location depending on the body of the flame. The very tips of the flames can be omitted from the analysis, but this can be counteracted by using the maximum value from the perimeter errors.

4 Measuring the Flames and Resulting Heat Release Rate

The flames from the experiments were isolated from the images as described in Sect. 3, and each pixel was then multiplied by the horizontal and vertical expansion matrices to obtain the area of each pixel. These were then summed to get a total area for each image. However, as the flame edges move towards the cameras the distance between the camera and the calibration plane reduces, meaning the flame areas are underestimated as the position of the flame is not considered. To counteract this, the calibration was changed depending on the distance to the flame edge. For example, as shown in Figure 10, camera ‘x’ can be used to measure how far the flame has moved towards cameras ‘y’ and ‘z’—likewise camera ‘y’ can be used to measure how far the flame has moved towards cameras ‘x’ and ‘z’, and camera ‘z’ can be used to measure how far the flame has moved towards cameras ‘x’ and ‘y’. As shown in Figure 2, this involves multiplying all the pixels by a new constant which changes at each interval in time. The total flame area was then calculated as a function of time from the images.

For the calculation of heat release rate (HRR) from total flame area from the images, a method is required to convert flame area to HRR. The volume of a turbulent, buoyant flame is approximately linearly proportional to the HRR (such that HRR [kW] = m × flame area (cm2) + c, where m and c are fuel-dependent constants) as first proposed in 1982 [20], and later confirmed for turbulent flames for a variety of fuels [13, 21]. This gives a method to calculate the heat released (HR), as once the areas have been calculated from the images, the HRR and HR can be calculated, by multiplying the area by the known HRR per unit area. The constants (m,c) linking flame area to heat release rate are dependent on the fuel causing the combustion. The HRR for most of the hydrocarbon fuels found in the ejecta of failed lithium ion batteries are all in a similar range [17, 22, 23], so for the purposes of this analysis the same constants (m,c) giving the relationship between HRR and flame area were used for all hydrocarbons emitted, namely that of the hydrocarbon methane.

The experimentally-validated relationship between HRR and flame area for methane [24] was therefore employed, and is given in Eq. 5 below:

$${\text{HRR }}\left[ {{\text{kW}}} \right] = 0.00{9489} \times {\text{flame\, area}}\left( {{\text{cm}}^{{2}} } \right) + 0.0{14722}$$
(5)

where (m,c) = (0.009489, 0.014722). The results presented in the following Figures 16, 17 and 18 focus on analysing the effects of the correction factors presented in Sect. 3 on the measured pixels used for the calculation of flame areas and HRR, but it is important to note that they can easily be adapted with different linear of flame area to HRR if the values for (m,c) are further refined in the future, especially to take into account effects such as battery state of charge or cathode composition which have been shown to affect gas release rates [22, 25]. In fact, it is important to note that as more detailed HRR experiments and detailed kinetic schemes are added to the lithium-ion battery literature in future years, the values of (m,c) and thus the relationship between HRR and flame area can be further refined with future data based on parameters such as slightly different electrolyte composition and cathode and anode compositions. However, the resulting accuracy in measuring the flame area from Sect. 3, using the correction factors, is independent of the HRR relationship from Eq. 5, making the results and percentage errors found in considering the visual method proposed valid irrespective of the values of (m,c).

Figure 16
figure 16

Plots of the heat release rate normalised, maximum, and minimum areas that can be calculated from the measured pixels of the flame. No correction factors were used

Full size image
Figure 17
figure 17

Plots of the heat release rate (mean) with and without the correction factor implemented

Full size image
Figure 18
figure 18

Plots of total heat release with and without correcting for flame movement

Full size image

To show the effect of the flame edge under-estimation on heat release rate without correction factors, Figure 16 shows the heat release rates calculated without applying the correction factors presented in Sect. 3. The minimum, normalised, and maximum are shown in the Figure, where the minimum and maximum areas are produced using the same method as that in Figures 15e and f, and the normalised is the actual value used. No adjustment has been made for the flame being closer or further away from the cameras in Figure 16, the analysis assumes the whole flame is in the same plane as the calibration box. To note, as stated in the introduction of this paper, that the flame thickness for these type of flames is of the order of 1 mm [14], so well below the flame lengths measured in these experiments.

When applying the correction factors presented in Sect. 3 to account for flame distances, the heat release rate found was significantly higher than as in Figure 16, as can be seen in Figure 17. This shows the importance of accounting for the position of each flame flare, pixel by pixel, which cannot be done with invasive measurement techniques, especially once multiple flares occur with highly irregular fast changing flame shapes, locations, and with flame spread in different directions rapidly changing as a function of times.

The effect of the error without the calibration is compounded when trying to calculate the total heat release rate from a lithium-ion battery fire: as can be seen in the figure below, the total heat release rate, once calibration is taken into account, is almost twice what would have been calculated without the calibration. This shows that, because flames are thin regions in space, the error in pixel measurements deriving from spatial resolution can create very large errors in the analysis, and shows why the methodology to fully correct the errors in measurement presented in Sect. 3 is essential for the use of digital imaging for the processing of heat release rate calculations for systems like lithium-ion battery pack fires where there can be multiple flame flares and simultaneously changing in different directions.

Figure 18 shows that the total heat release is approximately doubled by taking into account the movements of the flame, and so without the correction method developed in this paper, using imaging for heat release estimation very much underestimates the total heat released from the battery fire.

5 Conclusions

This paper proposed and tested a new methodology to accurately capture images of flames generated from the ignition of a lithium-ion module, post-process them to remove distortions and errors as well as background image noise, and finally use the images to reconstruct the total flame areas generated from the failure, even when multiple flares occurred simultaneously. The results in Sect. 4 show that, without appropriate calibration and error correction, using raw images of flame flares can give an error of almost 50%, significantly underestimating the actual heat release rate and total heat release of the flame flares. The extensive calibration and testing of this newly-developed image processing method presented in this paper ensures that a crucial non-invasive tool such as visual imaging can be used more widely in the measurements and quantification of the effects of fire from lithium-ion batteries. The tool is not useful solely for battery fires but can be further adapted to be used in scenarios where the fuel is drastically different, by altering the equation relating the heat release rate and flame area as a function of fuel, as long as the flame generated from the reactions is visible and an appropriate heat release rate relation between flame area and heat produced is known. Furthermore, as the gases released from different lithium-ion battery composition failures is studied in the literature, even more accurate relationships can be used to link the chemical composition of the ignited gases and the heat release rate measured from imaging.