1 Introduction

Image colourisation is a method of transforming a grayscale image into a colourised one. The intensity of each pixel is replaced by a colour representation: R, G and B channels or, as in YCrCb, two chrominance and one luminance channels. This task is not trivial and there are no explicit solutions for transforming a one-dimensional pixel value (intensity) into a three dimensional vector. Thus, such a process can be performed in various ways, which results in a wide range of developed colourisation approaches.

The first colourisation algorithms to enhance still images and video sequences were applied in 1984 [9]. Nowadays, with the increasing efficiency of modern computers, the methods are much more sophisticated and they make use of the computational power that is available. The algorithms, in general, can be applied to two methods: automatic and semi-automatic. The first one includes the algorithms that compare the grayscale image with a given colour reference. Using various approaches, they try to fit the parts of both images and to properly estimate the colour of each image region. In the second method, the user indicates the colour hints in the form of scribbled strokes of a given colour (the so-called scribbles). This way the colourisation process starts from the inserted scribbles and expands, to finally fill up the whole image.

While the quality of the colourisation outputs is currently at a very high level, which makes the results almost indistinguishable from the original colour images, the complexity of the methods has also increased. It should be noted, that the colourisation that employs the semi-automatic approach is an interactive process, in which a user indicates scribbles, observes the outputs and modifies or inserts additional scribbles in order to achieve the desired final effect. Hence, the computational efficiency should also be considered and maximised in order to reduce the time a user spends on colourisation. The simplicity of the implementation, as well as the reduction of the computational burden, while preserving a high output quality were, therefore, the main goals of the new isoline-based colourisation technique that is presented in this article.

The article is organised as follows. In Section 2, we present the range of possible colourisation algorithms that are available in the literature. In the next section, we introduce the basic idea of our method and present some modifications that lead to its better performance and lower computational complexity. In Section 4, we investigate the quality issues of colourisation, which include parameter optimisation and a performance comparison with other colourisation methods. In Section 5, we present an analysis of the computational complexity of the new algorithm as compared with some reference techniques. Finally, in the last two Sections, we present some possible applications of the proposed colourisation method and some final conclusions.

2 State-of-the-art colourisation methods

As mentioned in the introduction, there are two main types of colourisation methods: automatic and semi-automatic. The first requires the reference colour image, while the second one needs human interaction. In this brief literature review we enumerate some algorithms of both colourisation approaches.

The authors of [19]developed the seminal work on automatic image colourisation. They suggested employing the idea of colour transfer between images presented in [13]. The authors make use of l α β colour model [16] instead of the classic RGB space. They investigate the statistics of pixel intensities in a sliding processing window utilising their standard deviation and mean value. This way they transfer the colours to the parts of grayscale image which show similar luminance statistics. An exemplary application of the algorithm for biomedical image colourisation is presented in [23]. The authors add some modifications to the original idea so that the measure of similarity between image patches is tuned by a single parameter.

In [24] another method of automatic image colourisation using a reference image is presented. The authors suggest using the pattern continuity and spatial consistency approach instead of analyzing the image luminance properties. First, the image is segmented and then, using Gabor wavelet analysis, the vector of features used as a training set in the classification algorithm is calculated.

Another proposition of automatic approach is dedicated to the colourisation of infrared videos [20]. The authors suggest obtaining reference colour images from flash assisted photos taken during infrared video acquisition. They apply the colour transfer from the references to the whole sets of images between the so-called key frames in a video shot and utilise the luminance distance between the reference and given infrared image in each local window.

The semi-automatic methods, in contrast to the automatic ones, do require some additional user effort. It usually includes inserting the hints in a form of colour scribbles which indicate the regions of a given colour and are the starting points for the colour flooding performed by the algorithms.

An example of semi-automatic colourisation is presented in [7, 8]. The authors investigate the possibility of using the minimisation of colour difference between the adjacent pixels of similar intensity. The algorithm starts from the colour scribbles indicated by the user and expands over the whole image. The minimisation can be achieved by typical approaches, like the Least Square algorithm. The authors also show the ability of their algorithm to colourise the video sequences which is simply an extension to the spatio-temporal domain.

Another method of scribbled-based colourisation is described in [21]. In this method the authors investigate the digital paths between colour scribbles and grayscale image pixels. They use Dijkstra’s algorithm to find the most homogeneous (in terms of the intensity gradients) digital paths. The final colour of a pixel is calculated as a weighted average of all assigned colours, where the optimal weights are inversely proportional to the costs of the optimal paths.

In [6] the use of morphological distance transformation for the colourisation purposes is presented. The authors, like in [21], investigate the digital paths and the corresponding gradients. However, they use the estimation of the shortest paths provided by the so-called double-scan algorithm [14], in which the computation time is much lower than that of Dijkstra’s algorithm. The cost of the digital paths is calculated using squared intensity differences between adjacent pixels and considering an additional penalty for the path length.

The utilisation of distance transformation for the colourisation is also presented in [22]. The authors apply fuzzy clustering of a grayscale image using inserted scribbles as initial cluster centers. They make use of the estimation of Euclidean distance, provided by the distance transformation, to find the fuzzy membership of the pixels to each of the scribbles and then they perform weighted colourisation.

The use of local Markov properties of the image is provided in [1]. The colours are transferred from the scribbles to the neighboring pixels which minimise the difference in RGB space. This method represents the concept of seed pixels propagation. In [2] the image pixels are divided into unique regions using the watershed segmentation [11]. The initial oversegmantation is removed by region flooding, so that the pixels are being merged as long as the number of segments is equal to the number of colour scribbles indicated by a user.

Although most of the algorithms use pixel intensity analysis, articles [3–5] present the idea of utilising the textural properties of an image. The authors use two types of distance metrics: the intensity distance, which promotes smooth digital paths, and the textural distance, which privileges sharp intensity transitions. This way, the algorithms are able to properly colourise the parts of the image containing highly rugged structures.

3 Isoline based image colourisation

3.1 Basic concept

Our method utilises the semi-automatic colourisation approach which, like in similar methods, requires scribbles to be inserted into the grayscale image. The goal of the main part of our algorithm is to properly assign the degree of membership to a given colour/scribble to the pixels. Our idea is very similar to that of the isolines on the geographic maps. To do this, we look for the digital path from a scribble to the pixel that minimises the maximum intensity deviation. We calculate the deviation as the maximum difference between the intensity of a scribbled pixel and the intensity of any of the pixels on the digital path. The maximum intensity deviation D i,j between the pixels at positions i and j is defined as:

$$ D_{i, j} = \underset{\Phi}{\min}\{\max|I_{k}-I_{i}|\:,\:k\;\in\;\phi_{i,j} \}, $$
(1)

where I i and I k are the intensities of the pixels at i and k, ϕ i,j is the set of adjacent pixel positions that form a particular path between the pixels at i and j, Φ is the set of all the paths connecting the pixels at i and j.

In this way, we create the distance map (assignment map) for each pixel, that is very similar to a geographic representation with isolines depicting the points with the same properties, such as height or temperature. In our case the isolines represent the pixels with the same value of intensity deviation encountered on the path that leads to a given scribble. In Fig. 1 we show the isoline representation of the introduced distance map for a scribble that is indicated on an exemplary grayscale image.

Fig. 1
figure 1

Isoline distance visualisation

Full size image

In Fig. 2, we present the principle of our idea in a one-dimensional case. In this example, only one pixel, which is indicated in red, is scribbled. We should note that the isoline distance cannot decrease. For the last pixel (intensity 9), the isoline distance equals 3, although the intensity difference from the scribble (seed) is only 1.

Fig. 2
figure 2

One-dimensional isoline distance calculations

Full size image

In two-dimensional space the problem is more complicated because of the many possible paths that start from the scribble pixels. Moreover, one scribble usually consists of a group of pixels. Thus, the isoline calculation should involve a search among all of the possible paths from all of the pixels included in a given scribble. Figure 3 shows an example of a simple 2D isoline distance calculation using 4-neighborhood relation. Although computation on a 2D image can be executed sing Dijkstra’s algorithm, this requires a great deal of computational time and therefore we rejected such a solution. Instead, we developed an algorithm that is based on the local estimation of the introduced isoline distance.

Fig. 3
figure 3

Two-dimensional isoline distance calculation. Best path - gray, seed pixel - red, destination pixel - green.

Full size image

Our algorithm includes a modification of the double-scan method, which is exploited in the topographic distance transformations [6, 15]. This distance transformation allows the Euclidean distance to be estimated using very simple and computationally efficient operations in a local moving window. It employs the double scan algorithm, which considers only the information from the three neighbouring pixels whose selection depends on the direction of the scan (left-to-right, top-to-bottom or right-to-left, bottom-to-top, see Fig. 4) at each step. While the pixels initially have an infinite distance, only those within the scribbles are initialised with zeros (see Fig. 5 b). Then, at each step, the new distance value is calculated using the distances of the pixels within the window:

$$ D_{1} \leftarrow \min\left\{ D_{1}, D_{2}+1, D_{3}+1 \right\}, $$
(2)

where D 1, D 2 and D 3 are the distance values of the corresponding pixels P 1, P 2 and P 3. Figure 5 presents an example of classic distance calculations using the double-scan algorithm.

Fig. 4
figure 4

Double scan algorithm - moving window

Full size image
Fig. 5
figure 5

Example of a double-scan distance calculation

Full size image

Since the main advantage of the proposed algorithm is its high computational efficiency, the double scan was selected rather than Dijkstra’s algorithm. The predominance of the double-scan algorithm can be explained as follows. Assuming that the image matrix consists of M×N pixels, a connected graph of the pixel grid has E = N(M − 1) + M(N − 1) edges and V = M N nodes (no diagonal connections). The number of operations for calculating distance using Dijkstra’s algorithm (O D i j ) can be optimised by Fibonacci Heaps [10] to:

$$ O_{Dij}=E+V~\log V = 2V-M-N+V~\log V. $$
(3)

For the double-scan algorithm, this number is lower and equals:

$$ O_{DS}=2(M-1)(N-1) = 2V-2M-2N+2. $$
(4)

The dependencies of the number of operations required by Dijkstra’s algorithm and double-scan algorithm, on the number of pixels, are illustrated in Fig. 6. The number of pixels is selected so that it corresponds to the range of images: 100 × 100 to 400 × 400 pixels. For such a range, both dependencies are linear and the double-scan is approximately 3.6 times faster, which was also confirmed by the results of the evaluations presented in Section 5.

Fig. 6
figure 6

Comparison of computational burden of Dijkstra’s algorithm (red line) and double-scan algorithm (blue line)

Full size image

In our algorithm, we perform a series of operations using the pixels P 1, P 2 and P 3 to obtain one distance map for every colour scribble indicated in the image. First, we initialise the isolines distance array, denoted as \(\mathcal {J}\), so that each pixel that is included in the scribbles is given a 0 isoline distance, while the others are set to \(\infty \). We also establish an auxiliary propagated seed intensity (PSI) array, which is denoted as S, wherein only the seed pixels indicated with a given colour are initially given their grayscale value while others are set to \(\infty \). The propagated seed intensity array is crucial in our algorithm since it enables intensities from the seed (scribbled) pixels to be spread, so that eventually each pixel is given the propagated seed intensity that is equal to the intensity of the closest scribbled pixel.

In each step of the double scan algorithm, we perform the following calculations. First, we compute the absolute intensity difference between the values from PSI array S for pixels P 2, P 3 and the grayscale value of pixel P 1, and thus we create two initial isoline distance estimations for pixel P 1:

$$ \mathcal{J^{\prime}}_{2}=|S_{2}-I_{1}|, ~~\mathcal{J^{\prime}}_{3}=|S_{3}-I_{1}|, $$
(5)

where \(\mathcal {J^{\prime }}_{2}\) and \(\mathcal {J^{\prime }}_{3}\) are the preliminary isoline distance estimates, S 2 and S 3 are the values of the PSI array for pixels P 2 and P 3, I 1 is the grayscale intensity of pixel P 1.

In the next step, we compare the two above estimates with the isoline distance of the corresponding adjacent pixels and we always choose the greater one, so that our isoline distance will never decrease, (consider Figs. 2 and 3):

$$ \text{if}~(\mathcal{J^{\prime}}_{k}<\mathcal{J}_{k})~\text{then}~(\mathcal{J^{\prime}}_{k}\gets \mathcal{J}_{k}), $$
(6)

where \(\mathcal {J}_{k}\) is the isoline distances of pixel P k , (k = 2, 3).

Finally, in the last part of our algorithm, pixel P 1 is given the smallest isoline distance from the following values: \(\mathcal {J^{\prime }}_{2}\), \(\mathcal {J^{\prime }}_{3}\) and \(\mathcal {J}_{1}\):

$$ \mathcal{J}_{1} \leftarrow \min(\mathcal{J^{\prime}}_{2},\mathcal{J^{\prime}}_{3},\mathcal{J}_{1}). $$
(7)

Additionally, since each of the three values has its own assigned seed intensity, we transfer the selected one to pixel P 1, according to the smallest value selected:

$$ \begin{array}{ll} \text{if}~(\text{smallest} = \mathcal{J^{\prime}}_{2})&\text{then}~(S_{1} \leftarrow S_{2}),\\ \text{if}~(\text{smallest} = \mathcal{J^{\prime}}_{3})&\text{then}~(S_{1} \leftarrow S_{3}),\\ \text{if}~(\text{smallest} = \mathcal{J}_{1})&\text{then}~(S_{1} \leftarrow S_{1}). \end{array} $$
(8)

If the isoline distance has already been calculated in the previous scan, then it is possible, that initially the estimated distances will be larger in the second scan because of the presence of two (or more) scribbles of the same colour. In this case, the scribbles (the seed pixels) seem to compete with each other and the one, whose path to a given pixel has lower total intensity deviations, wins.

After performing the running window double scan operations, each pixel of the image receives its isoline distance. As can be observed, only four corner pixels are excluded from the analysis. Their values can easily be replaced using one of the adjacent pixels. In our algorithm, we decided to replace it with the mean of all three neighbours. Figure 7 illustrates an example of the results of the first and second scan for both the propagated seed intensity and the isoline distance arrays. We also provide a pseudo code to calculate the isoline distance (Algorithm 1).

figure c
Fig. 7
figure 7

Example of the calculation of the isoline distance and seed intensity arrays

Full size image

When the isoline distance maps are completed, the colour of each pixel can finally be determined using chrominance weighted averaging performed in the YCrCb colour space. The luminance Y component is obtained directly from the grayscale image while the new chrominance values are assigned to pixels using the weighted average of the chrominance values of inserted scribbles:

$$ \mathcal{C}_{j} = \frac{{\sum}_{i=1}^{N}C_{i}\:\exp\{-\frac{\mathcal{J}_{i,j}}{\sigma_{i}}\}}{{\sum}_{i=1}^{N} \exp\{-\frac{\mathcal{J}_{i,j}}{\sigma_{i}}\}}, $$
(9)

where \(\mathcal {C}_{j}\) is an assigned chrominance (Cr or Cb) to the pixel at position j, C i is the corresponding chrominance of the i-th scribble, \(\mathcal {J}_{i,j}\) is the calculated isoline distance from the i-th scribble to the pixel at position j, σ i is the parameter that is responsible for the level of colour blending assigned to the i-th scribble. In the experiments we did not discriminate the scribbles, so that σ i = σ, where σ is an experimentally found, default parameter value.

3.2 Algorithm parameters

In order to allow the results of our colourisation algorithm to be modified, we introduced two additional parameters that can be adjusted to obtain satisfactory colourisation effects. Their impact on the performance of the algorithm is discussed in detail in Section 4. These parameters allow the sensitivity of the colourisation to be increased to the topographic distance on the image domain or to the intensity deviation between the image pixels and the seeds of the scribbles.

The first parameter λ is used to raise the absolute difference between adjacent pixel intensities to the power and the second parameter δ is the step penalty that is added to each transition between neighbouring pixels. These parameters lead to the following modification of (5):

$$ \mathcal{J^{\prime}}_{2}=|S_{2}-I_{1}|^{\lambda}+\delta, ~~\mathcal{J^{\prime}}_{3}=|S_{3}-I_{1}|^{\lambda}+\delta. $$
(10)

Increasing the λ parameter is primarily useful for smooth images and the introduction of δ is beneficial as it prevents the so-called colour leakage that is sometimes visible in image regions that are far from a given scribble. The improvement of colourisation after applying the step penalty δ can be observed in Fig. 8.

Fig. 8
figure 8

Colour leakage reduction using the step penalty δ

Full size image

3.3 Improvement in computational speed

We noticed (as in [21]), that for most of the images, there is usually no need to mix more than three colours in order to achieve satisfactory results. Therefore, only the most significant colours, i.e. the closest to a given pixel, were considered in the colour blending phase defined by (9). Thus, the calculation of the isoline distance from a given scribble to each pixel (especially the one very far from the scribble) is not the optimal solution. To remove these redundant calculations we propose the following modification of our algorithm.

Instead of creating distance maps for each colour, we operate on three such maps - for the most prominent colour scribbles. Similarly, we use three chrominance arrays that hold the colour chrominances of the three closest scribbles. The seed arrays are also tripled. By doing this, the computation time and the memory usage are independent of the number of scribbles.

The first step of the algorithm, previously described by (5), is modified so that we can calculate the isoline distance from the two adjacent pixels and from each of their three closest scribbles γ 1, γ 2 and γ 3:

$$ \mathcal{J^{\prime}}_{2|\gamma_{k}}=|S_{2|\gamma_{k}}-I_{1}|^{\lambda}+\delta,~~\mathcal{J^{\prime}}_{3|\gamma_{k}}=|S_{3|\gamma_{k}}-I_{1}|^{\lambda}+\delta, $$
(11)

where \(\mathcal {J^{\prime }}_{2,\gamma _{k}}\) and \(\mathcal {J^{\prime }}_{3,\gamma _{k}}\) are the isoline distance estimations from the k-th colour scribble, \(S_{2|\gamma _{k}}\) and \(S_{3|\gamma _{k}}\) are the PSI values of pixels P 2 and P 3 from the seed array of the k-th scribble, (k = 1, 2, 3). This permits the total number of the six isoline distance estimates to be obtained rather than the two in the original algorithm. Moreover, we do not have to repeat the calculations for each scribble independently.

In the next step, we compare the isoline distances of adjacent pixels with the estimates described above, and apply the same conditional correction as in (6):

$$ \textrm{if } \mathcal{J^{\prime}}_{k|\gamma_{q}} < \mathcal{J}_{k|\gamma_{q}} \textrm{~~then~~} \mathcal{J^{\prime}}_{k|\gamma_{q}} \gets \mathcal{J}_{k|\gamma_{q}} , $$
(12)

where k = 2, 3 and q = 1, 2, 3.

At the end, we create an array which consists of six estimated distances and three own distances of the pixel P 1.The three smallest isoline distances are selected from this collection and their corresponding seed intensities and chrominances are transferred to pixel P 1.

In the final colour weighted averaging, there are always three colour candidates whose distance and chrominance can be obtained directly from the three arrays. Although this modification can easily be extended to a larger number of colours, we did not observe a significant improvement in the quality of the colourisation. It is sometimes actually better to reduce the number of colours to two. Very often, this is enough to achieve satisfactory colourisation results and such a simplification leads to a further improvement in the speed of the algorithm.

3.4 Double scan repetition

The double-scan algorithm is a very fast and a simple method to estimate distance in our application. However, it has one disadvantage. Due to the directions of the scan, it is impossible to accurately estimate the distance of sharply curved paths. This leads to considerable problems in the colourisation of more complicated shapes.

Repetition of the double scan can resolve this issue. We obtained very good results for all the colourised images after three runs of the double scan experimentally, and therefore it was selected as the default number of repetitions in all of the experiments (increasing the number of repetitions did not result in any further improvement in quality). However, we would like to mention, that this number may depend on the complexity of the image and the positioning of the scribbles. In order to use colourisation for the segmentation of curved objects, the required number of repetitions may vary and therefore it should be chosen experimentally (see the exemplary test in Fig. 9).

Fig. 9
figure 9

Repetition of the double-scan algorithm scanning

Full size image

4 Parameter optimisation

In the proposed method, a user can change three parameters. Parameter λ tunes the impact of any intensity changes on the colourisation results, δ is the step penalty, that reduces the colour leakage and σ is responsible for colour blending. We observed that for most grayscale images with intensities in the range of [0, 255], the last parameter can be set at σ = 10 without a drop of quality. This means that the operator only has to adjust two parameters.

Comparison of the colourisation results can be difficult because the reference colour image is usually unknown. However, to present the influence of the parameters on the colourisation quality, we converted the original colour images to a grayscale version by only taking the Y luminance channel, and then we colourised them with our algorithm. Because the reference is known, this procedure permits the utilisation of the colour image quality measures.

We applied the Peak Signal-to-Noise Ratio (PSNR) as it is one of the most widely used methods for image comparison. For colour images and 8-bit colour representation, PSNR is calculated using the mean square error (MSE) as follows:

$$\begin{array}{@{}rcl@{}} \textrm{MSE} = {\frac{1}{3NM}\sum\limits_{q\in \{R,G,B\}}\sum\limits_{i=1}^{N}\sum\limits_{j=1}^{M}[I^{q}(i,j)-{I_{C}^{q}}(i,j)]^{2}},\\ \textrm{PSNR}= 10~\log\left( \frac{255^{2}}{\textrm{MSE}}\right), \end{array} $$
(13)

where N,M are width and height of the image, q is the colour channel (R, G or B), I q(i,j) and \({I_{C}^{q}}(i,j)\) are the intensities of the pixel at position (i,j) in the reference and colourised image, respectively.

We chose the Normalised Colour Difference (NCD) [12, 17] operating in the L∗ a ∗ b ∗ colour space as the second indicator of the quality of the colourisation. This coefficient represents the distances between colours in a given colour space. Since the larger the NCD is, the worse the image quality is, we decided to visualise 1/NCD instead of NCD in order to improve the readability of presented optimisation plots. The NCD indicator is calculated using the following formula:

$$ \textrm{NCD} =\frac{\sum\limits_{i=1}^{N}\sum\limits_{j=1}^{M}\sqrt{\sum\limits_{q\in \{L^{*},a^{*},b^{*}\}}[I^{q}(i,j)-{I_{C}^{q}}(i,j)]^{2}}}{\sum\limits_{i=1}^{N}\sum\limits_{j=1}^{M}\sqrt{\sum\limits_{q\in \{L^{*},a^{*},b^{*}\}}I^{q}(i,j)^{2}}}. $$
(14)

The third quality measure is the Structural Similarity Index (SSIM) [18]. This coefficient is less concentrated on the intensity difference than the both of the previous ones. Rather, its results correspond very well with the human perception. Because the SSIM is only calculated in the local window (in our tests 5 ×5 pixels), we obtained the SSIM as the mean of SSIM values over the entire image. The SSIM can be calculated for a single colour channel, and therefore we averaged the results for each (R, G, B) in order to obtain the final similarity estimate of the colour image. For detailed formulas for calculating the structural similarity index, refer to [18].

In our studies we used five colour test images that represented different synthetic and real image types (see the first column in Fig. 10). They were scribbled and colourised using slightly different combinations of the parameters. The 3-D plots (see Fig. 11) present the results of the colourisation quality using the corresponding quality measures. We also summarise the optimal λ and δ settings in Table 1.

Fig. 10
figure 10

The test images used for the parameter optimisation and for the comparison with the state-of-the-art colourisation methods. From top: Balloons, Jupiter, Leaf, Pyramids and Puzzle

Full size image
Fig. 11
figure 11

Dependence of applied quality measures on the parameters λ and δ

Full size image
Table 1 Optimal parameter selection
Full size table

We observed that the optimal results were obtained with a combination of λ − δ parameters in the following range: λ ∈ (0.5, 2) and δ ∈ (0,2). We should also note that the surface plots for each parameter had very similar shapes, which indicate that the above range of the λ - δ combinations produces very similar and satisfactory quality of colourisation.

We performed test with more images and assessed the colourisation quality visually. We found that the most pleasing results were obtained for the combination of λ = 1 and δ = 1 which fits well the previously specified ranges. Therefore, this set of parameters values should be used as a default pair.

5 Comparison of the colourisation methods

We compared the results of the colourisation of the grayscale versions of the test images with three well established methods. We selected the ones, that are frequently cited and which, like to the presented algorithm, utilise the interactive colourisation that is based on user-indicated scribbles. For the first one - Colourisation Using Optimisation (CUO) [7] - we used the Matlab code that is available on-line. The two remaining algorithms were implemented in C# language: Chrominance Blending Colourisation utilizing Dijkstra’s algorithm (CBC) [21] and the Distance Transform based Colourisation (DTC) [6]. As no code is available for method [21], it was implemented in its basic version. For our method we used ISO abbreviation.

The grayscale versions of the test images were colourised using identical scribble arrangements as depicted in the middle column of Fig. 10. For our method and each test image we used the parameter combination that provided the highest PSNR (see Table 1). Figure 12 shows the outcomes of each method. We compared the quality of the colourisation using the quality metrics (QM) described previously. Table 2 presents the summary.

Fig. 12
figure 12

Results of colourisation using different algorithms

Full size image
Table 2 Comparison of colourisation quality
Full size table

The comparison that was performed proved the high quality of the isoline based colourisation results. The images that were colourised using the novel algorithm are of a similar, or sometimes even better, quality when compared with the other algorithms. Moreover, a visual examination confirmed that the proposed algorithm yields visually pleasing results.

However, it must be clearly stated that the quality of the colourisation of numerous methods (not only the ones selected in this evaluation) is very high and the results are are visually comparable. Moreover, the scribble configuration can, to some extent, influence the final results, thus changing the ”winners” in each of the images tested. Therefore, we would like to note that the main goal of our comparison was not to prove the superiority of the outputs obtained by the proposed method. Rather, it was conducted to illustrate that the quality of the colourisation quality is at the same, satisfactory level.

Because the fine details of the final result depend on the configuration of the scribbles and because the values of the parameters can be changed interactively by a user, the computational efficiency is of the highest priority. Furthermore, since the quality of the colourised images obtained using various methods is comparable, the algorithm speed should be treated as the most significant factor that has a direct influence on the time a user spends colourising.

In order to assess the computational burden of the described technique we used the simple test image depicted in Fig. 13. By changing the number of scribbles (three to nine scribbles) and the resolution of the grayscale image, we obtained the computational characteristics of the algorithms. The colourisation application with the competitive methods implemented worked in the Windows7Ⓒ environment (Intel Core i5-3470 CPU@3.2Ghz, 16GB RAM DDR3) and its process was set to the maximum priority. We turned off all unnecessary applications, thus the system background could not significantly affect the measurement of the computational time. Figures 14 and 15 present the corresponding results.

Fig. 13
figure 13

Chessboard image used for computational speed tests

Full size image
Fig. 14
figure 14

Dependence of the computation time on the number of image pixels

Full size image
Fig. 15
figure 15

Dependence of the computation time on the number of colour scribbles

Full size image

The tests proved the high computational efficiency of our framework. Isoline colourisation provided the lowest calculation times for every combination of the number of scribbles and image size. We would like to mention that all of the methods showed a linear dependency on the number of image pixels. This is consistent with the theoretical predictions presented in Section 3. The predominance of double-scan (ISO) over Dijkstra’s algorithm (CBC) was visible as was expected (see Fig. 6). The DTC algorithm also utilises the double-scan method, and therefore it is faster than CBC. However, in this colourisation, the double-scan was repeated for each colour scribble, while in our approach, it was limited to the three closest colours, which caused the difference in efficiency. The CUO algorithm utilises completely different colourisation approach (i.e. not based on distance maps) which appears to be significantly slower than the others.

The situation is different for the impact of the number of scribbles. Our isoline-based approach and CUO did not exhibit an increase in computational time with a growing number of scribbles. This is a very important advantage when colourising complex images, that require a greater number of scribbled details.

6 Applications

Colourisation can be applied in medicine, astronomy, biology and other disciplines. We should also consider colourisation for educational purposes as colour images are more attractive to a viewer who can benefit from such visualisation. The indication of some crucial regions within medical images may be of special interest to the physicians. Astrophotography can also take advantage of colourisation when capturing of colour images requires separate images obtained through RGB filters. With proper colourisation, such a procedure may be reduced to only single grayscale image photography. Figure 16 presents a few examples of the colourisation that was obtained using our algorithm.

Fig. 16
figure 16

Exemplary application of the proposed isoline colourisation technique

Full size image

7 Conclusion

In this work we presented a novel, very fast algorithm for image colourisation that is based on the introduced isoline concept. In our interactive colourisation method, a user indicates the colour scribbles in order to obtain the required colourisation result. By using a repeated double-scan algorithm, reducing the number of mixed colours and also by enabling parameter tuning, our method obtained a high accuracy and computational efficiency.

The method was compared with three well-known colourisation algorithms. The comparison included the aspects of computational complexity and the quality of the colourisation. We proved that our method can achieve satisfactory quality measured by objective fidelity metrics. The assessment of the results confirms that the outputs are comparable to, or sometimes better than, the results obtained using the competitive algorithms. Our method also showed a distinctively higher computational efficiency, which does not depend on the number of indicated scribbles. This is an important feature, which combined with the simplicity of its implementation, makes the proposed colourisation method very attractive.