1 Introduction

Since its invention in the nineteenth century, cinema has become one of the most important media of popular culture, becoming in fact, next to the books, our historical memory. Unfortunately, color films are subject to the aging of dyes contained in the emulsion and of the support itself. The decay is a natural process that usually introduces a color dominant, loss of contrast and/or color desaturation. Color negative and positive film, color print material, interpositives, and color motion-picture release print are all affected in the same way [29]. The decaying process is irreversible, and the most effective method to slow it is to store photographic and cinematographic material at controlled temperature and humidity [14].

It has been estimated that about 80 % of the movies produced between 1910 and 1920 have been lost [28]. If not restored, a large part of the film cultural heritage will be lost relatively quickly. Hence the strong ethical obligation to preserve and restore those moving images that represent our social memory.

The relevance of this problem is well-known, as stated by the European Broadcasting Union: “Differential dye fading is the most common image degradation phenomenon in archived colour films” [5]. Martin Scorsese, the famous movie director, has started several years ago the challenge of making people aware about the need of a wide film restoration movement: “Movies touch our hearts, and awaken our vision, and change the way we see things. They take us to other places. They open doors and minds. Movies are the memories of our lifetime. We need to keep them alive.” [39]. Since 1980, UNESCO has recognized “moving images” as an integral part of world’s Cultural Heritage [41].

Classic techniques of film restoration were based on the use of optical and photochemical tools directly on the original film material. These restoration techniques involved a significant human intervention and constant supervision of the work.

Thanks to the relevant technical progresses of the last years, the digital restoration of color has widely spread. Digital techniques allow an easier removal of color casts, equalization of the frame histogram, expansion of the dynamic range and adjustment of desaturated colors. Moreover, it can address artifacts not treatable by traditional photochemical techniques. Citing again the European Broadcasting Union: “Dye fading cannot be corrected by film treatment. A certain amount of fading is normally correctable in printing process but correcting for fading beyond 30  % in a film-to-film process is complex and uses specialized techniques” [5]. Finally, digital techniques allow easily multiple attempts, however, constant supervision by qualified personnel is still required. Moreover, a digital pipeline can address effectively the scaling of the restored content to the different formats and gamuts, necessary for the multiple different visualization devices currently available in cinemas.

Current state of the art in digital restoration is mainly based on color grading techniques: “primary” grading affects the levels of RGB, gamma, shadows and highlights in the entire frame, while “secondary” techniques apply changes only on a selected range of colors or windows in the frame.

In this paper, we collect and resume our personal experiences and results from the works and projects we have conducted in the last years in the digital film restoration field, and we describe the method used, based on the idea of recovering the appearance of color rather than the original color signal. This approach is based on the application of Spatial Color Algorithms (SCAs) [37], a family of algorithms inspired by the capabilities of the Human Vision System (HVS) of autonomously adjust to the variation of color and lightness in the scene.

In section 2 we present a description of the proposed approach. In sections 3 and 4 we introduce some of the SCAs we used for digital film restoration, and in section 5 we present a description of the proposed digital restoration pipeline. We conclude the paper presenting some examples of restoration we performed using the proposed approach (section 6), and a final discussion (section 7).

2 The proposed approach

In the elaboration of our proposal, we have started from some considerations about restoration criteria, requirements, and goals.

Film restoration is a complex, expensive and long process, and it is divided into technical and editorial activities. The technical activities are, e.g. : recognition and examination of the physical decay of the film, removal of damage and of chemical-physical defects, duplication and conservation. The editorial activity is based on the investigation of film history, reconstruction and recovery of missing parts and review of the editing. Usually, a preliminary historical analysis of the material to restore is performed, based on the research of information on space-time locations and subjects involved in the original movie production. Furthermore, a relevant contribution may be given by a careful analysis of all the available material related to the film (original or duplicate negatives, positive copies or cels-acetate sheets).

These two activities are strictly linked. Even if in this paper we will deal mainly with the technical restoration activity, each action performed on this side requires a historical/editorial strategy.

Current digital restoration techniques usually can be placed somewhere between two opposite approaches: on one side there is the conservative approach (i.e. the goal is to prevent further decay with a correct storage, without any correction), on the other side there is the high quality modern cinema approach, where all the possible issues are addressed using state of the art techniques, with the goal of achieving the best quality possible for the fruition with current visualization devices, overtaking the technological limits typical of the period the film was produced.

A trade-off solution may suggest to restore the defects staying as close as possible to the original. We may discuss in a broader way about what it must be considered “the original”. Must we consider original the movie as it was presented the first time to the public? Or is it the movie as it was intended in first stance by the director? Must we consider the original as the version at the best possible quality for the technology of its period? This last point is particularly relevant, because we have to consider that many devices, very different than the one used at the time of the movie production, are currently available for the digital fruition of movies, each with their own characteristics about gamuts, dynamics, illuminants, etc. All these considerations can affect the overall approach to film restoration.

In this paper, we propose an approach to digital color restoration, following the suggestion of E. Gombrich: “Restoration experts should be made aware that, in their responsible and difficult position, they should take into account not only the chemical aspects of the dyes, but also the psychology of perception […]. We are not asking them to restore the original dye colour, but something much more subtle and delicate: preserve ratios” ([11], translated from italian version).

Following the considerations made above about the unpractical possibility of having the same support and/or the same color gamut, gamma, and contrast as the original film, we follow the suggestion of Gombrich, proposing to restore each frame towards an estimate of the scene color appearance, instead of its original colors. Preserving appearance is achieved by means of algorithms simulating some mechanisms of HVS, in particular lightness constancy and color constancy. Lightness constancy enables perception of the scene robust to changes in the mean luminance intensity, while color constancy enables the perception of the scene robust to changes in the color of the illuminant. For HVS and its robust mechanisms, restoring the original appearance is more important than the fidelity to the original, that is its exact color reproduction. Following this approach, restoring film fading can be seen as a problem of chromatic noise removal, such as color constancy mechanisms [2, 3].

These algorithms are called Spatial Color Algorithms (SCAs) [37]. SCAs are almost unsupervised and based on a small set of parameters. For these reasons, SCAs can allow a great flexibility in the design of a digital film restoration pipeline.

The performance of these algorithms can be strongly dependent on the quantity and quality of the degradation of the input to restore. In some cases, the application of SCAs is the only step necessary for a satisfying restoration, while in other cases pre- and/or post-processing is needed to achieve the desired results. However, in these cases, SCAs filtering, considered as a kickoff process, shorten the rest of the work and, since unsupervised, contributes to the efficiency of the overall process.

3 Spatial color algorithms (SCAs)

The visual evidence at the basis of SCAs is that color sensation does not depend on the point-wise value of the stimulus, but on the spatial arrangement of the stimuli in the scene [20]. In fact, identical levels of radiance at a point can originate completely different color sensations according to the values in the rest of the image. Clear examples of this behavior are visual illusions [18, 19, 34]. To reproduce this behavior, SCAs perform spatial comparisons among the pixels in the input image. For this reason, SCAs are characterized by a high computational cost and a local and global behavior.

SCAs share a common structure of two phases. In the first phase, each pixel is recomputed from scratch according to the spatial distribution of the other pixels in the image (all, or some of them, as prescribed by each implementation). To form the output image, the matrix of a-dimensional values, spatially recomputed in the first phase, is scaled in the second phase onto the available output dynamic range, according to global anchoring principles and to the model goal. In this second phase, LUT and/or gamma non-linear adjustment can be added according to the SCA goal [7]. Due to this characteristic, SCAs can be suitable for High Dynamic Range (HDR) tone rendering, and/or for conversion of different dynamic range derived from changes of support, like e.g. gamut mapping or analog-to-digital dynamic range adjustment [19].

Among the many SCAs, we have chosen to consider algorithms derived from Retinex [16, 17]. Starting from the Retinex theory, during the last years, a range of alternative image appearance models has been implemented, like e.g. ACE [34], RSR [26], STRESS [15].

These SCAs process independently each band of the visible spectrum, determining for each channel the relative lightness values of the various areas of the scene. Every region of the image is compared with adjacent areas and the lightest local region found is used as local reference white. Details of these algorithms are presented in the following subsections. The order of presentation has no particular meaning.

3.1 RSR

The different Retinex algorithms proposed in literature [21, 30] are based mainly on a sampling or integrating approach. In the sampling approach, color correction is calculated in each pixel using a subset of samples in its neighborhood, while in the integrating approach, color elaboration is based on an averaged value of the data in the surround. In the sampling-based implementations, the neighborhood of a pixel is often investigated using paths into the image, starting from the considered pixel. According to the way pixel computing is scheduled across the path, the implementations can range from the classic Retinex [16], that updates each pixel along the path, to the MI-Retinex [21], that updates the pixel only at the end of each path.

RSR (Random Spray Retinex) [26] is an efficient MI-Retinex implementation, based on the results of the mathematical analysis published in [25]. In this analysis, the authors have proved the redundancy of the MI-Retinex path-wise approach, demonstrating that, along a path, only the pixel with the highest value is relevant for the lightness calculation. As a consequence, they have proposed a simplified reformulation, where paths are replaced by two-dimensional point sampling (called sprays) across the image: in each pixel p t , N sprays with n pixels are cast randomly using polar coordinates with uniform sampling. The spray results to be more dense near the target pixel p t than far away, enclosing the idea of locality [26] (see Fig. 1).

Fig. 1
figure 1

Example of RSR spray

Full size image

The typical ratio-reset operation [16] along each path is substituted by the search of the pixel with highest intensity in each spray, and by a single division with the original intensity of pixel p t . The new value p new in each channel is obtained averaging the contribution of all the ratios of the various sprays. Eq. 1 resumes RSR computation [25, 26], where N is the number of sprays applied to pixel p t , and s max i is the pixel with highest intensity in the spray i.

$$ {p}_{new}=\frac{1}{N}{\displaystyle \sum_{i=1}^N}\;\frac{p_t}{s_i^{max}} $$
(1)

3.2 STRESS

STRESS (Spatio-Temporal Retinex-inspired Envelope with Stochastic Sampling) [15] uses the same spray approach of RSR, but with an alternative technique that exploits two envelopes to characterize the local visual context. It calculates for each pixel not only the local maximum (reference white) like RSR, but also the local minimum (reference black), for each chromatic channel. Formally the envelopes are functions such that the image signal is always in between the envelopes or equal to one of them.

The maximum s max i and minimum s min i values of a spray S are calculated in this way:

$$ {s}_i^{max}=\underset{p_j\in S}{max}{p}_j $$
(2)
$$ {s}_i^{min}=\underset{p_j\in S}{min}{p}_j $$
(3)

Since p t is always one of the sample points, s min i  ≤ p t  ≤ s max i . The range r i of the samples and the relative value v i of the center pixel p t are calculated as:

$$ {r}_i={s}_i^{\max }-{s}_i^{\min } $$
(4)
$$ {v}_i=\left\{\begin{array}{ccc}\hfill 0.5\hfill & \hfill \mathrm{if}\hfill & \hfill {r}_i=0\hfill \\ {}\hfill \left({p}_t-{s}_i^{\min}\right)/{r}_i\hfill & \hfill \hfill & \hfill \mathrm{else}\hfill \end{array}\right. $$
(5)

The new value p new is obtained by averaging the values v i over N sprays:

$$ {p}_{new}=\frac{1}{N}{\displaystyle \sum_{i=1}^N}\;{v}_i, $$
(6)

In formula 5, in case of uniform areas, STRESS sets v i equal to 0.5. This implements a Gray Word normalization principle [1]. A classic case for a movie is during fade-in or fade-out, and it is always possible to substitute the value 0.5 with the original pixel value using the KOG function (see section 4).

A STRESS implementation is included in the GNU Image Manipulation Program GIMP, in the GEGL library starting from version 2.8.

3.3 ACE

The ACE (Automatic Color Equalization) algorithm [34, 35] shares many background principles with Retinex-based SCAs algorithms; however, it applies them differently, and considers also other equalization mechanisms, not included in classic Retinex methods.

In ACE, every pixel p t in the image I is compared to every other pixel independently in the RGB channels. The pixel to pixel comparison is made by arithmetical difference. This difference is modified by a non linear function r(⋅), and every comparison is then weighted by a function d(⋅), based on the distance between pixels. All these contributions are summed to get the final value. To make the weighting meaningful a normalization factor k t is used. The main formula is:

$$ {p}_{new}=\frac{1}{k_t}{\displaystyle \sum_{p_j\in I,{p}_j\ne {p}_t}}\; r\left({p}_t-{p}_j\right) d\left( t, j\right) $$
(7)
$$ {k}_t={\displaystyle \sum_{p_j\in I,{p}_j\ne {p}_t}}\; d\left( t, j\right) $$
(8)

where d(t, j) is the inverse of the Euclidean distance between pixels p t and p j . The function r(⋅) is the truncated gain function:

$$ r\left({p}_t-{p}_j\right)=\left\{\begin{array}{ccc}\hfill -1\hfill & \hfill \mathrm{if}\hfill & \hfill \left({p}_t-{p}_j\right)\le - thr\hfill \\ {}\hfill \left({p}_t-{p}_j\right)/ thr\hfill & \hfill \mathrm{if}\hfill & \hfill - thr<\left({p}_t-{p}_j\right)< thr\hfill \\ {}\hfill 1\hfill & \hfill \mathrm{if}\hfill & \hfill \left({p}_t-{p}_j\right)\ge thr\hfill \end{array}\right. $$
(9)

The r(⋅) function is a non-linear amplification of the normalized difference between pixel values. This non-linear amplification causes different spatial rearrangements of the pixel values according to the slope and consequently to the point of threshold thr (see Fig. 2).

Fig. 2
figure 2

ACE r(⋅) function

Full size image

A combination of ACE and RSR, called RACE, has also been designed [27].

3.4 Choosing and tuning SCAs

SCAs perform an almost unsupervised elaboration of the frames, based on a small set of parameters. The correct choice of these parameters depends on the input image, but a set of possible initial values can speed-up their fine tuning.

Regarding RSR and STRESS, a default starting value for the number of spray per pixel (N) can be chosen between 20 and 40, and the starting value for the number of points in each spray (n) can be chosen between 100 and 800. Both parameters may vary according to the size and content of the image. However, the algorithm has been proven to be very robust to relative variations of these parameters. In the examples of Fig. 3, very similar results have been obtained with N = 20, n = 200 (Fig. 3b and c) and N = 200, n = 20 (Fig. 3e and f). The criterion to fulfill is to have an adequate sampling around the pixel to compute, however, in order to reduce the filtering time, lower values can be chosen, leading to a higher noise.

Fig. 3
figure 3

Filtering examples using RSR, STRESS and ACE with different parameters settings

Full size image

ACE filtering effect is mainly dependent on the slope value of the r(⋅) function. Visually, it results in an increment/decrement of the perceived contrast: the higher the slope, the higher the contrast (see Fig. 3d and g). A discussion about ACE parameters tuning for movie restoration can be found in [4, 33]. A starting value for the contrast slope can be chosen around 20.

In Fig. 3, we show different filtering examples of the presented SCAs. From the images it can be noticed the differences among the three SCAs and the effect of different parameters settings.

3.5 Speeding up SCAs

SCAs are in general computationally expensive. Their algorithmic complexity varies between O(NlogN) and O(N 2), where N is number of pixels of the image. To overcome this problem, two approaches are possible: smarter software implementation, and/or efficient parallel processing.

An example of software speed-up is Local Linear Look Up Table (LLLUT) [8], that strongly reduces the computational time, at the cost of a relatively small loss of quality in the filtering process. In LLLUT, the SCA is applied to a subsampled version of the original image and, in a second step, local interpolation functions are determined in order to propagate the color computation effect to all the pixels of the original image, keeping the local filtering effect.

Moreover, the nature of SCAs computation is very well suited to be parallelized on many-cores or multi-cores architectures, like the modern GPUs. GPU implementations of SCAs are almost straightforward using high-level programming APIs like CUDA or OpenCL: the main difficulties are focused on the optimization of data transfer and access through GPU memory hierarchy.

In Fig. 4, we show a comparison (in logarithmic scale) between the computational times of a CPU and GPU implementations of the ACE algorithm (section 3.3). The CPU version has been implemented in C++, and the computational times are relative to single-core execution on a Intel Xeon Quad Core E5-2609 2.40GHz under Windows 7 operating system. The GPU version has been implemented using OpenCL API on a NVIDIA Quadro 4000, hosted on the same workstation. On the considered image sizes, the gain factor using GPU computation is between 10 and 80x (average gain factor 45x).

Fig. 4
figure 4

Comparison between computational performances of CPU and GPU implementations of ACE algorithm. The computational times are shown using a logarithmic scale

Full size image

With these methods, SCAs computational cost problem can be overcome, making them more applicable also to the processing of high resolution contents at high frame rates.

4 SCAs features for movie restoration

SCAs are widely used for general-purpose enhancement of digital images. However, in digital movie restoration, there are cases in which the editing of artistic choices of the director may have introduced some characteristics in the frames (like, e.g., fade-in and fade-out, a color dominant, or a reduced dynamic range) that should not be removed by the application of the SCA.

Therefore, specific functions for digital movie restoration need to be added to the original SCAs implementation, in order to address these particular cases in an efficient way. These functions are introduced in the second phase of SCAs filtering, i.e., when the output of the first stage is scaled onto the available output dynamic range.

4.1 Keep original gray (KOG)

The KOG function is used in order to not change the tonal characteristics of a low or high key image, avoiding the histogram average to move from the original one. It preserves the original average values (independently in R, G and B channels) instead of centering the chromatic channels around the medium gray. This function is important for restoring of fade-in and fade-out sequences.

In order to keep the original gray, the following proportion must be respected for each color channel:

$$ \frac{p^{avg}-{p}^{min}}{p^{max}}=\frac{g^{avg}-{g}^{min}}{g^{max}} $$
(10)

where p avg, p max and p min are the average, maximum and minimum values of the input image I, and g avg, g max and g min are the average, maximum and minimum values of the output dynamic range. Thus, the output gray value g avg is:

$$ {g}^{avg}=\frac{g^{max}}{p^{max}}\cdot \left({p}^{avg}-{p}^{min}\right)+{g}^{min} $$
(11)

This value is then used in the final scaling phase of SCAs, shifting the output histogram towards the original input average.

4.2 Keep original color cast (KOC)

Sometimes, movie directors introduce unnatural color in a sequence for artistic purposes (like, e.g., in the night scenes of “Nosferatu”, 1922, directed by F. W. Murnau [22]) . Even if in the first phase of SCAs the color cast is corrected, it is possible to perform a preliminary estimation of the color of the cast, and then to reintroduce it again in the second phase. Noticeable color cast can be easily estimated for each RGB channel c using the histogram distribution hist c , and finding the smallest l c that satisfies the following equation:

$$ \frac{{\displaystyle {\sum}_{l=0}^{l_c}}\; his{t}_c(l)}{{\displaystyle {\sum}_{l=0}^{l evel{ s}_c}}\; his{t}_c(l)}\ge 0.95 $$
(12)

where levels c is the maximum numbers of levels in each RGB channel of the output dynamic range. Again, this factor can be introduced in the scaling phase re-applying the original color cast present in the input frames.

4.3 Keep original dynamic range (KODR)

Sometimes in movies the directors decide to use only a part of the available dynamic range in order to obtain specific visual effects. Since SCAs tend to stretch the histogram in order to maximize the dynamic range, the KODR function has been designed: in the second phase of SCAs, the final scaling is applied onto the limited original input dynamic range, and not onto the whole available one.

5 Description of the proposed pipeline

SCAs are devised to work on single frames, and since their filtering depends on frame content, they need parameters setting. Within the same shot, since frame content is not supposed to change remarkably, the same set of parameters can be applied. Thus, a digital restoration pipeline based on the application of SCAs can be divided in these steps:

  • Shot detection

  • Key Frames extractions

  • SCAs parameters tuning on keyframes

  • Unsupervised filtering of each shot

  • Assessment and final corrections

  • Final re-editing and harmonization stage

5.1 Shot detection

The first stage regards the subdivision of the movie into shots (a continuous sequence of frames taken over a short period of time). Automatic techniques for shots subdivision are usually based on the detection of abrupt changes and fades. The case of abrupt changes is usually addressed by techniques based on frame difference measurements, computed from histograms and textures descriptors [36]. These techniques may be also modified to cope with fades [6, 36]. The results obtained by these techniques can be submitted for evaluation to a decision module, to address conflicting results.

5.2 Key frames extractions

The following stage copes with the extraction of relevant frames that summarize the content of each shot. These key frames will be the input of the tuning stage, for the assessment of the parameters of the SCAs.

This stage may be performed manually, but different methods can be used for an automatic selection. One of the possible approaches is to choose the first and/or the last frame in the shot as the key frame [38, 40]. Other approaches include time sampling of shots at regular intervals. As an alternative approach [38], the video is time sampled regardless of shot boundaries. Other approaches compute the differences between consecutive frames in a shot using color histograms, or other visual descriptions [12, 13], to measure the visual complexity of the shot; the key frames are selected by analyzing the values obtained. In [10, 42] the frames are classified in clusters, and the key frames are selected from the larger clusters, or by hierarchical clustering reduction.

5.3 SCAs parameters tuning on key frames

As presented in section 3, every SCA has a set of parameters. To run SCAs, initial values of these parameters must be chosen. Usually, the parameters tuning is not a critical step [32]. All the SCAs presented in this paper have a set of parameters values that leads to acceptable results on a very wide set of input images. Is the restoration expert (the curator) who helps to fine tune the initial parameters in order to maximize the filtering results according to the restoration strategy.

In this stage the features for movie restoration, presented in the section 4, can also be set as restoring parameters, according to the desired result.

5.4 Assessment and final corrections (if required)

Judging SCA results is not straightforward, since an absolute metric does not exist. The curator makes the final judgment about the restoration results of each set of frames. He/she has the responsibility of deciding if the SCA with the chosen parameters has produced the desired result. In many cases, it happens that the automatic SCA filtering produces a frame closer to the desired result, but that still requires little adjustment in e.g. chroma or contrast. These final retouches are made by the technician under the curator supervision, usually with standard correction tools that work globally on the frame, or on windows or levels selections. In some cases, a SCA filtering can be applied again after an extra global correction, in order to obtain a desired supplementary local effect. This latter case can be carried out according to the interesting property of SCAs that do not over-filter the image if its overall appearance does not require it [37].

5.5 Final re-editing and harmonization stage

Once all the frames have been filtered, they are rejoined in the same editing sequence, and synchronized with the audio track.

Since filtering result depends on frame content, and since frame content may vary among shots frames, this can result in changes of average lightness, producing unwanted flickering. Thus, a final smoothing stage can be required in the process of re-editing the filtered frames.

6 Restored movies and results

During the last 10 years, this approach has been tested on several sequences and single frames from many video typologies (movies, ads, documentaries, cartoons).

In this paper, we present examples and tests of restoration taken from three movies completely restored with the proposed method. The aim of the paper is not to perform any comparison among the presented SCAs, since all of them are suitable for the goal. Therefore, for each movie we present results obtained with different SCAs.

6.1 “Il racconto della Stura”

The first example concerns the color restoration of “Il racconto della Stura” (“The story of Stura”) [24], a 8 min and 51 seconds long documentary by Ermanno Olmi filmed in 1955, which documents the building in the valley of Stura di Demonte (a city in north-west Italy) of a hydroelectric plant by the Italian electric company. The movie has been kindly provided in digital format from the National Industry Film Archive of Ivrea (Italy). The frames analysis has revealed a non-uniform magenta color cast, and in Figs. 5 and 6 are presented some results of the application of ACE (section 3), with slope set at 80. For the final scaling, a linear mapping has been applied.

Fig. 5
figure 5

Restoration example of a frame from “Il racconto della Stura”. On the first column: an original frame (top), the frame restored using ACE (bottom). On the second column: luminance histograms. On the third column, polar plots of hues

Full size image
Fig. 6
figure 6

Restoration example of a frame from “Il racconto della Stura”. On the first column: an original frame (top), the frame restored using ACE (bottom). On the second column: luminance histograms. On the third column, polar plots of hues

Full size image

6.2 “La lunga calza verde”

The second example is the restoration of the animated short “La lunga calza verde” (The long green sock) [9]. This work was realized in 1960, directed by Roberto Gavioli, screenplay by Cesare Zavattini and drawings by Paolo Piffarerio, produced by Gamma Film and Istituto Luce. The short movie was kindly provided in digital format from the Luigi Micheletti Foundation in Brescia (Italy). We have worked on a DVD copy (MPEG-2) in standard resolution PAL (720 × 576). The initial analysis of the material has shown a moderate loss of contrast, color desaturation and color dominants on several frames. The original cels have been used as potential color reference but they have suffered of a considerable degradation due to the nature of the support and of the dyes. We show in Figs. 7 and 8 two restoration examples based on the application of STRESS (section 3), with 200 sprays, each composed by 20 sample points. For the final scaling, a linear mapping has been applied.

Fig. 7
figure 7

Restoration example of a frame from “La lunga calza verde”. On the first column: an original frame (top), the frame restored using STRESS (bottom). On the second column: luminance histograms. On the third column, polar plots of hues

Full size image
Fig. 8
figure 8

Restoration example of a frame from “La lunga calza verde”. On the first column: an original frame (top), the frame restored using STRESS (bottom). On the second column: luminance histograms. On the third column, polar plots of hues

Full size image

6.3 “La ciudad en la playa”

The third example is the restoration of “La ciudad en la playa” (The city on the beach) [23], a documentary directed by Ferruccio Musitelli in 1961, commissioned by Uruguay Ministry of Tourism. The short movie was kindly provided in digital format from Ferruccio and Rodolfo Musitelli. We have worked on a DV copy (M-JPEG) in standard resolution PAL (720 × 576). The analysis of the material has shown a moderate loss of contrast, noticeable color desaturation and a small decrement of the dynamic range. In Figs. 9 and 10 are presented restoration examples based on the application of RSR (section 3), with 200 sprays, each composed by 20 sample points. For the final scaling, a linear mapping has been applied.

Fig. 9
figure 9

Restoration example of a frame from “La ciudad en la playa”. On the first column: an original frame (top), the frame restored using RSR (bottom). On the second column: luminance histograms. On the third column, polar plots of hues

Full size image
Fig. 10
figure 10

Restoration example of a frame from “La ciudad en la playa”. On the first column: an original frame (top), the frame restored using RSR (bottom). On the second column: luminance histograms. On the third column, polar plots of hues

Full size image

7 Discussion and results

The digital restoration of color and dynamic is usually a manual or semi-automatic delicate operation performed by experts. In this paper, we propose an unsupervised or semi-unsupervised method for digital color, contrast and dynamic restoration. It can be used for a complete restoration, or as an unsupervised kickoff followed by further manual refinements. The proposed approach is based on a family of image enhancement algorithms called SCAs (for Spatial Color Algorithms), inspired by the dynamic behavior of the Human Visual System. The advantage of these algorithms is that they adjust automatically the strength of correction according to the enhancement required by the input. In this paper we present three of these SCAs, and some results of movie restoration. Moreover, we describe in details the pipeline to use them together with special features designed for movie restoration. The proposed method has been used to restore several movies and test examples. A demo reel can be seen at [31].