Saliency-based image correction for colorblind patients

Improper functioning, or lack, of human cone cells leads to vision defects, making it impossible for affected persons to distinguish certain colors. Colorblind persons have color perception, but their ability to capture color information differs from that of normal people: colorblind and normal people perceive the same image differently. It is necessary to devise solutions to help persons with color blindness understand images and distinguish different colors. Most research on this subject is aimed at adjusting insensitive colors, enabling colorblind persons to better capture color information, but ignores the attention paid by colorblind persons to the salient areas of images. The areas of the image seen as salient by normal people generally differ from those seen by the colorblind. To provide the same saliency for colorblind persons and normal people, we propose a saliency-based image correction algorithm for color blindness. Adjusted colors in the adjusted image are harmonious and realistic, and the method is practical. Our experimental results show that this method effectively improves images, enabling the colorblind to see the same salient areas as normal people.


Introduction
With the development of printing and screen display technology, color has become an important means of information exchange in people's daily life. Information exchange through color is convenient for ordinary people but is more of a challenge for those with color vision defects (CVD). CVD can cause various changes in color perception, from slight shadows to severely indistinguishable colors. CVD are of three main kinds: red-green color blindness, blue-yellow color blindness, and complete color blindness. Most CVD are congenital, affecting approximately one in every 40,000 newborn babies. However, these defects may also be caused by abnormal development of cone cells in the retina, or external damage. Currently, there are at least 200 million persons worldwide with dichromatic CVD, of which approximately 3.5% are school students. The prevalence of generalized CVD is even more difficult to assess. CVD are easily overlooked because many people think that these problems are not serious. However, most types of color blindness make it very difficult to distinguish, e.g., changes in traffic lights, which is very dangerous. Many other areas can be affected by these defects, such as the ability to read traffic signs and maps. To ensure the wellbeing and safety of colorblind persons, the problem of color blindness should not be ignored.
Solutions to the problem of CVD are of two main kinds: treatment of the person, and augmentation of images. Regarding the treatment of persons with CVD, certain physical therapies are generally used to reduce the effects of CVD and complications, but CVD often cannot be cured. For example, in one type of advanced treatment, a red light is used to stimulate the color perception of the cone cells. However, the success rate of this treatment is only 35%. Imagebased CVD solutions start outside the human body, processing images by computer and presenting the results to the colorblind person. Such methods avoid pain for the person and have high feasibility.
There are many methods for overcoming CVD based on images. In color scheme adjustment [1][2][3], brightness differences can be used to realize automatic color conversion.
Ref. [4] used an ordinary stereo display (non-autostereo display) to provide colorblind persons and ordinary users with two visual experiences, so that CVD and normal vision viewers can share the same content. A wearable vision improvement system [5] can enhance a person's Ishihara vision test score with enhanced mechanical equipment and a head-mounted display. Color filters [6] have been used for auxiliary color recognition for the colorblind, moving color filters [7][8][9] in front of the person's eyes to achieve rapid conversion between film observation and naked eye observation. For example, if you move the green film, the green color is not affected, but the red transmittance is reduced, which forms a region where the brightness continuously changes, helping the colorblind person to discriminate colors. Another static filter [10] converts the diffuse state of the color into an identifiable state. A more advanced method [11] uses a Heilmeier-type liquid crystal device to control the transmittance of green light by changing the voltage to achieve synchronous flashing of green objects.
An auxiliary color filter device transmits different colors through a film, thereby forming a color difference, but this method is only used for a certain colorblind tube, and different standards are required for different types of color blindness. It may not be usable in all cases. Wearable visual improvement technology is effective in improving the vision of persons with color vision defects, but the need to carry and wear the necessary equipment may be inconvenient.
People pay attention to the salient regions when looking at an image, helping them to quickly understand the information in the image. Because of lack of correct cone cell function, the color distribution in a image seen by the colorblind and normal people are different. Vivid colors for standard vision may no longer be vivid in the eyes of the colorblind. When normal and colorblind people see the same image, the color distribution in the image appears differently to them, so the salient regions may differ. To overcome this issue, and to achieve the same perception from colorblind and normal viewers, this paper proposes an approach which determines a salient region under normal vision and uses a simulated colorblind image to correct the salient region of the image as viewed by a colorblind person. See Fig. 1.
The main contributions of our method are as follows: • We determine the salient region of an image under normal vision, and use it to correct the salient region of a simulation of an image as seen by a colorblind subject. • We retain the original color of the image to highlight the salient area. Adjusted colors in the adjusted image are harmonious and realistic, and the method is practical.

Color vision defects
Human color perception is based on three lightsensitive pigments [12,13] and may be represented by a three-dimensional value, the power of each wavelength specifying the degree of the color stimulus. Trichromatic accuracy is determined by three types of light-sensitive pigment cells (L-, M-, and S-cones) in the retina. Different wavelengths of light stimulate the receptors differently. For example, yellow-green light stimulates L-cones and M-cones to the same degree, but only weakly stimulates S-cones. The human visual system combines information from various cone cells in response to light of different wavelengths. When any kind of cone cell is destroyed or loses function [14], color vision defects occur. CVD can be divided into color blindness and color weakness, and described as trichromacy (color weakness), bichromacy (color blindness), or monochromacy (full-color blindness). Persons with full-color blindness have lost functionality of two or three kinds of cone-cell and can perceive the intensity of light [15] but cannot distinguish colors; they see the world in black and white. Certain colorblind persons with bichromacy have lost red, green, or blue cone functions. Certain persons with trichromacy have functioning cones of all three kinds, but one of the photosensitivity spectra is shifted, thus causing a deviation in color perception.

Color vision defects simulation
Colorblind persons have color perception, but the abilities of these persons to capture color information differ from those of normal people, as shown in Fig. 2. Images can be adjusted so that people with CVD can better understand them. Image-based color vision defect-assisted processing requires CVD simulation; previous researchers have conducted many studies on simulated chromatic aberration [16,17]. Brettel [16] proposed a method of simulating redgreen blindness. According to the responses of three types of light-sensitive pigments cells, RGB color is represented as a vector in LMS color space. However, the algorithm makes many assumptions, and its applicability is limited. According to the theory of human color vision, MacHado et al. [18] simulated color through electrophysiology and handled normal color perception, trichromatic vision, and two-color vision. Chen et al. [19] devised a method to derive mapping relationships between images as seen by the colorblind and normal images to ensure that contrast is provided between each pair of representative colors. Okajima and Kanbe [20] improved brightness differences by defining a model based on personal color vision. Flat and Gutwin [21] considered all factors affecting vision and proposed a case-specific color difference model. Kuhn et al. [22] proposed an efficient and automatic two-color image recoloring technique that highlights important visual details. In addition, there are ready-made applications such as ColorDoctor.
Unlike the research of these previous investigators, this paper uses a deep convolutional neural network method to convert a normal image into a "colorblind image", i.e., a simulation of an image as experienced by a colorblind person. Normal images are input into the convolutional neural network. Colorblind images are used for training so that the generator generates near-real colorblind images.

Saliency detection
With the development of digital cameras and smart phones, the number of images has exploded. Saliency detection finds the area in a complex image that attracts attention, allowing a viewer to quickly determine the content of the image. The purpose of salient object detection methods is to highlight salient areas in an image and their development is relatively recent. Initially, non-deep learning models based on low-level features [23][24][25] relied on features such as image color contrast [26]. In order to obtain prominent objects with clear boundaries, methods such as superpixels [27] were usually also incorporated into the model. A more detailed overview is given in Ref. [28]. Since 2015, methods based on deep learning have emerged, from earlier saliency score processing units that extract features based on multilayer perceptron (MLP) classifiers [29,30], to salient object detection architectures based on fully convolutional networks (FCN) [31][32][33]. In addition, salient object detection methods may refine the detected salient objects into individual instances [34,35].
In addition to research on static scenes, salient object detection in dynamic scenes has also flourished: e.g., Ref. [36] has contributed to research on video saliency. Salient object detection can not only detect salient objects, but also has a wide range of applications, including unsupervised video object segmentation [37], semantic segmentation [38,39], automatic image cropping [40], image target retargeting [41], and other fields.
In this paper, saliency detection is performed to find the salient area. Traditional contrast-based object detection methods have limitations, overemphasizing the edges of the salient object without uniformly distributing saliency across the salient object. When the current experience is similar to the background, saliency detection errors arise. In order to alleviate these problems, this paper adopts a diffusion-based method [42], using the diffusion matrix and seed vector for salient area detection.

Saliency-based image processing
When we take a photo on a mobile phone, we sometimes find that an unimportant object is too prominent, distracting from the original intent. Faced with this kind of problem, we may wish to take another shot or fix the photo later. However, it is complicated to enhance specific objects in a photo using an image editor. It is necessary to consider the exposures of specific objects, reduce the background exposure, and increase saturation and background blur. For those lacking basic knowledge of color matching, this method is difficult. To solve this problem, we can learn the parameters of an image using a network and adjust the contrast of the target area, and other image parameters. This approach of improving the apparent contrast of an object by manipulating the color of the image is called attention redirection [43,44]. In addition to enhancing salience, object enhancement can be achieved through this idea [45], guiding viewers to pay attention to mixed reality [46] and color redirection [47] or augmenting images by removing the background [48], emphasizing the background [49], or changing image aesthetics [50]. Through attention-focused operations, objects of interest are made more prominent, and it is easier for people with color defects to find prominent objects.
The goal of attention redirection is to select the target area and correct this area to achieve target enhancement. Various methods are compared in Ref. [43]. Certain methods enhance objects in such a way that the enhanced color is unrealistic, e.g., resulting in a purple apple or a blue snake. Another patch-based approach [51] replaces patches in the target area with other patches from the same image. This replacement is not simple patch exchange. Based on the saliency of the image, an appropriate patch is selected for replacement, thereby enhancing the target area of the image to achieve a significant effect.

Colorblind image correction
Currently, there is no effective treatment for color blindness. In recent years, with the development of technology, various colorblind image re-coloring algorithms have appeared, providing color correction for colorblind images. Because of the defects in cone cells of colorblind persons, some colors are unrecognizable, so the main purpose of colorblind color correction is to help colorblind persons distinguish certain colors, rather than helping colorblind persons recognize certain colors. In order to do so, increasing the color contrast between different colors is the main approach to colorblind image correction.
These methods may be mainly divided into four categories: LMS space conversion, colorblind filters, Lab color correction, and shifting color algorithms. LMS space conversion is a more comprehensive method which converts the image from RGB space to LMS space, processes the image in LMS space, and re-converts the processed image to RGB space. The CBFS algorithm compares the color of each pixel in the image with red or green, and processes the approximately red or green pixels, so that a colorblind person can better perceive the image. This correction algorithm is aimed at protanopia. Lab color correction converts RGB images into Lab space, and modifies the sensitive red or green color for green blind persons, thus achieving color correction of colorblind images. This method only corrects for deuteranopia. The shifting color method is aimed at tritanopia. It converts the image to Lab space, adjusts the image brightness, and then adjusts and converts back to RGB color space to achieve color correction.
The above methods have various problems, such as unnatural coloring, artifacts in coloring results, and slowness. Huang et al. [52] proposed a more efficient image re-coloring algorithm. A Gaussian mixture model (GMM) is used to represent color information.
Extracted key colors are weighted to take into account the sensitivity of colorblind persons to different colors. During interpolation, color is interpolated according to the posterior probability of the Gaussian and the corresponding map, thereby ensuring smoothness of local colors in the image after re-coloring, effectively reducing artifacts.

Colorblind image saliency detection
Because of the defects in cone function, colorblind persons are unable to recognize or distinguish certain colors. When a normal and a colorblind person see the same image, the former is attracted to bright red, while the latter's attention is concentrated in other areas. If they discuss this image, they concentrate on different aspects.
For most color images, the color seen by CVD persons differs from that of normal people. Both the vividness of the color will be greatly reduced, and the information transmitted by the image itself will also change. Figure 3 shows that after converting a color image to a colorblind image, the salient area of the image changes. Originally it had bright orange petals and green leaves, but after CVD simulation, the petal color and leaf color changed, becoming similar. The resulting saliency detection may be deficient, with the significant area seen by a colorblind person being different that seen by a normal person. To the main purpose of the algorithm in this paper is to overcome this problem, so that colorblind persons and normal people may focus on common salient areas.

Proposed method
Persons with CVD cannot recognize certain colors due to deficiencies in the three kinds of light-sensitive pigment cells. We first use a CVD model to perform CVD simulation (Section 4.1), perform saliency detection on the standard image (Section 4.2), then use the saliency map to perform saliency correction on the colorblind image (Section 4.3), and finally inversely transform the corrected colorblind image into a standard color image (Section 4.4). The goal is that both persons with CVD and normal persons should pay attention to the same salient area in the corrected image. The algorithm pipeline is shown in Fig. 4.

Colorblind image simulation
Tanuwidjaja et al. [5] proposed a wearable augmented reality system based on Google Glass. For each color, the RGB color space is converted into LMS space. Using Eq. (1), the converted chromaticity automatically adapts to the scene viewed according to the type of color blindness. There are special algorithms for reproducing color. Chroma's wearable augmented reality device is an effective digital accessory that enhances vision in everyday activities to help colorblind persons recognize different colors.
Machado et al. [18] proposed a physiological model for simulating color perception based on electrophysiological research report: human color vision can be simulated to achieve normal color vision, abnormal trichromatic vision, and dichroic vision. This method simulates a colorblind image using Eq. (2): where Γ consists of the basic color sets WS(λ), YB(λ), RG(λ). Lin et al. [53] convert an RGB image to λ, Y-B, R-G color space via LMS space, using the CIECAM02 model, as below: (3) However, there is a certain amount of distortion in this method, and a method of recoloring the colorblind image is proposed.
The λ, Y-B, R-G spatial images are subjected to CVD simulation to obtain a difference image, the feature vector is extracted from the simulated image, and the color of the image after CVD simulation is altered so that a CVD person can perceive the color information better.
The simulated colorblind image can also be inversely transformed into an RGB image, using Color constraints are added to minimize the distortion caused by spatial transformation, and the altered color is converted from λ, Y-B, R-G space to RGB space [53], to achieve recoloring.
The aim during color transformation is to preserve color information in the original image as much as possible so that the recolored image is as natural as possible, and the whole image is harmonious in color. When recoloring colorblind images, most methods are based on experimental criteria. The designer uses a color scheme to reduce the limited CVD color palette. However extensive effort is still necessary to choose a color that is visually friendly to the CVD viewer. Moreover, these methods cannot be applied to existing natural images. Semiautomatic restoration of the image to accommodate color blindness provides the user with certain parameters to adjust the color map. However, the results are sensitive to the choice of parameters, and inappropriate parameters can cause the images to look unnatural.

Saliency map
First, we abstract the image to obtain a super-pixel form of the image. Each pixel can be regarded as a node V i in a graph G. The weight of edge e ij linking adjacent nodes V i and V j is used to represent the relationship between the nodes, where the node represents attributes in the image. The k-means algorithm is used to cluster the super-pixels, and the foreground and background are obtained from the tightness of the cluster. The saliency map is calculated according to the foreground and background seeds obtained, and then all saliency maps are obtained by the diffusion method. The calculated foreground is integrated with the superpixel saliency map of the background to produce a super-pixel-level saliency map.
Using the diffusion method, a sparse graph is constructed from the graph nodes and the neighbors of each node. After considering neighboring nodes, the largest common edge of the node is included in the statistical range. In this method, a two-layer sparse graph is constructed that effectively uses local spatial relationships in the image and removes redundant nodes as much as possible. We use the edge weight to represent the relationships between the nodes, as below: After transforming from RGB color space to Lab space, l i and l j represent the super-pixel means of the corresponding nodes v i and v j , and σ is the tradeoff parameter.
The nodes in the super-pixel are clustered into k classes by the k-means algorithm with cluster centers C = |c 1 , · · · , c k |. For each cluster, the similarity between nodes and clusters is determined using the method of popularity ranking [55]: where A indicates the similarity between v i and c i , W is the weight set for each side, D represents the sum of the weights on all node branches, H = [h ij ] N ×K is the similarity matrix after the diffusion process, and α is a constant. Using the similarity between nodes and clusters, the significance of nodes in the region is determined according to their probability; they are divided into significant nodes and background nodes. The salient node-set and the background node-set are called FG and BG, respectively. The nodes in FG form a saliency map according to (7) where q(i, j) is the normalization of the node weights, and p(j) is the probability that v j is a salient node. The formula shows that as the node weight w ij connected to the super-pixel node v i becomes larger, or d jj for pixel v j becomes larger, the larger q(i, j) becomes, and the greater the probability that v j is a salient node.
Referring to the saliency map construction method for the salient region, the saliency map for the background node is formed by The diffusion process for salient and background nodes ends, forming a super-pixel saliency cluster and a background cluster. These parameters are S FG and S BG , respectively, and the two parts complement each other to form a saliency map for the entire image.

Saliency driven color correction
To perform color correction in a region, we adjust the significant contrast of the target area to the rest of the image. We replace the patch block of the target area with the patch block outside the target area. The corrected image J is generated by inputting the target area mask R and the green blind image I, and the saliency contrast ΔS. The saliency of J is represented by S J . First, we select two patch datasets of size n × n in the input image. The patches are classified into significant patches D + = {q; S I (q) τ + } and background patches D − = {q; S I (q) τ − } where τ − , τ + is a threshold. The patch is manipulated by defining an energy function to enhance the significant patches, and the background patches are weakened. The energy function is defined as where D(q, p) is the sum of the squared distances between the patches q, p in Lab color space, E ∇ is the gradient of the original image I, and λ is used to weight the Lab color space and the original image.
To obtain a colorblind image I with saliency contrast S J , Eq. (9) is minimized so that the more prominent the patch in R is, the less significant the area outside R is. Therefore, by defining (10) where β top = 20%, the function can calculate the difference in saliency between the pixels in the target region R and the external pixels and further obtain the minimized energy term based on the saliency: Through greedy search, the patch library is continuously updated, as is the threshold, thereby updating the entire image and realizing color correction and thus enhancement of the salient region of the colorblind image. Experimental results are shown in Figs. 6(a) and 6(e).

Inverse transformation
The method in Section 4.3 achieves saliency correction of colorblind images, but our ultimate goal is to obtain a saliency corrected standard color image. Therefore, the corrected colorblind image must be inversely transformed into a standard color image. Conversion between normal and colorblind images is a visual problem that can be addressed by convolutional neural networks. For example, Ref. [56] used a generative adversarial network to generate realistic shadows for a source object. Unlike in traditional methods, we determine the mapping relationship between two different kinds of images through the network to perform conversion of colorblind images to color images.
Since the problem is an image-to-image conversion problem, the convolutional neural network is trained with the idea of confronting the network. We assume that the normal image is the X domain and the colorblind image is the Y domain. Our target is G yx : Y → X. A colorblind image is input to the network, and a standard color image corresponding to the image is output. As shown in Fig. 5(a), the network is divided into two branches. In order to facilitate unified training, a cyclic consistent loss function [54] is used to connect the two branches into a large cyclic network. The CycleGAN structure model performs unsupervised image style transfer, using a relatively simple network structure.
The generator and discriminator are unchanged in structure. In this paper, in order to enable the generator to learn more deep features, we use a combination of up-sampling, a residual block, and down-sampling: see Fig. 5(b). Due to the fast convergence and strong feature extraction capability of the residual network, high quality images are generated. The generator uses two down-sampling convolutional layers with a step size of 2, six residual blocks, and two up-sampling convolutional layers with a step size of 1/2. Training takes 200 epochs.
In this network, the forward loop network and the backward loop network are trained using the same loss function by using a cycle consistency loss function in two separate countermeasure networks. The training goal for the discriminator for a single generator in the forward network is to make the image generated by the generator as unrecognizable as possible to the discriminator, thereby maximizing the discriminator. The forward loss is min G xy max D y L GAN (G xy , D y , X, Y ), the loss function corresponding to the backward network is min G yx max D x L GAN (G yx , D x , X, Y ), and the cycle consistency loss function is where E( * ) denoted expected value, p data (x) is the target distribution of the X domain, and p data (y) is the target distribution of the Y domain.
The forward network loss function, the backward network loss function, and the cycle consistency loss function are accumulated in three parts. The overall loss function of the network is given by In Fig. 5, the corrected colorblind image is regarded as a Y domain, input into the network, and a corrected standard color image is output. In the backward network, the Y domain image is the input, and Y → G yx → G yx (Y ) → G xy → y ≈ Y , thereby realizing conversion of the colorblind image to a normal image: see Figs. 6(b) and 6(f). This completes saliency correction of the colorblind image.
The overall method flow is given in Algorithm 1.

Experiments
In order to prove the effectiveness of the proposed method, images processed by our method are compared to unprocessed images. Test images contain flowers, fruit, pedestrians, and natural scenery. Figure 7 shows saliency driven image correction for deuteranopia, while Fig. 8 shows saliency driven image correction for protanopia. The subfigures in Fig. 7 are referred to as 1-15, while subfigures in Fig. 8 are referred to as 16-30. Columns from left to right are the original image, saliency detection results for the original image, the colorblind simulation image, the saliency detection result for the colorblind images, the colorblind images with significant area enhancement, the inversely transformed saliency enhanced colorblind images (back to standard color), colorblind simulations of the restored color images, and significance detection in that inversely transformed image.
Our experimental results show that after converting a color image into a colorblind image, the salient Algorithm 1 Saliency-based image correction for color blindness Input: Normal color image Inorm and CVD simulation image of normal image I 1. Input: Image Inorm, Image I, and saliency contrast ΔS 2. Output: Manipulated and corrected image E Make a saliency map of the salient nodes and the background nodes according to Eqs. (7) and (8) 9. Object mask R = Norm(S fg + S bg ) 10. while ψ (S J , R) − ΔS > ε * do 11. (i) Update patch library 12.
Minimize Eq. (9) 15. end while 16. Fine-scale refinement 17. When τ + and τ − no longer change, the iteration ends, and a color-corrected colorblind image J is obtained. 18. J inversely transforms a colorblind image into a color image E through a convolutional neural network.
* Norm(·) is a function that normalizes its argument to a range of 0 to 1.
region of the image may move, or the size of the salient region may be reduced, making the salient region less noticeable. Our algorithm performs color correction on the salient region of the image to help colorblind persons to better perceive the image and understand the information conveyed by the image. Unlike traditional color correction algorithms, the focus of this paper is not to correct the colorblind image as a whole, but to focus on color correction of the salient region of the image, so that colorblind persons and normal people share a common salient region. Traditional colorblind color correction algorithms mainly adjust those colors that colorblind persons cannot distinguish. Colors in the the adjusted image are more vivid and the contrast is more obvious, which effectively helps colorblind persons to distinguish colors. But can color correction algorithms help colorblind persons to better understand images and capture the salient region of images?
To answer this question, we compare our method with the color correction algorithm proposed by Huang et al. [52]. Experimental results are shown in Fig. 9 (for green blind images) and Fig. 10 (for red blind images). Columns from left to right show: the original image, the salient area of the original image, the color corrected image using our method, the CVD simulation image corresponding to the corrected image, the saliency map for the CVD simulation, color corrected image using Huang's algorithm, its CVD simulation image, and its corresponding saliency map. Comparing the second and fourth columns in Figs. 9 and 10, it can be seen that the color correction algorithm can help persons with color vision defects to effectively distinguish colors, but the conventional color correction algorithm cannot provide comprehensive perception of the image for colorblind persons.

Qualitative analysis
The experimental results provide qualitative comparisons for different kinds of images. In the experiment, after the original images were subjected to CVD simulation, the vivid color information in the original images was lost to varying degrees. For example, red and green turned gray or grayish-brown under the vision of a colorblind person. Therefore, for normally visually bright colors, colorblind persons severely lose color information, and objects become less noticeable. In this case, the colorblind person misjudges the true salient area. A comparison between the second and fourth columns of Figs. 7 and 8 confirms this notion.
In this paper, saliency-based correction of colorblind images is achieved using the salient region of the original image and the colorblind image for   saliency, to obtain a corrected salient region, as in the fifth columns of Figs. 7 and 8. The corrected color strongly contrasts with the color of the rest of the image and is significantly more notable. However, this corrected color depends on the entire image. It is uncontrollable, and the color of the corrected portion does not match the hues of the rest of the image and is unrealistic. Therefore, the color-corrected colorblind image is inversely transformed into a color image.
Since the correction of the salient area color is based on the overall color of the image, if color correction is performed directly on the original image, the corrected color may still be color-insensitive red, so color correction has no effect. In order to avoid this phenomenon, color correction is performed on the colorblind image, and the corrected colorblind image is inversely transformed into a normal color image. The sixth columns of Figs. 9 and 10 show the effect of color correction directly on the original image. The color-corrected image has problems such as incomplete coloring and artifacts. The colorcorrected image is subjected to CVD simulation, and the simulated result is significantly detected; the detection result differs largely from the real image.
Comparing the second and eighth columns of the experimental results shows that the salient regions are basically consistent, demonstrating the effectiveness of our method. For images with blurred image boundaries and backgrounds, after colorblind simulation, the salient area and the background area are more difficult to distinguish. For colorblind persons, the salient area cannot be correctly identified. The saliency-driven color correction method proposed in this paper is resimulated. The colorblind image is compared with the colorblind simulation image of the original image. As shown in Fig. 7 and the third and seventh columns of Fig. 8, the salient areas have changed to different degrees, making the areas more obvious and original. In comparison to the CVD simulation results (see the fourth and eighth columns of Figs. 7 and 8) the proposed method rediscovers the significant area that was originally ignored, making the colorblind image saliency test result more accurate.

Error
The root means square (RMS) is used to measure the difference between the two images, and the significance of the experimental results is evaluated to measure the deviation between the observed value and the true value. RMS is defined by where a represents the value in the experimental result image and b represents the actual value in the image.
Images in Figs. 7 and 8 are numbered 1-15 and 16-30, respectively. RMS and mean absolute error results are shown in Table 1 for our method and for the original image subject to CVD simulation. Table 2 compares our algorithm with the traditional color correction algorithm.
The RMS data in Tables 1 and 2 are analyzed in Fig. 11: the lower the RMS value, the closer the detected saliency map is to the original image, indicating that the detection result is more accurate. The left graph in Fig. 11 gives RMS results for the deuteranopia image. The RMS values for the proposed method are lowest, followed by the colorblind color correction method. The RMS values of the directly simulated CVD image are higher, and the detected saliency map differs significantly from the original image. On the right are the corresponding results for the protanopia image. The traditional color correction method improves perception for the colorblind person, but does not achieve satisfactory results.
The MAE data in Tables 1 and 2 are analyzed in Fig. 12. On the left is the RMS result for the deuteranopia image. The MAE value for our method is lower and its distribution is relatively stable. Huang's method has large fluctuations and is unstable. Errors for the direct CVD simulation image are high. On the right are corresponding results for the protanopia image.

PR curve and F-measure
To evaluate the salient regions detected by our methods herein, a P-R curve is used. P (precision) represents the accuracy, the ratio of correctly detected significant pixels to all pixels. R (recall) represents the ratio of detected salient pixels to true salient pixels. The gray images output in this paper are classified according to 0-255, and a total of 256 recall pairs are used to draw the PR curves. Results are shown in Fig. 13; (a) and (c) compare the method proposed in this paper with the direct CVD simulation method, while (b) and (d) compare it   with the Huang's color correction algorithm. It is clear that for a given recall rate, our method is more accurate than direct detection of colorblind images, and than Huang's algorithm.
In most cases, the PR curve does not provide a comprehensive assessment of the significance of the image. Therefore, this paper uses the F-measure for comprehensive evaluation, defined as F-measure = 1 + γ 2 Precision × Recall γ 2 × Precision + Recall (15) where γ = 0.32 is a parameter. Figure 14 shows F-measure for the salient region for protanopia and deuteranopia, computed by our method. In most cases the value is above 0.65. Red-green blind simulation is performed by selecting different styles of images, and the simulated images are subjected to saliency-based color correction using our method. The higher the F-measure, the closer the detected salient region is to the salient region of the original image.

Conclusions
We have proposed a saliency-based image correction method that performs color correction on the salient region of the image to achieve saliency correction for colorblind images. In this paper, the method of adversarial networks is used to inversely transform the corrected colorblind image into a color image in agreement with the input. After this conversion, the color image is again subject to CVD simulation; we performed saliency detection on these simulated images, with results basically consistent with the saliency of standard images. The experimental results show that our proposed method effectively improves the ability of persons with visual defects to capture image salient regions. In future work, we wish to explore saliency correction in video.