Abstract
Appropriate color mapping for categorical data visualization can significantly facilitate the discovery of underlying data patterns and effectively bring out visual aesthetics. Some systems suggest predefined palettes for this task. However, a predefined color mapping is not always optimal, failing to consider users’ needs for customization. Given an input categorical data visualization and a reference image, we present an effective method to automatically generate a coloring that resembles the reference while allowing classes to be easily distinguished. We extract a color palette with high perceptual distance between the colors by sampling dominant and discriminable colors from the image’s color space. These colors are assigned to given classes by solving an integer quadratic program to optimize point distinctness of the given chart while preserving the color spatial relations in the source image. We show results on various coloring tasks, with a diverse set of new coloring appearances for the input data. We also compare our approach to state-of-the-art palettes in a controlled user study, which shows that our method achieves comparable performance in class discrimination, while being more similar to the source image. User feedback after using our system verifies its efficiency in automatically generating desirable colorings that meet the user’s expectations when choosing a reference.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Behrisch, M.; Blumenschein, M.; Kim, N. W.; Shao, L.; El-Assady, M.; Fuchs, J.; Seebacher, D.; Diehl, A.; Brandes, U.; Pfister, H.; Schreck, T.; Weiskopf, D.; Keim, D. A. Quality metrics for information visualization. Computer Graphics Forum Vol. 37, No. 3, 625–662, 2018.
Silva, S.; Sousa Santos, B.; Madeira, J. Using color in visualization: A survey. Computers & Graphics Vol. 35, No. 2, 320–333, 2011.
Harrower, M.; Brewer, C. A. ColorBrewer.org: An online tool for selecting colour schemes for maps. The Cartographic Journal Vol. 40, No. 1, 27–37, 2003.
Gramazio, C. C.; Laidlaw, D. H.; Schloss, K. B. Colorgorical: Creating discriminable and preferable color palettes for information visualization. IEEE Transactions on Visualization and Computer Graphics Vol. 23, No. 1, 521–530, 2017.
Lin, S.; Fortuna, J.; Kulkarni, C.; Stone, M.; Heer, J. Selecting semantically-resonant colors for data visualization. Computer Graphics Forum Vol. 32, No. 3pt4, 401–410, 2013.
Kim, H. R.; Yoo, M. J.; Kang, H.; Lee, I. K. Perceptually-based color assignment. Computer Graphics Forum Vol. 33, No. 7, 309–318, 2014.
Wang, Y.; Chen, X.; Ge, T.; Bao, C.; Sedlmair, M.; Fu, C. W.; Deussen, O.; Chen, B. Optimizing color assignment for perception of class separability in multiclass scatterplots. IEEE Transactions on Visualization and Computer Graphics Vol. 25, No. 1, 820–829, 2019.
Lin, S.; Ritchie, D.; Fisher, M.; Hanrahan, P. Probabilistic color-by-numbers. ACM Transactions on Graphics Vol. 32, No. 4, Article No. 37, 2013.
Lu, K. C.; Feng, M.; Chen, X.; Sedlmair, M.; Deussen, O.; Lischinski, D.; Cheng, Z.; Wang, Y. Palettailor: Discriminable colorization for categorical data. IEEE Transactions on Visualization and Computer Graphics Vol. 27, No. 2, 475–484, 2021.
Schlömer, T.; Heck, D.; Deussen, O. Farthest-point optimized point sets with maximized minimum distance. In: Proceedings of the ACM SIGGRAPH Symposium on High Performance Graphics, 135–142, 2011.
Healey, C. G. Choosing effective colours for data visualization. In: Proceedings of the 7th Annual IEEE Visualization, 263–270, 1996.
Trumbo, B. E. A theory for coloring bivariate statistical maps. The American Statistician Vol. 35, No. 4, 220–226, 1981.
Zeileis, A.; Hornik, K.; Murrell, P. Escaping RGBland: Selecting colors for statistical graphics. Computational Statistics & Data Analysis Vol. 53, No. 9, 3259–3270, 2009.
Bartram, L.; Patra, A.; Stone, M. Affective color in visualization. In: Proceedings of the CHI Conference on Human Factors in Computing Systems, 1364–1374, 2017.
O’Donovan, P.; Agarwala, A.; Hertzmann, A. Color compatibility from large datasets. ACM Transactions on Graphics Vol. 30, No. 4, Article No. 63, 2011.
Kita, N.; Miyata, K. Aesthetic rating and color suggestion for color palettes. Computer Graphics Forum Vol. 35, No. 7, 127–136, 2016.
Fang, H.; Walton, S.; Delahaye, E.; Harris, J.; Storchak, D. A.; Chen, M. Categorical colormap optimization with visualization case studies. IEEE Transactions on Visualization and Computer Graphics Vol. 23, No. 1, 871–880, 2017.
Lin, S.; Hanrahan, P. Modeling how people extract color themes from images. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 3101–3110, 2013.
Poco, J.; Mayhua, A.; Heer, J. Extracting and retargeting color mappings from bitmap images of visualizations. IEEE Transactions on Visualization and Computer Graphics Vol. 24, No. 1, 637–646, 2018.
Chang, H. W.; Fried, O.; Liu, Y. M.; DiVerdi, S.; Finkelstein, A. Palette-based photo recoloring. ACM Transactions on Graphics Vol. 34, No. 4, Article No. 139, 2015.
Tan, J. C.; Lien, J. M.; Gingold, Y. Decomposing images into layers via RGB-space geometry. ACM Transactions on Graphics Vol. 36, No. 4, Article No. 7, 2017.
Tan, J. C.; Echevarria, J.; Gingold, Y. Efficient palette-based decomposition and recoloring of images via RGBXY-space geometry. ACM Transactions on Graphics Vol. 37, No. 6, Article No. 262, 2018.
Aksoy, Y.; Aydin, T. O.; Smolić, A.; Pollefeys, M. Unmixing-based soft color segmentation for image manipulation. ACM Transactions on Graphics Vol. 36, No. 4, Article No. 19, 2017.
Zhang, Q.; Xiao, C. X.; Sun, H. Q.; Tang, F. Palette-based image recoloring using color decomposition optimization. IEEE Transactions on Image Processing Vol. 26, No. 4, 1952–1964, 2017.
Nguyen, R. M. H.; Price, B.; Cohen, S.; Brown, M. S. Group-theme recoloring for multi-image color consistency. Computer Graphics Forum Vol. 36, No. 7, 83–92, 2017.
Phan, H. Q.; Fu, H. B.; Chan, A. B. Color orchestra: Ordering color palettes for interpolation and prediction. IEEE Transactions on Visualization and Computer Graphics Vol. 24, No. 6, 1942–1955, 2018.
Setlur, V.; Stone, M. C. A linguistic approach to categorical color assignment for data visualization. IEEE Transactions on Visualization and Computer Graphics Vol. 22, No. 1, 698–707, 2016.
Lee, S.; Sips, M.; Seidel, H. P. Perceptually driven visibility optimization for categorical data visualization. IEEE Transactions on Visualization and Computer Graphics Vol. 19, No. 10, 1746–1757, 2013.
Bohra, M.; Gandhi, V. ColorArt: Suggesting colorizations for graphic arts using optimal color-graph matching. In: Proceedings of the Graphics Interface, 95–102, 2020.
Sharma, G.; Wu, W. C.; Dalal, E. N. The CIEDE2000 color-difference formula: Implementation notes, supplementary test data, and mathematical observations. Color Research & Application Vol. 30, No. 1, 21–30, 2005.
Stone, M.; Szafir, D. A.; Setlur, V. An engineering model for color difference as a function of size. In: Proceedings of the Color and Imaging Conference, 253–258, 2014.
Leordeanu, M.; Hebert, M.; Sukthankar, R. An integer projected fixed point method for graph matching and MAP inference. In: Proceedings of the 22nd International Conference on Neural Information Processing Systems, 1114–1122, 2009.
Bridson, R. Fast Poisson disk sampling in arbitrary dimensions. In: Proceedings of the ACM SIGGRAPH Sketches, 22-es, 2007.
Lecun, Y.; Bottou, L.; Bengio, Y.; Haffner, P. Gradient-based learning applied to document recognition. Proceedings of the IEEE Vol. 86, No. 11, 2278–2324, 1998.
Van der Maaten, L.; Hinton, G. Visualizing data using t-SNE. Journal of Machine Learning Research Vol. 9, No. 86, 2579–2605, 2008.
Shapira, L.; Shamir, A.; Cohen-Or, D. Image appearance exploration by model-based navigation. Computer Graphics Forum Vol. 28, No. 2, 629–638, 2009.
TinEye. Color extraction. 2021. Available at https://labs.tineye.com/color/.
Shugrina, M.; Kar, A.; Fidler, S.; Singh, K. Nonlinear color triads for approximation, learning and direct manipulation of color distributions. ACM Transactions on Graphics Vol. 39, No. 4, Article No. 97, 2020.
Acknowledgements
We thank the reviewers for their valuable comments and constructive suggestions. This work was supported in parts by National Natural Science Foundation of China (U2001206, 61872250), GD Talent Program (2019JC05X328), GD Natural Science Foundation (2020A0505100064, 2021B1515020085), DEGP Key Project (2018KZDXM058), and Shenzhen Science and Technology Key Program (RCJC20200714114435012, JCYJ20210324120213036).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
The authors have no competing interests to declare that are relevant to the content of this article.
Additional information
Qian Zheng received her doctoral degree in computer science from Shenzhen Institutes of Advanced Technology (SIAT), Chinese Academy of Sciences, in 2015. She is now a lecturer at Suzhou University of Science and Technology. Her interests include computer graphics and information visualization.
Min Lu is an assistant professor at Shenzhen University. She received her B.Sc. degree in computer engineering from Beijing Normal University, China, in 2011, and received her Ph.D. degree in computer science from Peking University in 2017. Her major research interests include visualization methodology and visual analytics.
Sicong Wu received his bachelor degree in computer science from Southwest University of Science and Technology in 2020. He is currently working towards a master degree in Shenzhen University. His research interests include visualization and visual analytics.
Ruizhen Hu is an associate professor at Shenzhen University. She received her Ph.D. degree from Zhejiang University. Previously, she spent two years visiting Simon Fraser University, Canada. Her research interests are in computer graphics, with a recent focus on applying machine learning to advance the understanding and generative modeling of visual data including 3D shapes and indoor scenes. She is an editorial board member of The Visual Computer and IEEE CG & A.
Joel Lanir is a senior lecturer and faculty member in the Information Systems Department at the University of Haifa, Israel, where he heads the Human-Computer Interaction Lab. He received his Ph.D. degree in computer science from the University of British Columbia in 2009. His research interests lie in the fields of human-computer interaction, ubiquitous computing, and information visualization.
Hui Huang is a distinguished professor at Shenzhen University, where she directs the Visual Computing Research Center. She received her Ph.D. degree in applied math from The University of British Columbia in 2008. Her research interests span computer graphics, computer vision, and visualization. She is currently a senior member of IEEE/ACM/CSIG, a distinguished member of CCF, and is on the editorial boards of ACM TOG and IEEE TVCG.
Electronic supplementary material
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.
The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.
About this article
Cite this article
Zheng, Q., Lu, M., Wu, S. et al. Image-guided color mapping for categorical data visualization. Comp. Visual Media 8, 613–629 (2022). https://doi.org/10.1007/s41095-021-0258-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41095-021-0258-0