Abstract
A collage is a composite artwork made from the spatial layout of multiple pictures on a canvas, collected from the Internet or user photographs. Collages, usually made by skilled artists, involve a complex manual process, especially when searching for component pictures and adjusting their spatial layout to meet artistic requirements. In this paper, we present a visual perception driven method for automatically synthesizing visually pleasing collages. Unlike previous works, we focus on how to design a collage layout which not only provides easy access to the theme of the overall image, but also conforms to human visual perception. To achieve this goal, we formulate the generation of collages as a mapping problem: given a canvas image, first, compute a saliency map for it and a vector field for each sub-region of it. Second, using a divide-and-conquer strategy, generate a series of patch sets from the canvas image, where the salient map and the vector field are used to determine each patch’s size and direction respectively. Third, construct a Gestalt-based energy function to choose the most visually pleasing and orderly patch set as the final layout. Finally, using a semantic-color metric, map the picture set to the patch set to generate the final collage. Extensive experimental and user study results show that this method can generate visual pleasing collages.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Wang, J. D.; Quan, L.; Sun, J.; Tang, X. O.; Shum, H. Y. Picture collage. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 347–354, 2006.
Battiato, S.; Ciocca, G.; Gasparini, F.; Puglisi, G.; Schettini, R. Smart photo sticking. In: Adaptive Multimedia Retrieval: Retrieval, User, and Semantics. Lecture Notes in Computer Science, Vol. 4918. Boujemaa, N.; Detyniecki, M.; Nürnberger, A. Eds. Springer Berlin Heidelberg, 211–223, 2008.
Yu, Z. Q.; Lu, L.; Guo, Y. W.; Fan, R. F.; Liu, M. M.; Wang, W. P. Content-aware photo collage using circle packing. IEEE Transactions on Visualization and Computer Graphics Vol. 20, No. 2, 182–195, 2014.
Bianco, S.; Ciocca, G. User preferences modeling and learning for pleasing photo collage generation. ACM Transactions on Multimedia Computing, Communications, and Applications Vol. 12, No. 1, Article No. 6, 2015.
Han, X. T.; Zhang, C. Y.; Lin, W. Y.; Xu, M. L.; Sheng, B.; Mei, T. Tree-based visualization and optimization for image collection. IEEE Transactions on Cybernetics Vol. 46, No. 6, 1286–1300, 2016.
Lee, M. H.; Singhal, N.; Cho, S.; Park, I. K. Mobile photo collage. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition — Workshops, 24–30, 2010.
Mei, T.; Yang, B.; Yang, S. Q.; Hua, X. S. Video collage: Presenting a video sequence using a single image. The Visual Computer Vol. 25, No. 1, 39–51, 2009.
Liu, L.; Zhang, H.; Jing, G.; Guo, Y.; Chen, Z.; Wang, W. Correlation-preserving photo collage. IEEE Transactions on Visualization and Computer Graphics Vol. 24, No. 6, 1956–1968, 2018.
Kim, J.; Pellacini, F. Jigsaw image mosaics. ACM Transactions on Graphics Vol. 21, No. 3, 657–664, 2002.
Lee, H. Y. Automatic photomosaic algorithm through adaptive tiling and block matching. Multimedia Tools & Applications Vol. 76, No. 22, 24281–24297, 2017.
Pavić D.; Ceumern, U.; Kobbelt, L. GIzMOs: Genuine image mosaics with adaptive tiling. Computer Graphics Forum Vol. 28, No. 8, 2244–2254, 2009.
Goferman, S.; Tal, A.; Zelnik-Manor, L. Puzzle-like collage. Computer Graphics Forum Vol. 29, 459–468, 2010.
Huang, H.; Zhang, L.; Zhang, H. C. Arcimboldo-like collage using Internet images. In: Proceedings of the SIGGRAPH Asia Conference, Article No. 155, 2011.
Nan, L. L.; Sharf, A.; Xie, K.; Wong, T. T.; Deussen, O.; Cohen-Or, D.; Chen, B. Conjoining Gestalt rules for abstraction of architectural drawings. Journal of Computer-Aided Design & Computer Graphics Vol. 30, No. 6, 1–10, 2012.
Qi, Y. G.; Guo, J.; Li, Y.; Zhang, H. G.; Xiang, T.; Song, Y. Z. Sketching by perceptual grouping. In: Proceedings of the IEEE International Conference on Image Processing, 270–274, 2013.
Qi, Y. G.; Song, Y. Z.; Xiang, T.; Zhang, H. G.; Hospedales, T.; Li, Y.; Guo, J. Making better use of edges via perceptual grouping. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1856–1865, 2015.
Zhang, J. H.; Chen, Y. L.; Li, L.; Fu, H. B.; Tai, C. L. Context-based sketch classification. In: Proceedings of the Joint Symposium on Computational Aesthetics and Sketch-Based Interfaces and Modeling and Non-Photorealistic Animation and Rendering, Article No. 3, 2018.
Lun, Z. L.; Zou, C. Q.; Huang, H. B.; Kalogerakis, E., Tan, P.; Cani, M. P.; Zhang, H. Learning to group discrete graphical patterns. ACM Transactions on Graphics Vol. 36, No. 6, Article No. 225, 2017.
Liu, C.; Rosales, E.; Sheffer, A. StrokeAggregator: Consolidating raw sketches into artist-intended curve drawings. ACM Transaction on Graphics Vol. 37, No. 4, Article No. 97, 2018.
Judd, T.; Ehinger, K.; Durand, F.; Torralba, A. Learning to predict where humans look. In: Proceedings of the IEEE 12th International Conference on Computer Vision, 2106–2113, 2009.
Bylinskii, Z.; Judd, T.; Borji, A.; Itti, L.; Durand, F.; Oliva, A.; Torralba, A. Mit saliency benchmark. 2015. Available at http://saliency.mit.edu/.
Patterson, G.; Xu, C.; Su, H.; Hays, J. The SUN attribute database: Beyond categories for deeper scene understanding. International Journal of Computer Vision Vol. 108, Nos. 1–2, 59–81, 2014.
Cheung, V. Shape collage. 2007. Available at http://www.shapecollage.com/.
Acknowledgements
This work was supported by the National Natural Science Foundation of China (No. 61772440), the Aeronautical Science Foundation of China (No. 20165168007), and Science and Technology of Electro-optic Control Laboratory.
Author information
Authors and Affiliations
Corresponding author
Additional information
Zuyi Yang received his bachelor degree in the Artificial Intelligence Department, Xiamen University, China, in 2018. Currently he is a master candidate in Xiamen University. His research interests include computer graphics and layout optimization.
Qinghui Dai received his master degree in the Artificial Intelligence Department, Xiamen University, in 2018. Currently he is a technology staff member of Tencent. His research interest is in computer graphics.
Junsong Zhang received his Ph.D. degree in computer science from the State Key Lab of CAD&CG, Zhejiang University, Hangzhou, China, in 2008. He is currently an associate professor at the Mind, Art and Computation Group, Artificial Intelligence Department, Xiamen University. His main research interests include computer graphics, human computer interaction, and cognitive science.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.
The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.
About this article
Cite this article
Yang, Z., Dai, Q. & Zhang, J. Visual perception driven collage synthesis. Comp. Visual Media 8, 79–91 (2022). https://doi.org/10.1007/s41095-021-0226-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41095-021-0226-8