Abstract
We present a novel method to produce discernible image mosaics, with relatively large image tiles replaced by images drawn from a database, to resemble a target image. Compared to existing works on image mosaics, the novelty of our method is two-fold. Firstly, believing that the presence of visual edges in the final image mosaic strongly supports image perception, we develop an edge-aware photo retrieval scheme which emphasizes the preservation of visual edges in the target image. Secondly, unlike most previous works which apply a pre-determined partition to an input image, our image mosaics are composed of adaptive tiles, whose sizes are determined based on the available images in the database and the objective of maximizing resemblance to the target image. We show discernible image mosaics obtained by our method, using image collections of only moderate size. To evaluate our method, we conducted a user study to validate that the image mosaics generated present both globally and locally appropriate visual impressions to the human observers. Visual comparisons with existing techniques demonstrate the superiority of our method in terms of mosaic quality and perceptibility.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Wikipedia-contributors. Photographic mosaic. Available at https://doi.org/en.wikipedia.org/wiki/Photographic_mosaic.
Silvers, R.; Hawley, M. Photomosaics. Henry Holt and Co., Inc., 1997.
Kim, J.; Pellacini, F. Jigsaw image mosaics. ACM Transactions on Graphics Vol. 21, No. 3, 657–664, 2002.
Di Blasi, G.; Gallo, G.; Petralia, M. P. Smart ideas for photomosaic rendering. In: Proceedings of the Eurographics Italian Chapter Conference, Vol. 2006, 267–272, 2006.
Orchard, J.; Kaplan, C. S. Cut-out image mosaics. In: Proceedings of the 6th International Symposium on Non-Photorealistic Animation and Rendering, 79–87, 2008.
Pavić, D.; Ceumern, U.; Kobbelt, L. GIzMOs: Genuine image mosaics with adaptive tiling. Computer Graphics Forum Vol. 28, No. 8, 2244–2254, 2009.
Miller, J.; Mould, D. Accurate and discernible photocollages. In: Proceedings of the 8th Annual Symposium on Computational Aesthetics in Graphics, Visualization, and Imaging, 115–124, 2012.
Zhang, L.; Ma, K.-L.; Yu, J. Adaptively tiled image mosaics utilizing measures of color and region entropy. In: Proceedings of the 9th International Symposium on Visual Information Communication and Interaction, 122–129, 2016.
Battiato, S.; Di Blasi, G.; Farinella, G. M.; Gallo, G. Digital mosaic frameworks—An overview. Computer Graphics Forum Vol. 26, No. 4, 794–812, 2007.
Dobashi, Y.; Haga, T.; Johan, H.; Nishita, T. A method for creating mosaic images using Voronoi diagrams. In: Proceedings of the Eurographics, Vol. 2, 341–348, 2002.
Faustino, G. M.; de Figueiredo, L. H. Simple adaptive mosaic effects. In: Proceedings of the XVIII Brazilian Symposium on Computer Graphics and Image Processing, 315–322, 2005.
Zhang, L.; Yu, J. Image mosaics with irregular tiling. In: Proceedings of the 12th International Conference on Computer-Aided Design and Computer Graphics, 155–162, 2011.
Lai, Y.-K.; Hu, S.-M.; Martin, R.-R. Surface mosaics. The Visual Computer Vol. 22, Nos. 9–11, 604–611, 2006.
Hausner, A. Simulating decorative mosaics. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, 573–580, 2001.
Liu, Y.; Veksler, O.; Juan, O. Generating classic mosaics with graph cuts. Computer Graphics Forum Vol. 29, No. 8, 2387–2399, 2010.
Battiato, S.; Di Blasi, G.; Gallo, G.; Guarnera, G. C.; Puglisi, G. Artificial mosaics by gradient vector flow. In: Proceedings of the Eurographics (Short Papers), 53–56, 2008.
Wu, H.; Lyu, X.; Wen, Z. Automatic texture exemplar extraction based on global and local textureness measures. Computational Visual Media Vol. 4, No. 2, 173–184, 2018.
Barnes, C.; Zhang, F. L. A survey of the state-of-the-art in patch-based synthesis. Computational Visual Media Vol. 3, No. 1, 3–20, 2017.
Barnes, C.; Zhang, F.-L.; Lou, L.; Wu, X.; Hu, S.-M. Patchtable: Efficient patch queries for large datasets and applications. ACM Transactions on Graphics Vol. 34, No. 4, Article No. 97, 2015.
Di Blasi, G.; Gallo, G.; Petralia, M. Puzzle image mosaic. In: Proceedings of the IASTED/VIIP, 33–37, 2005.
Kwan, K. C.; Sinn, L. T.; Han, C.; Wong, T. T.; Fu, C. W. Pyramid of arclength descriptor for generating collage of shapes. ACM Transactions on Graphics Vol. 35, No. 6, Article No. 229, 2016.
Huang, H.; Zhang, L.; Zhang, H.-C. Arcimboldo-like collage using internet images. ACM Transactions on Graphics Vol. 30, No. 6, Article No. 155, 2011.
Reinert, B.; Ritschel, T.; Seidel, H.-P. Interactive by-example design of artistic packing layouts. ACM Transactions on Graphics Vol. 32, No. 6, Article No. 218, 2013.
Zou, C.; Cao, J.; Ranaweera, W.; Alhashim, I.; Tan, P.; Sheffer, A.; Zhang, H. Legible compact calligrams. ACM Transactions on Graphics Vol. 35, No. 4, Article No. 122, 2016.
Hu, S.-M.; Zhang, F.-L.; Wang, M.; Martin, R. R.; Wang, J. PatchNet: A patch-based image representation for interactive library-driven image editing. ACM Transactions on Graphics Vol. 32, No. 6, Article No. 196, 2013.
Zhang, F.-L.; Wang, J.; Shechtman, E.; Zhou, Z.-Y.; Shi, J.-X.; Hu, S.-M. PlenoPatch: Patch-based plenoptic image manipulation. IEEE Transactions on Visualization and Computer Graphics Vol. 23, No. 5, 1561–1573, 2017.
Rother, C.; Bordeaux, L.; Hamadi, Y.; Blake, A. AutoCollage. ACM Transactions on Graphics Vol. 25, No. 3, 847–852, 2006.
Yu, Z.; Lu, L.; Guo, Y.; Fan, R.; Liu, M.; Wang, W. Content-aware photo collage using circle packing. IEEE Transactions on Visualization and Computer Graphics Vol. 20, No. 2, 182–195, 2014.
Goferman, S.; Tal, A.; Zelnik-Manor, L. Puzzle-like collage. Computer Graphics Forum Vol. 29, No. 2, 459–468, 2010.
Oliva, A.; Torralba, A.; Schyns, P. G. Hybrid images. ACM Transactions on Graphics Vol. 25, No. 3, 527–532, 2006.
Chu, H.-K.; Hsu, W.-H.; Mitra, N. J.; Cohen-Or, D.; Wong, T.-T.; Lee, T.-Y. Camouflage images. ACM Transactions on Graphics Vol. 29, No. 4, Article No. 51, 2010.
Tong, Q.; Zhang, S.-H.; Hu, S.-M.; Martin, R. R. Hidden images. In: Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Non-Photorealistic Animation and Rendering, 27–34, 2011.
Farbman, Z.; Fattal, R.; Lischinski, D.; Szeliski, R. Edge-preserving decompositions for multi-scale tone and detail manipulation. ACM Transactions on Graphics Vol. 27, No. 3, Article No. 67, 2008.
Muja, M.; Lowe, D. G. Flann: Fast library for approximate nearest neighbors. 2009. Available at https://doi.org/www.cs.ubc.ca/research/ann/.
Dalal, N.; Triggs, B. Histograms of oriented gradients for human detection. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 1, 886–893, 2005.
Zitnick, C. L.; Dollár, P. Edge boxes: Locating object proposals from edges. In: Computer Vision — ECCV 2014. Lecture Notes in Computer Science, Vol. 8693. Fleet, D.; Pajdla, T.; Schiele, B.; Tuytelaars, T. Eds. Springer Cham, 391–405, 2014.
Zhou, B.; Lapedriza, A.; Khosla, A.; Oliva, A.; Torralba, A. Places: A 10 million image database for scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 40, No. 6, 1452–1464, 2018.
Rapid-Mosaic. Foto-mosaic-edda. Available at https://doi.org/fmedda.com/en/home.
Acknowledgements
We thank the anonymous reviewers and the editors for their valuable comments. This work was supported in part by the National Natural Science Foundation of China (Nos. 61602310, 61522213, and 61528208), Guangdong Science and Technology Program (No. 2015A030312015), Shenzhen Innovation Program (Nos. JCYJ20170302154106666, KQJSCX20170727101233642), and NSERC (No. 611370).
Author information
Authors and Affiliations
Corresponding author
Additional information
Pengfei Xu is an assistant professor in the Visual Computing Research Center at the College of Computer Science and Software Engineering, Shenzhen University. He received his bachelor degree in math from Zhejiang University in 2009, and his Ph.D. degree in computer science from Hong Kong University of Science and Technology in 2015. His primary research lies in human computer interaction and computer graphics.
Jianqiang Ding is a master student with the Visual Computing Research Center at the College of Computer Science and Software Engineering, Shenzhen University. He received his bachelor degree from Harbin Engineering University in 2015. His research interest is in computer graphics.
Hao Zhang is a full professor in the School of Computing Science at Simon Fraser University, where he directs the GrUVi lab. He obtained his Ph.D. degree from the University of Toronto, and both M.Math. and B.Math. degrees from the University of Waterloo. His research is in computer graphics with a focus on geometry modeling, shape analysis, 3D content creation, and computational design and fabrication.
Hui Huang is a distinguished professor at Shenzhen University, where she directs the Visual Computing Research Center. She received her Ph.D. degree in applied math from the University of British Columbia in 2008 and another Ph.D. degree in computational math from Wuhan University in 2006. Her research interests are in computer graphics and scientific computing, focusing on point-based modeling, image processing, geometric analysis, 3D acquisition and creation.
Rights and permissions
This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.
The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
To view a copy of this licence, visit https://doi.org/creativecommons.org/licenses/by/4.0/.
Other papers from this open access journal are available free of charge from https://doi.org/www.springer.com/journal/41095. To submit a manuscript, please go to https://doi.org/www.editorialmanager.com/cvmj.
About this article
Cite this article
Xu, P., Ding, J., Zhang, H. et al. Discernible image mosaic with edge-aware adaptive tiles. Comp. Visual Media 5, 45–58 (2019). https://doi.org/10.1007/s41095-019-0130-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41095-019-0130-7