Abstract
The makeup transfer task aims to transfer makeup styles from a reference makeup image to another non-makeup image. Previous methods achieved great progress with the same face angle, but failed to achieve good results when transferring between images with large spatial misalignment of face angles. In this paper, We propose a method for facial makeup transfer for large-angle spatial misalignment which based on generative adversarial networks. It first utilizes the Neural Head Reconstruction Module to process the reference image to obtain a new reference image. The new reference image maintains the original makeup style but has the same face angle as the source image, and it still has high definition and realism. Thus, the subsequent makeup transfer will be much easier. In addition, in order to ensure the authenticity and clarity of local details as much as possible, we also introduce the concept of local perception to transfer the makeup while keeping the color of the original character’s eyeballs, ears and neck unchanged. Besides, we can also realize controllable and partial makeup transfer. Experimental results show that our method achieves the state-of-the-art compared to existing methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A.C., Bengio, Y.: Generative adversarial nets. In: NeurIPS (2014)
Zhu, J.-Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycleconsistent adversarial networks. In: ICCV (2017)
Burkov, E., Pasechnik, I., Grigorev, A., et al.: Neural Head Reenactment with Latent Pose Descriptors. IEEE (2020)
Zhu, W., Lin, L.: Beautygan: Instance-Level Facialmakeup Transfer with Deep Generative Adversarial Network. In: ACM MM (2018)
Chen, H.-J., Hui, K.-M., Wang, S.-Y., Tsao, L.-W., Shuai, H.-H., Cheng, W.-H.: Beautyglow: Ondemand makeup transfer framework with reversible generative network. In: CVPR pp. 10042–10050 (2019)
Chang, H., Lu, J., Yu, F., Finkelstein, A.: Pairedcyclegan: Asymmetric style transfer for applying and removing makeup. In: CVPR (2018)
Gu, Q., Wang, G., Chiu, M.T., Tai, Y.-W., Tang, C.-K.: Ladn: Local adversarial disentangling network for facial makeup and de-makeup. In: ICCV (2019)
Jiang, W., et al.: Psgan: Pose and expression robust spatial-aware gan for customizable makeup transfer. In: CVPR, pp. 5194–5202 (2020)
Vaswani, A., et al.: Attention is all you need. In: NeurIPS (2017)
Gatys, L.A., Ecker, A.S., Bethge, M.: A neural algorithm of artistic style. arXiv preprint arXiv:1508.06576 (2015)
Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 694–711. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_43
Dumoulin, V., Shlens, J., Kudlur, M.: A learned representation for artistic style. ArXiv, abs/1610.07629, 2016. 2
Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: CVPR (2016)
Luan, F., Paris, S., Shechtman, E., Bala, K.: Deep photo style transfer. In: CVPR (2017)
Zakharov, E., Shysheya, A., Burkov, E., Lempitsky, V.: Few-shot adversarial learning of realistic neural talking head models. In: Proc. ICCV (2019)
Huang, X., Belongie, S.: Arbitrary style transfer in realtime with adaptive instance normalization. In: Proc. ICCV (2017)
Milletari, F., Navab, N., Ahmadi, S.-A.: V-net: Fully convolutional neural networks for volumetric medical image segmentation. pp. 565–571 (2016)
Deng, H., Han, C., Cai, H., Han, G., He, S.: Spatially-invariant style-codes controlled makeup transfer. In: CVPR, pp. 6549–6557 (2021)
Zhao, M., Zhu, S.-C.: Portrait painting using active templates. In: Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Non-Photorealistic Animation and Rendering, pp. 117–124 (2011)
Park, T., Liu, M.-Y., Wang, T.-C., Zhu, J.-Y.: Semantic image synthesis with spatially-adaptive normalization. In: CVPR (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, C., Cai, W., Li, Z. (2022). Makeup Transfer Based on Generative Adversarial Network for Large Angle Spatial Misalignment. In: Pimenidis, E., Angelov, P., Jayne, C., Papaleonidas, A., Aydin, M. (eds) Artificial Neural Networks and Machine Learning – ICANN 2022. ICANN 2022. Lecture Notes in Computer Science, vol 13531. Springer, Cham. https://doi.org/10.1007/978-3-031-15934-3_15
Download citation
DOI: https://doi.org/10.1007/978-3-031-15934-3_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-15933-6
Online ISBN: 978-3-031-15934-3
eBook Packages: Computer ScienceComputer Science (R0)