Skip to main content

Advertisement

SpringerLink
Log in

Creating a visualization system for changes in facial shape and color

  • Original Article
  • Published:
Artificial Life and Robotics Aims and scope Submit manuscript

Abstract

During the course of treatment in plastic surgery and dermatology, especially aesthetic surgery, it is essential to regularly observe and record any appearance changes of the body. Physicians typically use an interview, photographs of the patient’s face before and after treatment, and anatomical data including size, position, and color. However, it is difficult to photograph the affected area under the same conditions every time because the range of the photograph varies depending on the angle and distance. In addition, measurements with a measuring tape are similarly difficult to reproduce. In this study, we used an infrared depth sensor and RGB sensor built into a smartphone to measure the shape of the face and the color of the skin in three dimensions. The face shape and color were measured before and after modulation, and the changes in volume and skin pigmentation of the skin color were calculated and visualized. It was then possible to analyze the shape and skin tone independent of the viewing angle and direction of illumination light. This research shows the possibility of monitoring changes in facial shape and skin tone with a depth camera built into a smartphone. This technology is expected to contribute to the development of telemedicine, where patients can measure their faces at home and receive treatment consultations remotely.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Krowchuk DP et al (2019) Clinical practice guideline for the management of infantile hemangiomas. Pediatrics 143.1:e20183475

    Article  Google Scholar 

  2. Canfield Scientific, Inc. (2020) VECTRA H1 3D imaging system. https://www.canfieldsci.com/imaging-systems/vectra-h1-3d-imaging-system/. Accessed 27 Mar 2020

  3. Camison L et al (2018) Validation of the Vectra H1 portable three dimensional photogrammetry system for facial imaging. Int J Oral Maxillofac Surg 47(3):403–410

    Article  Google Scholar 

  4. Seitz SM et al (2006) A comparison and evaluation of multi-view stereo reconstruction algorithms. In: 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR’06), vol 1, IEEE

  5. Masui K, Kiyomitsu K, Ogawa-Ochiai K et al (2020) Technology for visualizing the local change in shape of edema using a depth camera. Artif Life Robot. https://doi.org/10.1007/s10015-019-00541-1

    Article  Google Scholar 

  6. Apple Inc. (2020) iPhone. https://www.apple.com/iphone/. Accessed 27 Mar 2020

  7. Bellus3D Inc. (2020) Bellus 3D:High-quality 3D face scanning. https://www.bellus3d.com/. Accessed 27 Mar 2020

  8. Besl PJ, Neil DM (1992) Method for registration of 3-D shapes. Sensor fusion IV: control paradigms and data structures. Int Soc Opt Photon 1611:5

    Google Scholar 

  9. Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 24(6):381–395

    Article  MathSciNet  Google Scholar 

  10. Moller T, Trumbore B (1997) Fast, minimum storage ray—triangle intersection. J Graph Tools 21:21–28

    Article  Google Scholar 

  11. Mukaida S et al (2002) Facial image synthesis system: FUTON evaluation as tools for cognitive research on face processing. Trans IEICE 85(10):1126–1137

    Google Scholar 

  12. Tsumura N et al (2003) Image-based skin color and texture analysis/synthesis by extracting hemoglobin and melanin information in the skin. ACM Trans Graph (TOG) 22(3):770–779

    Article  Google Scholar 

Download references

Acknowledgements

We thank Ashleigh Cooper, Phd, from Edanz (https://jp.edanz.com/ac) for editing a draft of this manuscript.

Author information

Authors and Affiliations

  1. Department of Imaging Science, Graduate School of Science and Engineering, Chiba University, Chiba, Japan

    Reimei Koike & Ikumi Nomura

  2. Department of General Medicine, Kampo Clinical Center, Hiroshima University Hospital, Hiroshima, Japan

    Keiko Ogawa-Ochiai

  3. Department of Imaging Science, Graduate School of Engineering, Chiba University, Chiba, Japan

    Norimichi Tsumura

  4. Oyumino Central Hospital, Chiba, Japan

    Naoki Rikihisa

Authors
  1. Reimei Koike
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ikumi Nomura
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Keiko Ogawa-Ochiai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Norimichi Tsumura
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Naoki Rikihisa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Reimei Koike.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Koike, R., Nomura, I., Ogawa-Ochiai, K. et al. Creating a visualization system for changes in facial shape and color. Artif Life Robotics 27, 29–37 (2022). https://doi.org/10.1007/s10015-022-00733-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10015-022-00733-2

Keywords

Search

Navigation