Multimedia Tools and Applications

, Volume 77, Issue 8, pp 9827–9847 | Cite as

Monitoring skin condition using life activities on the SNS user documents

  • Jehyeok Rew
  • Eenjun Hwang
  • Young-Hwan Choi
  • Seungmin Rho


Social networks are not new to the IT landscape. Starting from bulletin boards and chat rooms, they have evolved to include desktop and mobile device applications such as Facebook, Twitter, and Flickr, which are used by millions of people daily. People can post diverse types of information such as what they are doing, what kind of foods they are eating, and where they are going. Some of these activities are known to have direct/indirect effect on the condition of their skin. Typical examples include lack of sleeping, excessive drinking, and persistent sunlight exposure. In this paper, we propose a scheme for evaluating the condition of a user’s skin based on their everyday activities collected from their postings. For such an evaluation, users should regularly send microscopic images of their skin to a server using their smartphone. Meanwhile, the server collects the user postings from their SNS and analyzes them to identify activities that might have an influence on their skin. Finally, the server provides the user with a report containing a comparison of their past and current skin conditions, a statistical summary of their occasional events collected from their SNS, and a set of advices for improving their skin condition including skin care products. We built a prototype system and performed various experiments to show the effectiveness of our scheme. We report some of the results.


Skin image ananlysis Skin condition Skin care SNS Life activity ananlysis 



This work was supported by Institute for Information & communications Technology Promotion (IITP) grant funded by the Korea government (MSIP) (No. R0190-16-2012, High Performance Big Data Analytics Platform Performance Acceleration Technologies Development).


  1. 1.
    Akazaka S, Nakagawa H, Kazama H, Osanai O, Kawai M, Takema Y, Imokawa G (2002) Age-related changes in skin wrinkles assessed by a novel three-dimensional morphometric analysis. Br J Dermatol 147:689–695Google Scholar
  2. 2.
    Castro D, Hickson S, Bettadapura V, Thomaz E, Abowd G, Christensen H, Essa I (2015) Predicting daily activities from egocentric images using deep learning. In: Proceedings of the 2015 ACM International symposium on Wearable Computers, pp 75–82Google Scholar
  3. 3.
    Choi Y, Tak Y, Rho S, Hwang E (2013) Skin feature extraction and processing model for statistical skin age estimation. Multimedia Tools and Application 64(2):227–247CrossRefGoogle Scholar
  4. 4.
    Choi Y, Kim D, Kim B, Hwang E (2014) Skin texture aging trend analyis using dermoscopy images. Skin Res Technol 20(4):486–497Google Scholar
  5. 5.
    Cula GO, Bargo PR, Nkengne A, Kollias N (2013) Assessing facial wrinkles: automatic detection and quantification. Skin Res Technol 19(1):243–251CrossRefGoogle Scholar
  6. 6.
    Doherty AR, Smeaton AF, Alan F (2008) Automatically segmenting lifefog data into events’, image analysis for multimedia interactive services, WIAMIS'08. Ninth International workshop, pp 20–23Google Scholar
  7. 7.
    Edwards C, Heggie R, Marks R (2003) A study of differences in surface roughness between sun-exposed and unexposed skin with age. Photodermatology, photoimmunology & photomedicine 19(4):169–174Google Scholar
  8. 8.
  9. 9.
    Hamer MA, Jacobs LC,  Lall JS, Wollstein A, Hollestein LM, Rae AR, Gossage KW, Hofman A, Liu F, Kayser M, Nijsten T, Gunn DA (2015) Validation of image analysis techniques to measure skin aging features from facial photographs. Skin Res Technol 21(4):392–402.Google Scholar
  10. 10.
  11. 11.
    Rekimoto J, Miyaki T, Ishizawa T (2007) LifeTag: WiFi-based continuous location logging for life pattern analysis. LoCA, vol. 2007. pp 35-49Google Scholar
  12. 12.
    Kim K, Choi Y, Hwang E (2009) Wrinkle feature-based skin age estimation scheme. In: Proceedings of International Conference on Multimedia and Expo, pp 1222–1225Google Scholar
  13. 13.
    Kwon Y, Kang K, Bae C, Chung H, Kim J (2014) Lifelog agent for human activity pattern analysis on health avatar platform. Healthcare informatics research 20(1):69–75CrossRefGoogle Scholar
  14. 14.
    Masuda Y, Oguri M, Morinaga T, Hirao T (2014) Three-dimensional morphological characterization of the skin surface micro-topography using a skin replica and changes with age. Skin Res Technol 20(3):299–306CrossRefGoogle Scholar
  15. 15.
    McDuff D, Karlson A, Kapoor A, Roseway A, Czerwinski M (2012) AffectAura: an intelligent system for emotional memory. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp 849–858Google Scholar
  16. 16.
    Miyamoto K, Nagasawa H, Inoue Y, Nakaoka K, Hirano A, Kawada A (2013) Development of new in vivo imaging methodology and system for the rapid and quantitative evaluation of the visual appearance of facial skin firmness. Skin Res Technol, 19(1):525–531.Google Scholar
  17. 17.
    Kawanishi N, Tamai M, Hasegawa A, Takeuchi Y, Tajika A, Ogawa Y, Furukawa T (2015) Lifelog-based estimation of activity diary for cognitive behavioral therapy. In: Proceedings of the 2015 ACM International joint Conference on pervasive and ubiquitous Computing and Proceedings of the 2015 ACM International symposium on wearable computers, pp 1251–1256Google Scholar
  18. 18.
    OpenWeatherMap API.
  19. 19.
    Rew J, Choi Y, Kim D, Rho S, Hwang E (2014) Evaluating skin Herediatary traits based on daily activities. Frontier and Innovation in Future Computing and Communications, pp 261–270Google Scholar
  20. 20.
    Shinohara A, Ito T, Ura T, Nishiguchi S, Ito H, Yamada M, Yoshitomi H, Furu M, Okamoto K, Aoyama T (2013) Development of lifelog sharing system for rheumatoid arthritis patients using smartphone. Annual International Conference of the IEEE engineering in medicine and biology society, pp 7266-7269.Google Scholar
  21. 21.
    Tanaka H, Nakagami G, Sanada H, Sari Y, Kobayashi H, Kishi K, Konya C, Tadaka E (2008) Quantitative evaluation of elderly skin based on digital image analysis. Skin Res Technol 14(2):192–200Google Scholar
  22. 22.
  23. 23.
    Jacobi U, Chen M, Frankowski G, Sinkgraven R, Hund M, Rzany B, Lademann J (2004) In vivo determination of skin surface topology using an optical 3D device. Skin Res Technol 10(4):207–214Google Scholar
  24. 24.
    Wang P, Sun L, Yang S, Smeaton AF, Gurrin C (2016) Characterizing everyday activities from visual lifelogs based on enhancing concept representation. Comput Vis Image Underst 148:181–192CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  • Jehyeok Rew
    • 1
  • Eenjun Hwang
    • 1
  • Young-Hwan Choi
    • 2
  • Seungmin Rho
    • 3
  1. 1.School of Electrical EngineeringKorea UniversitySeoulSouth Korea
  2. 2.Kiturami Research Planning Center, Co., Ltd.IncheonSouth Korea
  3. 3.Department of Media SoftwareSungkyul UniversityAnyangSouth Korea

Personalised recommendations