Facial Skin Classification Using Convolutional Neural Networks

  • Jhan S. AlarifiEmail author
  • Manu Goyal
  • Adrian K. Davison
  • Darren Dancey
  • Rabia Khan
  • Moi Hoon Yap
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10317)


Facial skin assessment is crucial for a number of fields including the make-up industry, dermatology and plastic surgery. This paper addresses skin classification techniques which use conventional machine learning and state-of-the-art Convolutional Neural Networks to classify three types of facial skin patches, namely normal, spots and wrinkles. This study aims to accomplish the pivotal work on the basis of these three classes to provide the collective facial skin quality score. In this work, we collected high quality face images of people from different ethnicities to create a derma dataset. Then, we outlined the skin patches of 100 \(\times \) 100 resolution in the three pre-decided classes. With extensive parameter tuning, we ran a number of computer vision experiments using both traditional machine learning and deep learning techniques for this 3-class classification. Despite the limited dataset, GoogLeNet outperforms the Support Vector Machine approach with Accuracy of 0.899, F-Measure of 0.852 and Matthews Correlation Coefficient of 0.779. The result shows the potential use of deep learning for non-clinical skin images classification, which will be more promising with a larger dataset.


Facial skin CNNs Classification Skin quality assessment 


  1. 1.
    Ng, C.-C., Yap, M.H., Costen, N., Li, B.: Automatic wrinkle detection using hybrid Hessian filter. In: Cremers, D., Reid, I., Saito, H., Yang, M.-H. (eds.) ACCV 2014. LNCS, vol. 9005, pp. 609–622. Springer, Cham (2015). doi: 10.1007/978-3-319-16811-1_40 Google Scholar
  2. 2.
    Ng, C.-C., Yap, M.H., Costen, N., Li, B.: Wrinkle detection using Hessian line tracking. IEEE Access 3, 1079–1088 (2015)CrossRefGoogle Scholar
  3. 3.
    Prats-Montalbán, J.M., Ferrer, A., Bro, R., Hancewicz, T.: Prediction of skin quality properties by different multivariate image analysis methodologies. Chemometr. Intell. Lab. Syst. 96(1), 6–13 (2009)CrossRefGoogle Scholar
  4. 4.
    Mizukoshi, K., Takahashi, K.: Analysis of the skin surface and inner structure around pores on the face. Skin Res. Technol. 20(1), 23–29 (2014)CrossRefGoogle Scholar
  5. 5.
    Luebberding, S., Krueger, N., Kerscher, M.: Comparison of validated assessment scales and 3D digital fringe projection method to assess lifetime development of wrinkles in men. Skin Res. Technol. 20(1), 30–36 (2014)CrossRefGoogle Scholar
  6. 6.
    Cula, G.O., Bargo, P.R., Nkengne, A., Kollias, N.: Assessing facial wrinkles: automatic detection and quantification. Skin Res. Technol. 19(1), e243–e251 (2013)CrossRefGoogle Scholar
  7. 7.
    Wang, L.: Support Vector Machines: Theory and Applications, vol. 177. Springer Science & Business Media, Heidelberg (2005)CrossRefzbMATHGoogle Scholar
  8. 8.
    Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)Google Scholar
  9. 9.
    Liao, H.: A deep learning approach to universal skin disease classificationGoogle Scholar
  10. 10.
    Schmidhuber, J.: Deep learning in neural networks: an overview. Neural Netw. 61, 85–117 (2015)CrossRefGoogle Scholar
  11. 11.
    Yuan, X., Yang, Z., Zouridakis, G., Mullani, N.: SVM-based texture classification and application to early melanoma detection. In: 28th Annual International Conference of the IEEE, Engineering in Medicine and Biology Society, EMBS 2006, pp. 4775–4778. IEEE (2006)Google Scholar
  12. 12.
    Khan, R., Hanbury, A., Stöttinger, J., Bais, A.: Color based skin classification. Pattern Recogn. Lett. 33(2), 157–163 (2012)CrossRefGoogle Scholar
  13. 13.
    Wang, L., Sng, D.: Deep learning algorithms with applications to video analytics for a smart city: a survey. arxiv preprint (2015). arXiv:1512.03131
  14. 14.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp. 1097–1105 (2012)Google Scholar
  15. 15.
    Esteva, A., Kuprel, B., Novoa, R.A., Ko, J., Swetter, S.M., Blau, H.M., Thrun, S.: Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115–118 (2017)CrossRefGoogle Scholar
  16. 16.
    Ekman, P.: Facial expressions. Handb. Cogn. Emot. 16, 301–320 (1999)Google Scholar
  17. 17.
    Guo, Z., Zhang, L., Zhang, D.: A completed modeling of local binary pattern operator for texture classification. IEEE Trans. Image Process. 19(6), 1657–1663 (2010)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Yap, M.H., Ugail,H., Zwiggelaar, R., Rajoub, B., Doherty, V., Appleyard, S., Hurdy, G.: A short review of methods for face detection and multifractal analysis. In: International Conference on CyberWorlds, CW 2009, pp. 231–236. IEEE (2009)Google Scholar
  19. 19.
    Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM International Conference on Multimedia, pp. 675–678. ACM (2014)Google Scholar
  20. 20.
    Liu, D., Wang, Y.: Monza: image classification of vehicle make and model using convolutional neural networks and transfer learningGoogle Scholar
  21. 21.
    Singh, B., De, S., Zhang, Y., Goldstein, T., Taylor, G.: Layer-specific adaptive learning rates for deep networks. In: 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA), pp. 364–368. IEEE (2015)Google Scholar
  22. 22.
    Duchi, J., Hazan, E., Singer, Y.: Adaptive subgradient methods for online learning and stochastic optimization. J. Mach. Learn. Res. 12, 2121–2159 (2011)MathSciNetzbMATHGoogle Scholar
  23. 23.
    Yap, M.H., Edirisinghe, E., Bez, H.: Processed images in human perception: a case study in ultrasound breast imaging. Eur. J. Radiol. 73(3), 682–687 (2010)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Jhan S. Alarifi
    • 1
    Email author
  • Manu Goyal
    • 1
  • Adrian K. Davison
    • 1
  • Darren Dancey
    • 1
  • Rabia Khan
    • 1
  • Moi Hoon Yap
    • 1
  1. 1.Manchester Metropolitan UniversityManchesterUK

Personalised recommendations