Skip to main content

Advertisement

SpringerLink
Log in

Estimating sex and age from a face: a forensic approach using machine learning based on photo-anthropometric indexes of the Brazilian population

  • Original Article
  • Published:
International Journal of Legal Medicine Aims and scope Submit manuscript

Abstract

The facial analysis permits many investigations, some of the most important of which are craniofacial identification, facial recognition, and age and sex estimation. In forensics, photo-anthropometry describes the study of facial growth and allows the identification of patterns in facial skull development, for example, by using a group of cephalometric landmarks to estimate anthropological information. Previous works presented, as indirect applications, the use of photo-anthropometric measurements to estimate anthropological information such as age and sex. In several areas, automation of manual procedures has achieved advantages over and similar measurement confidence as a forensic expert. This manuscript presents an approach using photo-anthropometric indexes, generated from frontal faces cephalometric landmarks of the Brazilian population, to create an artificial neural network classifier that allows the estimation of anthropological information, in this specific case age and sex. This work is focused on four tasks: (i) sex estimation on ages from 5 to 22 years old, evaluating the interference of age on sex estimation; (ii) age estimation from photo-anthropometric indexes for four age intervals (1 year, 2 years, 4 years, and 5 years); (iii) age group estimation for thresholds of over 14 and over 18 years old; and; (iv) the provision of a new data set, available for academic purposes only, with a large and complete set of facial photo-anthropometric points marked and checked by forensic experts, measured from over 18,000 faces of individuals from Brazil over the last 4 years. The proposed binary classifier obtained significant results, using this new data set, for the sex estimation of individuals over 14 years old, achieving accuracy values higher than 0.85 by the F1 measure. For age estimation, the accuracy results are 0.72 for the F1 measure with an age interval of 5 years. For the age group estimation, the F1 measures of accuracy are higher than 0.93 and 0.83 for thresholds of 14 and 18 years, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. When the term “avoiding” is used throughout the manuscript, it means information from a specific class was not used in the proposed tests.

  2. The data set and the developed machine learning model will be available to download soon, after the review process is completed, in attendance of the journal’s submission criteria.

  3. All boxplots are available in the specific Section of the Supplementary Material files (File PAIs_sex_age_boxplots_supplemental.pdf).

  4. See Fig. 2, for example, where outliers can be identified as the points well above or well below the boxes in the boxplot.

References

  1. Marquez-Grant N (2015) An overview of age estimation in forensic anthropology: perspectives and practical considerations. Annals of human biology 42(4):308–322

    PubMed  Google Scholar 

  2. Silva RF, Franco A, Dias PEM, Gonçalves AS, Paranhos LR (2013) Interrelationship between forensic radiology and forensic odontology–a case report of identified skeletal remains. Journal of Forensic Radiology and Imaging 1(4):201–206

    Google Scholar 

  3. Adserias-Garriga J, Thomas C, Ubelaker DH, Zapico SC (2018) When forensic odontology met biochemistry: Multidisciplinary approach in forensic human identification. Archives of oral biology 87:7–14

    CAS  PubMed  Google Scholar 

  4. INTERPOL (2018) INTERPOL disaster victim identification guide. https://www.interpol.int/INTERPOL-expertise/Forensics/DVI-Pages/DVI-guide, URL https://www.interpol.int/

  5. Zaghetto C, Aguiar LHM, Zaghetto A, Ralha CG, de Barros Vidal F (2017) Agent-based framework to individual tracking in unconstrained environments. Expert Systems with Applications 87:118 – 128. https://doi.org/10.1016/j.eswa.2017.05.065. URL https://www.sciencedirect.com/science/article/pii/S0957417417303925

    Article  Google Scholar 

  6. Cattaneo C, Obertová Z, Ratnayake M, Marasciuolo L, Tutkuviene J, Poppa P, Gibelli D, Gabriel P, Ritz-Timme S (2012) Can facial proportions taken from images be of use for ageing in cases of suspected child pornography? A pilot study. International journal of legal medicine 126(1):139–144

    PubMed  Google Scholar 

  7. Ratnayake M, Obertová Z, Dose M, Gabriel P, Bröker HM, Brauckmann M, Barkus A, Rizgeliene R, Tutkuviene J, Ritz-Timme S, et al. (2014) The juvenile face as a suitable age indicator in child pornography cases: a pilot study on the reliability of automated and visual estimation approaches. International journal of legal medicine 128(5):803–808

    CAS  PubMed  Google Scholar 

  8. de Melo Nunes LF, Zaghetto C, de Barros Vidal F (2018) 3d face recognition on point cloud data - an approaching based on curvature map projection using low resolution devices. In: Proceedings of the 15th International Conference on Informatics in Control, Automation and Robotics - Volume 2: ICINCO, INSTICC. SciTePress, pp 266–273, DOI https://doi.org/10.5220/0006843702660273, (to appear in print)

  9. Borges DL, Vidal FB, Flores MRP, Melani RFH, Guimarães MA, Machado CEP (2018) Photoanthropometric face iridial proportions for age estimation: An investigation using features selected via a joint mutual information criterion. Forensic Science International 284:9 – 14

    PubMed  Google Scholar 

  10. Cattaneo C, Ritz-Timme S, Gabriel P, Gibelli D, Giudici E, Poppa P, Nohrden D, Assmann S, Schmitt R, Grandi M (2009) The difficult issue of age assessment on pedo-pornographic material. Forensic science international 183(1):e21–e24

    PubMed  Google Scholar 

  11. Machado CEP, Flores MRP, Lima LNC, Tinoco RLR, Franco A, Bezerra ACB, Evison MP, Guimarães MA (2017) A new approach for the analysis of facial growth and age estimation: Iris ratio. PLOS ONE 12(7):e0180330

    PubMed  PubMed Central  Google Scholar 

  12. Flores MRP, Machado CEP, Gallidabino MD, de Arruda GHM, da Silva RHA, de Vidal FB, Melani RFH (2018) Comparative assessment of a novel photo-anthropometric landmark-positioning approach for the analysis of facial structures on two-dimensional images. Journal of forensic sciences

  13. Gonzales PS, Machado CEP, Michel-Crosato E (2018) Photoanthropometry of the face in the young white brazilian population. Brazilian dental journal 29(6):619–623

    PubMed  Google Scholar 

  14. Zhu G, van der Aa S (2017) Trends of age of consent legislation in europe: A comparative study of 59 jurisdictions on the european continent. New Journal of European Criminal Law 8(1):14–42

    Google Scholar 

  15. Carpenter B, O’Brien E, Hayes S, Death J (2014) Harm, responsibility, age, and consent. New Criminal Law Review: In International and Interdisciplinary Journal 17(1):23–54

    Google Scholar 

  16. Cericato GO, Franco A, Bittencourt MAV, Nunes MAP, Paranhos LR (2016) Correlating skeletal and dental developmental stages using radiographic parameters. Journal of forensic and legal medicine 42:13–18

    PubMed  Google Scholar 

  17. Machado MA, Júnior ED, Fernandes MM, Lima IFP, Cericato GO, Franco A, Paranhos LR (2018) Effectiveness of three age estimation methods based on dental and skeletal development in a sample of young brazilians. Archives of oral biology 85:166–171

    PubMed  Google Scholar 

  18. Lanitis A, Draganova C, Christodoulou C (2004) Comparing different classifiers for automatic age estimation. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 34(1):621–628. https://doi.org/10.1109/TSMCB.2003.817091

    Article  Google Scholar 

  19. Fu Y, Huang TS (2008) Human age estimation with regression on discriminative aging manifold. IEEE Transactions on Multimedia 10(4):578–584. https://doi.org/10.1109/TMM.2008.921847

    Article  Google Scholar 

  20. Guo G, Fu Y, Dyer CR, Huang TS (2008) Image-based human age estimation by manifold learning and locally adjusted robust regression. IEEE Transactions on Image Processing 17(7):1178–1188. https://doi.org/10.1109/TIP.2008.924280

    Article  PubMed  Google Scholar 

  21. Xiao B, Yang X, Zha H, Xu Y, Huang TS (2009) Metric learning for regression problems and human age estimation. In: Muneesawang P, Wu F, Kumazawa I, Roeksabutr A, Liao M, Tang X (eds) Advances in Multimedia Information Processing - PCM 2009. Springer Berlin Heidelberg, Berlin, Heidelberg, pp 88–99

  22. Chang K, Chen C, Hung Y (2010) A ranking approach for human ages estimation based on face images. In: 2010 20th International Conference on Pattern Recognition, pp 3396–3399, DOI https://doi.org/10.1109/ICPR.2010.829, (to appear in print)

  23. Chang K, Chen C (2015) A learning framework for age rank estimation based on face images with scattering transform. IEEE Transactions on Image Processing 24(3):785–798. https://doi.org/10.1109/TIP.2014.2387379

    Article  PubMed  Google Scholar 

  24. Chen S, Zhang C, Dong M, Le J, Rao M (2017) Using ranking-CNN for age estimation. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 742–751, DOI https://doi.org/10.1109/CVPR.2017.86, (to appear in print)

  25. Zhang L, Shi Z, Cheng M-M, Liu Y, Bian J-W, Zhou JT, Zheng G, Zeng Z (2019) Robust regression via deep negative correlation learning. arXiv:1908.09066

  26. Taheri S, Toygar O (2019) On the use of dag-CNN architecture for age estimation with multi-stage features fusion. Neurocomputing 329:300–310

    Google Scholar 

  27. Xie J-C, Pun C-M (2019) Chronological age estimation under the guidance of age-related facial attributes. IEEE Transactions on Information Forensics and Security

  28. International Organization for Standardization (2005) ISO/IEC 19794-5: Information technology – Biometric data interchange formats – Part 5: Face image data. Standard, International Organization for Standardization

  29. Pinheiro-Flores MR, Palhares-Machado CE (2017) Manual of facial photoanthropometry: landmarks in frontal view from visual references, 1st edn. https://facisgroup.org/facial_landmarks

  30. Pinheiro-Flores MR (2014) Proposta de metodologia de análise fotoantropométrica para identificação humana em imagens faciais em norma frontal. Master’s Thesis, Faculdade de Odontologia de Ribeirão Preto, Universidade de São Paulo

  31. Porto LF, Lima LNC, Flores MRP, Valsecchi A, Ibanez O, Palhares CEM, de Barros Vidal F (2019) Automatic cephalometric landmarks detection on frontal faces: An approach based on supervised learning techniques. Digital Investigation 30:108 – 116. https://www.sciencedirect.com/science/article/pii/S1742287619301288, https://doi.org/10.1016/j.diin.2019.07.008

    Google Scholar 

  32. Caple J, Stephan C (2016) A standardized nomenclature for craniofacial and facial anthropometry. International Journal of Legal Medicine 130(3):863–879

    PubMed  Google Scholar 

  33. Farkas LG (1994) Anthropometry of the head and face. Raven Pr ed 2. New York, Raven Press

  34. Brown RE, Kelliher TP, Tu PH, Turner WD, Taister MA, Miller KWP (2004) A survey of tissue-depth landmarks for facial approximation. Forensic Sci. Commun, 6(1)

  35. Phillips PJ, Moon H, Rizvi S, Rauss PJ, et al. (2000) The FERET evaluation methodology for face-recognition algorithms. Pattern Analysis and Machine Intelligence, IEEE Transactions on 22 (10):1090–1104

    Google Scholar 

  36. Apeland S (2019) Intel AI devcloud. URL https://www.intel.ai/devcloud/ Accessed: 2019-01-22

  37. Chollet F, et al. (2015) Keras. https://keras.io

  38. Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, Corrado GS, Davis A, Dean J, Devin M, Ghemawat S, Goodfellow I, Harp A, Irving G, Isard M, Jia Y, Jozefowicz R, Kaiser L, Kudlur M, Levenberg J, Mané D, Monga R, Moore S, Murray D, Olah C, Schuster M, Shlens J, Steiner B, Sutskever I, Talwar K, Tucker P, Vanhoucke V, Vasudevan V, Viégas F, Vinyals O, Warden P, Wattenberg M, Wicke M, Yu Y, Zheng X (2015) TensorFlow: Large-scale machine learning on heterogeneous systems. https://www.tensorflow.org/, Software available from tensorflow.org

  39. Gardner MW, Dorling SR (1998) Artificial neural networks (the multilayer perceptron)–a review of applications in the atmospheric sciences. Atmospheric environment 32(14-15):2627–2636

    CAS  Google Scholar 

  40. Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv:1412.6980

  41. Powers DMW (2011) Evaluation: From precision, recall and f-measure to roc., informedness, markedness & correlation. Journal of Machine Learning Technologies 2(1):37–63

    Google Scholar 

  42. Provost F, Kohavi R (1998) On applied research in machine learning. In: Machine learning, pp 127–132

  43. Hastie T, Tibshirani R, Friedman J (2009) The elements of statistical learning: data mining, inference and prediction. Springer, 2nd edn Springer-Verlag New York. https://www-stat.stanford.edu/tibs/ElemStatLearn/

  44. Kohavi R, et al. (1995) A study of cross-validation and bootstrap for accuracy estimation and model selection. In: Ijcai, vol 14, Stanford, CA, pp 1137–1145

  45. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Ghahramani Z, Welling M, Cortes C, Lawrence ND, Weinberger KQ (eds) Advances in Neural Information Processing Systems 27. https://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf. Curran Associates, Inc., pp 2672–2680

  46. Wilk MB, Shapiro SS (1965) An analysis of variance test for normality (complete samples). Biometrika 52(3-4):591–611. https://doi.org/10.1093/biomet/52.3-4.591. https://oup.prod.sis.lan/biomet/article-pdf/52/3-4/591/962907/52-3-4-591.pdf

    Article  Google Scholar 

  47. Kutner MH (2005) Applied linear statistical models. McGrwa-Hill international edition, McGraw-Hill Irwin. https://books.google.com.br/books?id=0xqCAAAACAAJ

  48. Schmeling A, Olze A, Reisinger W, Geserick G (2001) Age estimation of living people undergoing criminal proceedings. The Lancet 358(9276):89–90

    CAS  Google Scholar 

  49. Schmeling A, Dettmeyer R, Rudolf E, Vieth V, Geserick G (2016) Forensic age estimation: methods, certainty, and the law. Deutsches Ärzteblatt International 113(4):44

    PubMed  PubMed Central  Google Scholar 

  50. Silva RF, Mendes SDSC, do Rosário Júnior AF, Dias PEM, Martorell LB (2013) Evidência documental X evidência biológica para estimativa da idade–relato de caso pericial. Revista Odontológica do Brasil Central 22(60):6–10

    Google Scholar 

  51. Machado ALR, Dezem TU, Bruni AT, da Silva RHA (2017) Age estimation by facial analysis based on applications available for smartphones. The Journal of forensic odonto-stomatology 35(2):55

    Google Scholar 

  52. Deitos AR, Costa C, Michel-Crosato E, Galić I, Cameriere R, Biazevic MGH (2015) Age estimation among brazilians: younger or older than 18? Journal of forensic and legal medicine 33:111–115

    PubMed  Google Scholar 

  53. Santiago BM, Almeida L, Cavalcanti YW, Magno MB, Maia LC (2018) Accuracy of the third molar maturity index in assessing the legal age of 18 years: a systematic review and meta-analysis. International journal of legal medicine 132(4):1167– 1184

    PubMed  Google Scholar 

  54. Franco A, Thevissen P, Fieuws S, Souza PHC, Willems G (2013) Applicability of willems model for dental age estimations in brazilian children. Forensic science international 231(1-3):401–e1

    PubMed  Google Scholar 

  55. Graupner H (2000) Sexual consent: The criminal law in europe and overseas. Archives of Sexual Behavior 29(5):415–461

    CAS  PubMed  Google Scholar 

  56. Machado CEP, Santiago BM, Lima LNC, Gonzales PS, Franco A, de Barros Vidal F, Aguilera IA, Guimarães MA (2019) Applicability of a pre-established set of facial proportions from frontal photographs in forensic age estimation of a brazilian population. Forensic science international 301:e1–e7

    PubMed  Google Scholar 

  57. Baldasso RP, Damascena NP, Deitos AR, Palhares Machado CE, Franco A, de Oliveira RN (2019) Morphologic alterations ear, nose and lip detected with aging through facial photoanthropometric analysis. Journal of Forensic Odonto-Stomatology 37(2):25–34

    CAS  PubMed  Google Scholar 

  58. Balaji SM (2016) Facial feminization-surgical modification for indian, european and african faces. Annals of maxillofacial surgery 6(2):210

    CAS  PubMed  PubMed Central  Google Scholar 

  59. Kloess JA, Woodhams J, Whittle H, Grant T, Hamilton-Giachritsis CE (2017) The challenges of identifying and classifying child sexual abuse material. Sexual Abuse, p 1079063217724768

  60. Cummaudo M, Guerzoni M, Gibelli D, Cigada A, Obertovà Z, Ratnayake M, Poppa P, Gabriel P, Ritz-Timme S, Cattaneo C (2014) Towards a method for determining age ranges from faces of juveniles on photographs. Forensic science international 239:107–e1

    PubMed  Google Scholar 

  61. Tummon HM, Allen J, Bindemann M (2019) Facial identification at a virtual reality airport. i-Perception 10(4):2041669519863077

    PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgments

The authors would like to acknowledge the team of Federal Police of Brazil, especially the forensic experts of the National Institute of Criminalistic.

Funding

This work was conducted with financial support from Coordination for the Improvement of Higher Education Personnel (CAPES), Edital DPI UnB # 04/2019 (Researcher Support), and Federal Police of Brazil (Pro-Forenses 25/2014 CAPES Finance Code 001).

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Brasilia, Brasilia, Brazil

    Lucas Faria Porto & Flavio de Barros Vidal

  2. Department of Odontology, Federal University of Maranhao, São Luís, Maranhao, Brazil

    Laíse Nascimento Correia Lima

  3. Division of Oral Radiology, Faculdade São Leopoldo Mandic, Campinas, Brazil

    Ademir Franco

  4. Department of Statistics, University of Brasilia, Brasilia, Brazil

    Donald Pianto

  5. National Institute of Criminalistic, Federal Police, Brasilia, Brazil

    Carlos Eduardo Palhares Machado

Authors
  1. Lucas Faria Porto
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Laíse Nascimento Correia Lima
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ademir Franco
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Donald Pianto
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Carlos Eduardo Palhares Machado
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Flavio de Barros Vidal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Flavio de Barros Vidal.

Ethics declarations

Conflict of interest

The authors declare that there is no conflict of interest.

Ethical approval

17017213.0.0000.5440 (Local Committee of ethics in human research)

Informed consent

The images used to illustrate the present study were acquired and used with the signed consent of the photographed subject or his/her legal guardians.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Appendices

Appendix A: Photo-anthropometric indexes

Table 5 Description of the 208 photo-anthropometric indexes (PAIs)
Full size table

Appendix B: Confusion matrices

Fig. 13
figure 13

Confusion matrix: age estimation at age intervals of 2 years without sex information

Full size image
Fig. 14
figure 14

Confusion matrix: age estimation at age intervals of 2 years for female sex

Full size image
Fig. 15
figure 15

Confusion matrix: age estimation at age intervals of 2 years for male sex

Full size image
Fig. 16
figure 16

Confusion matrix: age estimation with age intervals of 4 years

Full size image
Fig. 17
figure 17

Confusion matrix: age estimation with age intervals of 5 years

Full size image

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Porto, L.F., Lima, L.N.C., Franco, A. et al. Estimating sex and age from a face: a forensic approach using machine learning based on photo-anthropometric indexes of the Brazilian population. Int J Legal Med 134, 2239–2259 (2020). https://doi.org/10.1007/s00414-020-02346-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00414-020-02346-5

Keywords

Search

Navigation