On Using Soft Biometrics in Forensic Investigation

  • Paulo Lobato Correia
  • Peter K. Larsen
  • Abdenour Hadid
  • Martin Sandau
  • Miguel Almeida
Part of the Advances in Computer Vision and Pattern Recognition book series (ACVPR)


This chapter addresses the usage of biometric recognition tools in the context of forensic investigations. In particular, the authors are concerned with the extraction of evidence from video sequences captured by surveillance cameras. In such scenarios many of the biometric traits traditionally used for recognition purposes, such as fingerprints, palmprints or iris, are not available. Therefore, the focus is on the extraction of soft biometrics, which encompasses personal characteristics used by humans to recognize or help to recognize an individual. This work starts by reviewing how forensic casework relying on surveillance video information is conducted nowadays. Then, a software platform, BioFoV, is proposed to automate many of the required procedures and including some initial implementation of soft biometric extraction tools. Furthermore, novel biometric methods to analyze human gait and facial traits are described and experimentally validated as a demonstration of future perspectives in soft biometrics.


Face Image Support Vector Machine Classifier Facial Expression Recognition Video Shot Ethnicity Classification 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Ariyanto G, Nixon M (2012) Marionette mass-spring model for 3D gait biometrics. In: Proceedings of the International Conference on Biometrics (ICB), pp 354–359Google Scholar
  2. 2.
    Baluja S, Rowley H (2007) Boosting sex identification performance. Int J Comput Vision 71:111–119CrossRefGoogle Scholar
  3. 3.
    Baumgart BG (1974) Geometric modeling for computer vision. Stanford UniversityGoogle Scholar
  4. 4.
    Biber K (2009) Visual jurisprudence: the dangers of photographic identification evidence. CJM 78:35–37Google Scholar
  5. 5.
    Bouchrika I, Goffredo M, Carter J, Nixon M (2011) On using gait in forensic biometrics. J Forensic Sci 56(4):882–889CrossRefGoogle Scholar
  6. 6.
    Bouguet J-Y (2011) MATLAB calibration tool. Accessed 26 Mar 2015
  7. 7.
    Breiman L (2001) Random forests. Mach Learn 45:5–32CrossRefzbMATHGoogle Scholar
  8. 8.
    Dantcheva A, Velardo C , D’angelo A, Dugelay JL (2011) Bag of soft biometrics for person identification: new trends and challenges. Multimed Tools Appl 51(2):739–777Google Scholar
  9. 9.
    Fu Y, Guo G, Huang TS (2010) Age synthesis and estimation via faces: a survey. IEEE Trans Pattern Anal Mach Intell (T-PAMI), 32(11):1955–1976Google Scholar
  10. 10.
    Fu S, He H, Hou Z (2014) Learning race from face: a survey. IEEE Trans Pattern Anal Mach Intell (TPAMI)Google Scholar
  11. 11.
    Furukawa Y, Ponce J (2010) Accurate, dense and robust multi-view stereopsis, vol 32Google Scholar
  12. 12.
    Gallagher AC, Chen T (2009) Understanding images of groups of people. In: Proceedings of IEEE CVPRGoogle Scholar
  13. 13.
    Guan Y, Li C-T (2013) A robust speed-invariant gait recognition system for walker and runner identification. In: The 6th IAPR international conference on biometrics: IAPR, pp 1–8Google Scholar
  14. 14.
    Guo G, Mu G, Fu Y, Huang T (2009) Human age estimation using bio-inspired features. In: CVPR’09, pp 112–119Google Scholar
  15. 15.
    Hadid A, Pietikäinen M, Li SZ (2007) Learning personal specific facial dynamics for face recognition from videos. In: IEEE international workshop on analysis and modeling of faces and gestures (in conjunction with ICCV 2007), pp 1–15Google Scholar
  16. 16.
    Hadid A, Pietikäinen M (2008) Combining motion and appearance for gender classification from video sequences. In: 19th international conference on pattern recognition (ICPR 2008), p 4Google Scholar
  17. 17.
    Han H, Jain AK (2014) Age, Gender and Race Estimation from Unconstrained Face Images, MSU Technical report (2014): MSU-CSE-14-5Google Scholar
  18. 18.
    Harris C, Stephens M (1988) A combined corner and edge detector. In: Alvey vision conference: Manchester, UK, p 50Google Scholar
  19. 19.
    Hartley R, Zisserman A (2003) Multiple view geometry in computer vision, 2nd edn. Cambridge University PressGoogle Scholar
  20. 20.
    Hautamaki S (2011) Forevid: an open source software for forensic video analysis. MSc Thesis, Tampere University of TechnologyGoogle Scholar
  21. 21.
    Horn BKP (1970) Shape from shading: a method for obtaining the shape of a smooth opaque object from one viewGoogle Scholar
  22. 22.
    Iwama H, Muramatsu D, Makihara Y, Yagi Y (2012) Gait-based person-verification system for forensics. In: 2012 IEEE 5th international conference on biometrics: theory, applications and systems (BTAS), Sept 2012, pp 113–120Google Scholar
  23. 23.
    Jain A, Dass S, Nandakumar K (2004) Soft biometric traits for personal recognition systems. In: Proceedings of the international conference on biometric authentication, ICBA, LNCS 3072, pp 731–738Google Scholar
  24. 24.
    Jain A, Russ A (2015) Bridging the gap: from biometrics to forensics. Philos Trans Roy Soc BGoogle Scholar
  25. 25.
    Kanade T, Cohn JF, Tian Y (2000) Comprehensive database for facial expression analysis. In: IEEE international conference on automatic face and gesture recognition, pp 46–53Google Scholar
  26. 26.
    Klare B, Klum S, Klontz J, Taborsky E, Akgul T, Jain AK (2014) Suspect identification based on descriptive facial attributes. In: Proceedings of the international joint conference on biometricsGoogle Scholar
  27. 27.
    Klontz JaC, Jain AK (2013) A case study of automated face recognition: the Boston marathon bombings suspects. IEEE Comput 46(11):91–94Google Scholar
  28. 28.
    Kwon YH, da Vitoria Lobo N (1994) Age classification from facial images. In: CVPR’94, pp 762–767Google Scholar
  29. 29.
    Lanitis A, Taylor C, Cootes T (2002) Toward automatic simulation of aging effects on face images. TPAMI 24(4):442–455CrossRefGoogle Scholar
  30. 30.
    Larsen PK, Hansen L, Simonsen EB, Lynnerup N (2008) Variability of bodily measures of normally dressed people using PhotoModeler® Pro 5. J Forensic Sci 53:1393–1399CrossRefGoogle Scholar
  31. 31.
    Larsen PK, Lynnerup N, Henriksen M, Alkjær T, Simonsen EB (2010) Gait recognition using joint moments, joint angles, and segment angles. J Forensic Biomech 1:7CrossRefGoogle Scholar
  32. 32.
    Larsen PK, Simonsen EB, Lynnerup N (2008) Gait analysis in forensic medicine. J Forensic Sci 53:1149–1153CrossRefGoogle Scholar
  33. 33.
    Lienhart R, Maydt J (2002) An extended set of haarlike features for rapid object detection. In: Proceedings of the international conference on image processing, Rochester, USAGoogle Scholar
  34. 34.
    Lucy D (2005) Introduction to statistics for forensic scientists. WileyGoogle Scholar
  35. 35.
    Makinen E, Raisamo R (2008) An experimental comparison of gender classification methods. Pattern Recogn Lett 29(10):1544–1556CrossRefGoogle Scholar
  36. 36.
    Marr D, Hildreth E (1980) Theory of Edge Detection. Proc Roy Soc B: Biol Sci 207:31CrossRefGoogle Scholar
  37. 37.
    Moghaddam B, Yang M-H (2002) Learning gender with support faces. IEEE Trans Pattern Anal Mach Intell 24(5):707–711CrossRefGoogle Scholar
  38. 38.
    Ng CB, Tay YH, Goi B-M (2012) Recognizing human gender in computer vision: a survey. PRICAI 335–346Google Scholar
  39. 39.
    Ocean Systems (2015) Ocean systems forensic video and image analysis solutions. Accessed 26 Mar 2015
  40. 40.
    Ojala T, Pietikäinen M, Mäenpää T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24:971–987CrossRefzbMATHGoogle Scholar
  41. 41.
    OpenCV (2015) Open source computer vision and machine learning software library. Accessed 26 Mar 2015
  42. 42.
    Panis G, Lanitis A (2014) An overview of research activities in facial age estimation using the FG-NET aging database. In: International ECCV workshop on soft biometricsGoogle Scholar
  43. 43.
    Qt (2015) Cross-platform application and UI framework., Accessed 26 Mar 2015
  44. 44.
    Reid DA, Nixon MS, Stevenage SV (2013) Soft biometrics; human identification using comparative descriptions. IEEE Trans Biom Compend Pattern Anal Mach intell. 36(6)Google Scholar
  45. 45.
    Ricanek K, Tesafaye T (2006) MORPH: a longitudinal image database of normal adult age-progression. In: Proceedings of FGGoogle Scholar
  46. 46.
    Sanderson C, Paliwal KK (2003) Noise compensation in a person verification system using face and multiple speech feature. Pattern Recogn 36(2):293–302CrossRefGoogle Scholar
  47. 47.
    Thornton J, Peterson J (2002) The general assumptions and rationale of forensic identification. In: Modern scientific evidence: the law and science of expert testimony, vol 3. West Publishing CompanyGoogle Scholar
  48. 48.
    Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition, 2001. CVPR 2001, vol 1, pp I–511–I–518Google Scholar
  49. 49.
    Will PM, Pennington KS (1971) Grid coding: a preprocessing technique for robot and machine vision. Artif Intell 2:319–329CrossRefGoogle Scholar
  50. 50.
    Xu Z, Schwarte R, Heinol H-G, Buxbaum B, Ringbeck T (1998) Smart pixel: photonic mixer device (PMD); new system concept of a 3D-imaging camera-on-a-chip. In: Eheung EHM (ed) Proceedings: M2VIP ‘98; Nanjing, China, 10–12 Sept 1998, Hong Kong. pp 259–64Google Scholar
  51. 51.
    Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intell 22(11):1330–1334CrossRefGoogle Scholar
  52. 52.
    Zhao G, Pietikäinen M (2007) Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans Pattern Anal Mach Intell 29(6):915–928CrossRefGoogle Scholar
  53. 53.
    Zivkovic Z (2004) Improved adaptive Gaussian mixture model for background subtraction. In: Proceedings of the 17th international conference on pattern recognition, 2004. ICPR 2004, Aug 2004, vol 2, pp 28–31Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Paulo Lobato Correia
    • 1
  • Peter K. Larsen
    • 2
  • Abdenour Hadid
    • 3
  • Martin Sandau
    • 4
  • Miguel Almeida
    • 1
  1. 1.Instituto de Telecomunicações, Instituto Superior Técnico, Universidade de LisboaLisbonPortugal
  2. 2.University of CopenhagenCopenhagenDenmark
  3. 3.University of OuluOuluFinland
  4. 4.Danish Institute of Fire and Security TechnologyHvidovreDenmark

Personalised recommendations