Biometrics in ambient intelligence

  • Massimo TistarelliEmail author
  • Ben Schouten
Original Research


The security concerns due to the September 11 and later terroristic attacks, fostered the development of more advanced techniques for biometric identification. This had a positive impact to research and deployment of these technologies, founding a basis for security-related applications. At the same time, the privacy concerns for the misuse of personal information have hindered the application of the same technologies wherever their introduction could not be enforced. The risk is for the scientific development to be blocked by contradictory needs, which, in turn, often derive from misconceptions or misunderstanding of the real potential of biometric technologies. Within this context, ambient intelligence allows to consider the relation between biometrics and privacy under a different perspective. In fact, as most of times we are able to maintain social relations without the need to know each other’s identity (consider for example the case of a customer relating with an attendant at a department store), in the same way biometric technologies can facilitate the man–machine interaction (to better provide useful services) without the need to determine the user’s full identity. Also in the case of security applications, most often may be sufficient to retrieve ancillary information about a subject rather than determining his/her identity. This paper analyzes the potential of biometric technologies within the general scope of ambient intelligence, trying to identify some key technological issues which may respond to privacy concerns. Some example applications are considered where by exploiting the information contained in biometric data, such as the facial expression or other, non visual, measurements, it is possible to better relate the user with the environment and provide a substantial input to drive the services provided, without compromising his privacy.


Biometrics Human machine interaction Pattern recognition  Cognition Computer vision Robotics 



Funding from the European Union COST action 2101 “Biometrics for Identity Documents and Smart Cards” and from the Sardinia Regional Research Authority are also acknowledged.


  1. Aarts E, Diederiks E (2006) Ambient lifestyle, from concept to experience. Bis Publishers, AmsterdamGoogle Scholar
  2. Ambekar O, Pauwels E, Schouten B (2007) Binding low level features to support opportunistic person identification, 28th Symposium on information theory, Enschede, The Netherlands, May 24–25Google Scholar
  3. AMIGO-Ambient Intelligence for the Networked Home Environment (2004) European project IST 004182 [online]. Available:
  4. Arandjelovic O and Cipolla R (2004) Face Recognition from face motion manifolds using Robust kernel resistor-average distance. In: Proceedings of FaceVideo, p 88Google Scholar
  5. Arun A, Ross, Karthik Nandakumar, Anil K Jain (2006) Handbook of Biometrics. Springer, ISBN: 978-0-387-22296-7Google Scholar
  6. Bicego M, Grosso EE, Tistarelli M (2006) Person authentication from video of faces: a behavioral and physiological approach using Pseudo Hierarchical Hidden Markov Models. In: Proceedings first international conference on Biometrics LNCS 3832, Hong Kong, China, pp 113–120Google Scholar
  7. Black J, Ellis J, Rosin J (2002) Multi-view image surveillance and tracking, IEEE Workshop on Motion and Video ComputingGoogle Scholar
  8. Busso C, Deng Z, Yildirim S, Bulut M, Lee, Kazemzadeh A, Lee S, Neumann U, Narayanan S (2004) Analysis of emotion recognition using facial expressions, speech and multimodal information. In: Proceedings of the 6th international conference on multimodal interfaces, (State College, PA, USA, ICMI 2004, pp 205–211.
  9. Castellano G, Kessous L, Caridakis G (2008) Emotion recognition through multiple modalities: face, body gesture, speech. In: Peter C, Beale R (eds) Affect and emotion in human computer interaction LNCS, vol 4868. Springer, HeidelbergGoogle Scholar
  10. CAVIAR<context aware vision using image-based active recognition (2002) European project IST 2001 37540 [online]. Available:
  11. Cehllappa R, Zhou SK (2004) Face tracking and recognition from video. In: Jain AK, Li SZ (eds) Handbook of face recognition. Springer, New York, pp 169–192Google Scholar
  12. Chen LS (2000) Joint processing of audio-visual information for the recognition of emotional expressions in humancomputer interaction. PhD Thesis, University of Illinois at Urbana, ChampaignGoogle Scholar
  13. Chen LS and Huang TS (2000) Emotional expressions in audiovisual human computer interaction. In: Proceedings of ICME 2000, New York, pp 423–426Google Scholar
  14. CHIL-Computers In the Human Interaction Loop (2004) European project IST 506909 [online]. Available:
  15. COGNIRON-the cognitive robot companion (2004) European project IST 002020 [online]. Available:
  16. Cohen I, Sebe N, Garg A, Chen LS, Huang TS (2003) Facial expression recognition from video sequences: temporal and static modeling. Comput Vis Image Underst 91(12):160187Google Scholar
  17. Cook DJ (2005) Prediction algorithms for smart environments in smart environments: technologies, protocols, and applications. In: Cook DJ, Das SK (eds) Series on parallel and distributed computing. Wiley, pp 175–192Google Scholar
  18. Cook Diane J, Das Sajal K (2005) Smart environments: technology, protocols and applications]. Wiley-Interscience, USAGoogle Scholar
  19. Cook D, Youngblood M, Heierman E, Gopalratnam K, Rao S, Litvin A, Khawaja F (2003) Mavhom: an agent-based smart home. In: Proceedings of IEEE PerCom, pp 521–524Google Scholar
  20. Datcu D, Rothkrantz L (2008) Automatic bi-modal emotion recognition system based on fusion of facial expressions and emotion extraction from speech. IEEE face and gesture conference FG2008, ISBN: 978-1-4244-2154-1Google Scholar
  21. Dee HM, Velastin SA (2007) How close are we to solving the problem of automated visual surveillance? A review of real-world surveillance, scientific progress and evaluative mechanisms. Springer, New YorkGoogle Scholar
  22. Eagle N, Pentland A (2006) Eigenbehaviors: identifying structure in routine, In: Proceedings of UbicompGoogle Scholar
  23. Ekman P (1994) Strong evidence for universals in facial expressions: a reply to Russells mistaken critique. Psychol Bull 115(2):268–287CrossRefGoogle Scholar
  24. Ekman P, Friesen WV (1978) Facial action coding system: investigators guide. Consulting Psychologists Press, Palo AltoGoogle Scholar
  25. Essa I, Pentland A (1995) Facial expression recognition using visually extracted facial action parameters. In: Proceedings of international workshop on automatic face and gesture recognition, Zurich, SwitzerlandGoogle Scholar
  26. Fasel B, Luettin J (2003) Automatic facial expression analysis: a survey. Pattern Recogn 36:259275CrossRefGoogle Scholar
  27. Fatakusi O, Kittler J, Poh N (2008) Quality controlled multimodal fusion of biometric experts in progress in pattern recognition, image analysis and applications Lecture notes Computer Science, vol 4756. Springer, Berlin, pp 881–890Google Scholar
  28. Fleuret F, Berclaz J, Lengagne R, Fua P (2008) Multi-camera people tracking with a prob-abilistic occupancy map. IEEE Trans Pattern Analysis Mach Intell 30(2):267–282CrossRefGoogle Scholar
  29. Friedewald M, Da Costa O, Punie Y, Alahuhta P, Heinonen S (2004) Perspectives of ambient intelligence in the home environment, telematics and informatics, vol 22. Elsevier publishing, The Netherlands, pp 221–238Google Scholar
  30. Gunes H and Piccardi M (2006) A bimodal face and body gesture database for automatic analysis of human nonverbal affective behavior. In: Proceedings of the 18th International Conference on Pattern Recognition (ICPR 2006), vol 1, pp 1148–1153Google Scholar
  31. Gunes H, Piccardi M (2009) Automatic temporal segment detection and affect recognition from face and body display. IEEE Trans Syst Man Cybern B, Special issue human comput 39(1):64–84Google Scholar
  32. Gunes H, Piccardi M, Pantic M (2008) From the lab to the real world: affect recognition using multiple cues and modalities. In: Jimmy O (ed) Affective computing, focus on emotion expression, synthesis and recognition, I-Tech Education and Publishing, Vienna, Austria, ISBN: 978-3-902613-23-3Google Scholar
  33. Hall D, Pelisson F, Riff O, Crowley JL (2004) Brand identification using gaussian derivative histograms. Mach Vis Appl 16(1):41–46CrossRefGoogle Scholar
  34. Haritaǒglu S, Harwood D, Davis L (2000) W4: real-time surveillance of people and their activities. IEEE Trans Pattern Analysis Mach Intell 22(8):809–830CrossRefGoogle Scholar
  35. Horn BKP, Schunck BG (1981) Determining optical flow. Artif Intell 17(1–3):185–204CrossRefGoogle Scholar
  36. Howell AJ, Buxton H (1996) Towards unconstrained face recognition from image sequences. In: Proceedings of the IEEE International conference on Automatic Face and Gesture Recognition (FGR 1996), Killington, VT, pp 224–229Google Scholar
  37. Jain AK, Dass SC, Nandakumar K (2004) Soft biometric traits for personal recognition systems, In: Proceedings of international conference on Biometric Authentication (ICBA), LNCS 3072, pp 731–738Google Scholar
  38. Kanade T, Cohn J, Tian Y (2000) Comprehensive database for facial expression analysis. In: Proceedings of the 4th IEEE international conference on automatic face and gesture recognition (FG 2000), pp 46–53Google Scholar
  39. Kang J, Cohen I, Medioni G (2004) Tracking people in crowded scenes across multiple cameras, Asian Conference on Computer VisionGoogle Scholar
  40. Kidd C, Orr R, Abowd G, Atkeson C, Essa I, MacIntyre B, Mynatt E, Starner T, Newstetter W (1999) The aware home: a living laboratory for ubiquitous computing research. In: Proceedings of CoBuild, pp 191–199Google Scholar
  41. Kittler J, Poh N, Fatakusi O, Messer K, Kryszczuk K, Richiardi J, Drygajlo A (2007) Quality dependent fusion of intramodal and multimodal biometric experts. In: Proceedings of SPIE defense and security symposium, Workshop on Biometric Technology for Human Identification, vol 6539Google Scholar
  42. Li Y, Gong S, Liddell H (2000) Support vector regression and classification based multiview face detection and recognition. In: Proceedings of the IEEE international conference on Automatic Face and Gesture Recognition (FGR 2000), Grenoble, France, pp 300–305Google Scholar
  43. Li Y, Gong S, Liddell H (2001) Modelling faces dynamically across views and over time. In: Proceedings IEEE international conference on Computer Vision, Vancouver, Canada, pp 554–559Google Scholar
  44. Liberty CCTV (2008) issues/3-privacy/32-cctv/index.shtml
  45. Linnartz JP, Tuyls P (2003) New shielding functions to enhance privacy and prevent misuse of biometric templates. In: Proceedings AVBPA 2003, LNCS: 2688, Springer, Berlin, pp 393–402Google Scholar
  46. Liu X, Chen T (2003) Video-based face recognition using adaptive hidden Markov models. In: Proceedings CVPR03 (I), pp 340–345Google Scholar
  47. Lucas SM, Huang TK (2004) Sequence recognition with scanning N-tuple ensembles. In: Proceedings ICPR04 (III), vol 135, pp 410–413Google Scholar
  48. Lucas SM (1998) Continuous n-tuple classifier and its application to real-time face recognition. In: IEE Proceedings-Vision Image and Signal Processing, vol 145, p 343Google Scholar
  49. Magnusson MS (2000) Discovering hidden time patterns in behavior: T-patterns and their detection. Behav Res Methods Instrum Comput 32(1):93–110CrossRefGoogle Scholar
  50. Maltoni D, Maio D, Jain AK, Prabhakar S (2008) Handbook of fingerprint recognition, 2nd edn. Springer, HeidelbergGoogle Scholar
  51. Matey JR, Naroditsky O, Hanna K, Kolczynski R, LoIacono D, Mangru S, Tinker M, Zappia T, Zhao WY, Iris on the Move™: acquisition of images for Iris recognition in less constrained environments. In: Proceedings of IEEE. 94(11):1936–47Google Scholar
  52. Maurer DE, Baker JP (2007) Fusing multimodal biometrics with quality estimates via a Bayesian belief network. Pattern Recogn 41(3):821–832CrossRefGoogle Scholar
  53. Mittal A, Davis L (2003) M2tracker: a multi-view approach to segmenting and tracking people in a cluttered scene. Int J Comput Vis 51(3):189–203CrossRefGoogle Scholar
  54. Mitsubishi Electric Research Laborotories: ambient intelligence for better buildings (2009) [online]. Available:
  55. Morros R, Salah Albert Ali, Schouten Ben, Perales Carlos Segura, Serrano Jordi Luque, Ambekar Onkar, Kayalar Ceren, Keskin Cem, Akarun Lale (2008) Multimodal identification and localization of users in a smart environment. J Multimodal Interfaces 2(2):75–91CrossRefGoogle Scholar
  56. Mozer MC (2005) Lessons from an adaptive home. In: Cook DJ, Das SK (eds) Smart environments: technologies, protocols and applications. Wiley Series on Parallel and Distributed Computing, New York, pp 273–294Google Scholar
  57. National Science and Technology Council (2007) Subcommittee on Biometrics (Duane Blackburn co-chair). The National Biometrics Challenge.
  58. Nixon MS, Tan TN, Chellappa R (2006) Human identification based on gait. Springer, New YorkCrossRefGoogle Scholar
  59. OXYGEN-MIT project Oxygen, Pervasive Human-Centered Computing (2001) MIT research project (2001). Available:
  60. Paleari M, Benmokhtar R, Huet B (2009) Evidence theory based multimodal emotion recognition MMM, 15th International Multimedia Modeling Conference, Antipolis, France, pp 435–446Google Scholar
  61. Pantic M, Rothkrantz LJM (2000) Automatic analysis of facial expressions: the state of the art. IEEE Trans PAMI 22(12):1424–1445Google Scholar
  62. Pauwels E, Salah AA, Tavenard R (2007) Sensor Networks for Ambient Intelligence, Workshop on Multimedia Signal Processing (MMSP), CreteGoogle Scholar
  63. Phillips PJ, Flynn PJ, Scruggs T, Bowyer KW, Worek W (2006) Preliminary face recognition grand challenge results. In: Proceedings 7th International Conference On Automatic Face And Gesture Recognition, pp 15–24Google Scholar
  64. Phillips PJ, Wechsler H, Huang J, Rauss P (1998) The FERET database and evaluation procedure for face-recognition algorithms. Image Vis Comput J 16(5):295–306CrossRefGoogle Scholar
  65. Poh N, Heusch G, Kittler J (2007) On combination of face authentication experts by a mixture of quality dependent fusion qualifiers. In: LNCS 4472, Multiple classifiers systems (MCS), Prague, pp 344–356Google Scholar
  66. Ratliff MS, Patterson E (2008) Emotion recognition using facial expressions with active appearance models. In: Proceedings of IASTED human computer interaction, pp 92–138Google Scholar
  67. Raytchev B, Murase H (2001) Unsupervised face recognition from image sequences. In: Proceedings ICIP01(I), pp 1042–1045Google Scholar
  68. Raytchev B, Murase HH (2002) VQ-Faces: unsupervised face recognition from image sequences. In: Proceedings ICIP02 (II), pp 809–812Google Scholar
  69. Raytchev B, Murase H (2003) Unsupervised recognition of multi-viewface sequences based on pairwise clustering with attraction and repulsion. Comput Vis Image Underst 91(1–2):22–52CrossRefGoogle Scholar
  70. Rejman-Greene M (2004) Privacy issues in the application of biometrics: a european perspective in biometric systems. In: Wayman J, Jain A, Maltoni D, Maio D (eds) Biometric systems. Springer, BerlinGoogle Scholar
  71. Rosalind W Picard (2000) Affective computing MIT Press, LondonGoogle Scholar
  72. Schouten B, Jacobs B (2008) Biometrics and their use in e-passports, image and vision computing (IMAVIS), special issue on multimodal biometrics, Elsevier Publishers, The NetherlandsGoogle Scholar
  73. Schouten BAM, Deravi F, García-Mateo C, Tistarelli M, Snijder M, Meints M, Dittmann J (2008) BioSecure: white paper for research in biometrics beyond BioSecure. CWI report 2008, PNA-R0803, ISSN:1386–3711Google Scholar
  74. Sebe N, Cohen I, Cozman FG, Gevers T, Huang TS (2005) Learning probabilistic classifiers for human computer interaction applications. Multimed Syst 10(6):484–498CrossRefGoogle Scholar
  75. Sebe N, Cohen I, Gevers T, Huang TS (2006) Emotion recognition based on joint visual and audio cues. In: Proceedings of the 18th international conference on pattern recognition (ICPR 2006), vol 1, pp 1136–1139Google Scholar
  76. Shan C, Braspenning R (2009) Recognizing Facial Expressions Automatically from Video. In: Nakashima H, Augusto J, Aghajan H (eds) Handbook of Ambient Intelligence and Smart Environments, SpringerGoogle Scholar
  77. Tabassi E, Wilson C, Watson C (2004) NIST Fingerprint Image Quality, NIST Res Rep NISTIR7151Google Scholar
  78. The BioSecure Network of Excellence (2004).
  79. The Ministry of the Interior and Kingdom Relations, The Netherlands (2005) 2b or not to 2b.
  80. Tistarelli M (1996) Multiple constraints to compute optical flow. IEEE Trans PAMI 18(12):1243–1250Google Scholar
  81. Tistarelli M, Bicego M, Grosso E (2008) Dynamic face recognition: from human to machine vision. In: Tistarelli M and Bigun J (eds) Image Vis Comput special issue on multimodal biometrics. doi: 10.1016/j.imavis.2007.05.006
  82. Tretiak O, Pastor L (1984) Velocity estimation from image sequences with second order differential operators. In: Proceedings of 7th IEEE international conference on pattern recognition, pp 16–19Google Scholar
  83. Velastin SA, Boghossian BA, Lo BPL, Sun J, Vicencio-Silva MA (2005) PRISMATICA: towards ambient intelligence in public transport environments. Part A. IEEE Trans Syst Man Cybern 35(1):164–182Google Scholar
  84. Waibel A, Steusloff H, Stiefelhagen R (2004) Chil—computers in the human interaction loop, NIST ICASSP Meeting Recognition Workshop, Montreal, CanadaGoogle Scholar
  85. Wayman J, Jain A, Maltoni D, Maio D (2004) Biometric systems. Springer, LondonGoogle Scholar
  86. Wechsler H (2007) Reliable face recognition methods: system design, implementation and evaluation. Springer, HeidelbergCrossRefGoogle Scholar
  87. Wren C, Minnen D, Rao S (2006) Similarity-based analysis for large networks of ultra-low resolution sensors. Pattern Recogn 39:1918–1931zbMATHCrossRefGoogle Scholar
  88. Video-based Threat Assessment and Biometrics Network - ViTAB (2006).
  89. Zeng Z, Pantic M, Roisman GI, Huang TS (2009) A survey of affect recognition methods: audio, visual, and spontaneous expressions. IEEE Trans PAMI 31(1):39–58Google Scholar
  90. Zhang DD (2004) Palmprint authentication. Springer, LondonGoogle Scholar
  91. Zhou SK, Krueger V, Chellappa R (2003) Probabilistic recognition of human faces from video. Comput Vis Image Underst 91(1–2):214–245CrossRefGoogle Scholar
  92. Zhou SK, Chellappa R, Moghaddam B (2004) Visual tracking and recognition using appearance-adaptive models in particle filters. Image Process 13(11):1491–1506CrossRefGoogle Scholar
  93. Zhou SK, Chellappa R, Zhao W (2006) Unconstrained face recognition. Springer, New YorkzbMATHGoogle Scholar

Copyright information

© Springer-Verlag 2010

Authors and Affiliations

  1. 1.Computer Vision LaboratoryUniversity of SassariAlgheroItaly
  2. 2.Fontys University of Applied SciencesEindhoven R1The Netherlands

Personalised recommendations