Neural Computing and Applications

, Volume 16, Issue 4–5, pp 327–339 | Cite as

Charting the behavioural state of a person using a backpropagation neural network

  • Janet Rothwell
  • Zuhair Bandar
  • James O’Shea
  • David McLean
Original Article


This paper describes the application of a backpropagation artificial neural network (ANN) for charting the behavioural state of previously unseen persons. In a simulated theft scenario participants stole or did not steal some money and were interviewed about the location of the money. A video of each interview was presented to an automatic system, which collected vectors containing nonverbal behaviour data. Each vector represented a participant’s nonverbal behaviour related to “deception” or “truth” for a short period of time. These vectors were used for training and testing a backpropagation ANN which was subsequently used for charting the behavioural state of previously unseen participants. Although behaviour related to “deception” or “truth” is charted the same strategy can be used to chart different psychological states over time and can be tuned to particular situations, environments and applications.


Behaviour Deception Nonverbal Multichannels Backpropagation Chart 


  1. 1.
    Scherer KR, Ekman P (eds) (1982) Handbook of methods in nonverbal behavior research. Cambridge University Press, CambridgeGoogle Scholar
  2. 2.
    Zuckerman M, Driver RE (1985) Telling lies: Verbal and nonverbal correlates of deception. In: Siegman AW, Feldstein S (eds) Multichannel integrations of nonverbal behavior. L Erlbaum Associates, HillsideGoogle Scholar
  3. 3.
    Gavrila DM (1999) The visual analysis of human movement: a survey. Comput Vis Image Underst 3(1):82–98CrossRefGoogle Scholar
  4. 4.
    Turk MA, Pentland AP (1991) Face recognition using Eigenfaces. In: IEE computer society conference on computer vision and pattern recognition pp 586–591Google Scholar
  5. 5.
    Valentin D, Abdi H, O’Toole AJ, Cottrell GW (1994) Connectionist models of face processing: a survey. Pattern Recognit 27(9):1209–1230CrossRefGoogle Scholar
  6. 6.
    Cottrell GW, Metcalfe J (1991) Empath: Face, emotion and gender recognition using holons. In: Lippman et al (ed) Advances in neural processing systems 3. Morgan-Kaufmann Publishers Inc, San Mateo, pp 573–577Google Scholar
  7. 7.
    Golomb BA, Lawrence DT, Sejnowski TJ (1991) SEXNET: a neural network identifies sex from human faces. In: Lippman et al (ed) Advances in neural information processing systems 3. Morgan Kaufmann Publishers Inc, San MateoGoogle Scholar
  8. 8.
    Zhang M (1996) Face recognition using artificial neural network group-based adaptive tolerance (G.A.T.) trees. IEEE Trans Neural Netw 7(3)Google Scholar
  9. 9.
    Bartlett MS, Hager JC, Ekman P, Sejnowski TJ (1999) Measuring facial expressions by computer image analysis. Psychophysiology 36:253–263CrossRefGoogle Scholar
  10. 10.
    Rosenblum M, Yacoob Y, Davis LS (1996) Human expression recognition from motion using a radial bias function network architecture. IEEE Trans Neural Netw 7(5):1121–1138CrossRefGoogle Scholar
  11. 11.
    Fels S, Hinton GE (1998) Glove Talk 2, a neural network interface which maps gestures to parallel formant speech synthesizer controls. IEEE Trans Neural Netw 9(1):205–212CrossRefGoogle Scholar
  12. 12.
    Rothwell J (2002) Artificial neural networks for psychological profiling using multichannels of nonverbal behaviour. PhD Thesis, Manchester Metropolitan UniversityGoogle Scholar
  13. 13.
    Pavidis I, Eberhardt NL, Levine JA (2002) Human behavior Seeing through the face of deception (brief communication). Nature 415:35CrossRefGoogle Scholar
  14. 14.
    Cohn JF, Katz GS (1998) Bimodal expression of emotion by face and voice. In: Proceedings of the sixth ACM international conference on multimedia: workshop on face/gesture recognition and their applications. ACM, New York, pp 41–44Google Scholar
  15. 15.
    Ekman P (1988) Lying and nonverbal behavior: theoretical issues and new findings. J Nonverbal Behav 12:163–175CrossRefGoogle Scholar
  16. 16.
    Johnson RC (1999) Computer program recognizes facial expressions. Electronic Engineering Times, pp 51Google Scholar
  17. 17.
    Bartlett MS, Donato G, Movellan JR, Hager JC, Ekman P, Sejnowski TJ (1999) Face image analysis for expression measurement and detection of deceit. In: Proceedings of the 6th annual joint symposium on neural computation. The Institute For Neural Computation, University of California, San DiegoGoogle Scholar
  18. 18.
    Iwano Y, Sukegawa H, Kasahara Y, Shirai K (1995) Extraction of speaker’s feeling using facial image and speech. In: IEEE international workshop on robot and human communication, pp 101–106Google Scholar
  19. 19.
    Rothwell J, Bandar Z, O’Shea JD, McLean D (2006) Silent talker: a new computer-based system for the analysis of facial cues to deception. Appl Cogn Psychol (in press)Google Scholar
  20. 20.
    Freud S (1959) Fragments of an analysis of a case of hysteria, collected papers, Vol. 3 Basic books, New York (Original work published 1905)Google Scholar
  21. 21.
    Vrij A (1994) The impact of information and setting on detection of deception by police detectives. J Nonverbal Behav 18:117–136CrossRefGoogle Scholar
  22. 22.
    Stone M (1974) Cross-validatory choice and assessment of statistical predictions. J R Stat Soc Ser B 36:111–147zbMATHGoogle Scholar
  23. 23.
    Hassoun MH (1995) Fundamentals of neural networks. MIT, CambridgezbMATHGoogle Scholar
  24. 24.
    McLean D (1995) Improving generalisation in continuous data domains. Ph.D. Thesis, Manchester Metropolitan UniversityGoogle Scholar
  25. 25.
    Rumelhart DE, McClelland J (1986) Parallel distributed processing. In: PDP Research Group (eds) Explorations in the micro structure of cognition Vol 1, Bradford Books, MIT, Cambridge, MAGoogle Scholar
  26. 26.
    Wessels LFA, Barnard E (1992) Avoiding false local minima by proper initialization of connections. IEEE Trans Neur Netw 3(6):899–905CrossRefGoogle Scholar
  27. 27.
    Kolen JF, Pollack JB (1991) Back propagation is sensitive to initial conditions. In: Lippmann et al (eds) Advances in neural information processing systems 3. Morgan Kaufmann, San Mateo, pp 860–867Google Scholar
  28. 28.
    McLean D, Bandar Z, O’Shea J (1997) The evolution of a feed forward neural network trained under backpropagation. In: Proceedings of the third international conference on artificial neural network and genetic algorithms (ICANGA97). Springer, Berlin Heidelberg New York, pp 518–522Google Scholar
  29. 29.
    Plaut D, Nowlan SJ, Hinton GE (1986) Experiments on learning by back propagation. Computer Science Department, Carnegie Mellon University, Pittsburgh, PA 15213Google Scholar
  30. 30.
    Schraudolph NN, Sejnowski TJ (1996) Tempering backpropagation networks: not all weights are created equal. In: Touretzky et al (eds) Advances in neural information processing systems 8. MIT, CambridgeGoogle Scholar
  31. 31.
    Cybenko G (1989) Approximation by superpositions of a sigmoidal function. Math Control Signals Syst 2:303–314zbMATHCrossRefMathSciNetGoogle Scholar
  32. 32.
    Cybenko G (1998) Continuous values neural networks with two hidden layers are sufficient. Technical Report, Department of Computer Science, Tufts University, Medford, MAGoogle Scholar
  33. 33.
    McCormick D, Roach A (1987) Measurement, statistics and computation. Wiley, LondonGoogle Scholar
  34. 34.
    Bond CF (1990) Lie detection across cultures. J Nonverbal Behav 14:189–204CrossRefGoogle Scholar
  35. 35.
    Feldman RS, Rime B (1991) Fundamentals of non-verbal behavior. Cambridge University Press, CambridgeGoogle Scholar
  36. 36.
    Vrij A (2000) Detecting lies and deceit: the psychology of lying and the implications for professional practice. Wiley, New YorkGoogle Scholar
  37. 37.
    Porter S, Woodworth M, Birt AR (2000) Truth, lies and videotape: an investigation of the ability of federal parole officers to detect deception. Law Hum Behav 24:6:643–658CrossRefGoogle Scholar
  38. 38.
    Ekman P, O’Sullivan M, Friesen WV, Scherer KR (1991) Invited Article: face, voice and body in detecting deceit. J Nonverbal Behav 15:125–135CrossRefGoogle Scholar
  39. 39.
    Ekman P (1985) Telling lies: clues to deceit in the marketplace, politics and marriage. W.W. Norton and Company, New YorkGoogle Scholar
  40. 40.
    Williams D (2001) How to sting the polygraph. Sting Publications. P.O. Box 720568, Norman, OK, USAGoogle Scholar
  41. 41.
    Singh M, Vaid J, Sakhuja T (2000) Reading/writing vs handedness influences on line length estimation. Brain Cogn 43:398–402Google Scholar
  42. 42.
    Reuter-Lorenz PA, Kinsbourne M, Moscovitch M (1990) Hemispheric control of spatial attention. Brain Cogn 12:240–266CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2006

Authors and Affiliations

  • Janet Rothwell
    • 1
  • Zuhair Bandar
    • 1
  • James O’Shea
    • 1
  • David McLean
    • 1
  1. 1.Department of Computing and MathematicsManchester Metropolitan UniversityManchesterUK

Personalised recommendations