Introduction

Understanding nonverbal communication (NVC) is important in a range of situations and settings, from daily relationships to interrogations (Krauss et al. 1996). Research has shown that NVC includes both reflexive and nonreflexive movements of the body, which communicate an emotional message to others (Tamietto and De Gelder 2010). Correct interpretation of individuals’ emotional states and intentions is obviously important for effective communication; however, a particularly important area for NVC is in police and legal investigations. During a police interview or interrogation, investigators may implicitly or explicitly use NVC reading techniques to identify the concealment of emotions, which may indicate an intent to deceive or a line of questioning worth pursuing (Hartwig and Bond 2011; Mann et al. 2004). Behavioural cues of mental effort, memories and emotions are used by security personnel to identify behavioural ‘hot spots’ that indicate that a topic is worth considering and investigating further (Frank et al. 2008). Typically, research has focused on behaviours related to particular emotions or concealment of information (Ekman and O’Sullivan 1991). Sporer and Schwandt (2007) outline several reliable indicators of deception, such as nodding, movement of feet and legs and hand movements; however, they showed no systematic relationship between avoidance of eye contact and deception. However, researchers and professional practitioners in the field have suggested that clusters of behaviours are better indicators of changes in emotion and possible psychological discomfort, which may arise from attempting to deceive someone (DePaulo et al. 2003; Navarro and Karlins 2008, Navarro 2011; Vrij et al. 2004). The present research continues a more recent trend in NVC by investigating the possibility that it is not only discrete behaviours or clusters of movements that can be used to indicate emotion or distress but also the sequence that these movements occur in (Burgoon et al. 2014; Burgoon et al. 2015). Previous research that has focused on sequences of behaviour has used a software programme that detects patterns in time-ordered data. These T-patterns (see Casarrubea et al. 2015) are a multivariate approach to identifying temporal structure in behaviour. Burgoon et al. (2014), therefore, used a very similar statistical approach to the one taken in the current study; however, their series of studies focused on mock theft, cheating game behaviours and group discussions and deception. The current study uses a similar statistical approach, but with real-world deception.

Understanding and interpreting NVC typically start by identifying discrete, single behaviours, referred to as tells (Collett 2003; Ekman et al. 1991; Navarro and Karlins 2008). Hartwig and Bond (2011) outlined the leakage hypothesis as a means by which individuals leak informative tells while trying to conceal the truth. For instance, poker players who have become successful at reading NVC have the ability to identify particular behaviours of their opponents. This line of research, however, has limitations in terms of individual differences in the ability to mask emotion and deception (Blanck et al. 1981).

In contrast to looking at single behaviours or micro reactions, researchers have suggested that ‘clusters’ of behaviours are better indicators of changes in emotion and psychological distress, which may reflect attempts to deceive (DePaulo et al. 2003; Hartwig and Bond 2011; Navarro and Karlins 2008; Vrij et al. 2004). Therefore, simply recognising one single behaviour as an indication of emotions or lying is not completely reliable, though Hartwig and Bond (2011) highlight that while multiple cues may be better than single cues, individual behaviours still contribute a lot to the detection of deception. The cluster approach, which is based on more applied settings, outlines that NVC involves the changes of behaviours across the entire body (i.e., movement of feet, hands, face, etc.). This research has gained support from applied fields of forensic and investigative psychology and police work (Furnham and Taylor 2011). While the cluster approach has proven effective, there may be an additional step in terms of understanding the sequence of body movements. For instance, it may be possible that body movements occur in sequence (i.e., the feet change movement first, followed by the face). Research has shown that if an individual attempts to mask one behaviour, they may leak behaviours on other parts of their body (Ekman and Friesen 1969). Looking at patterns of behaviours in deceptive and truthful interactions has been previously researched by Burgoon et al. (2014, 2015). In their seminal research, patterns of nonverbal behaviour, across time, were analysed, showing clear differences between deceivers and truthful participants. Burgoon and colleagues’ research provides a clear foundation to begin future development in investigating patterns and sequences of behaviours, rather than individual behaviours or clusters. It is to this direction the present research builds on, by using real-world deceptive statements, rather than mock crime experiments.

Behaviour Sequence Analysis

Behaviour sequence analysis (BSA), also referred to as lag sequence analysis (LSA), is a useful method for understanding the dynamic relationship between progressions of behaviours and social interactions occurring over time (Beune et al. 2010; Keatley et al. 2016; Taylor et al. 2008). Sequence analysis typically involves three key stages (Clarke and Crossland 1985). First, unitisation involves taking a person’s whole response to a question and cutting it into discrete behaviours or events. Second, classification involves placing the behaviours or events into distinct categories that are functionally similar. Finally, analysis involves the statistical measurement of whether transitions between behaviours are occurring significantly above the level of chance. For example, if we imagine a simple chain of behaviours, question asked (category a), respondent shakes head (category b), respondent taps finger (category c) and respondent looks up and right (category d), first-order or lag-one behaviour sequence analysis tests whether ‘a-b’, ‘a-c’, b-c’ and ‘c-d’ pairs, for instance, are more or less likely to occur than by chance alone. Therefore, sequence analysis begins by first finding stimuli (e.g., video clips of individuals caught lying or telling the truth) that can be analysed. A coding scheme is then developed so that each individual behaviour shown in the stimuli can be categorised (e.g., ‘head-nod’, ‘eyes-looks down’). When an exhaustive and mutually exclusive list is developed, all stimuli are then coded. The sequence analysis then measures transitions between pairs of behaviours, to see if pairs occur above chance.

BSA is underpinned by Markov models, which involve studying transitions between behaviour pairs (Ivanouw 2007). The first event in a pairing is the antecedent (e.g., nods head); the second behaviour in a pair is the sequitur (e.g., taps finger). In the simplest form of BSA, the analysis determines whether the antecedent causes the sequitur to be more likely to occur, than expected by chance. Sequence analysis has been used in a variety of social interactions and behaviours, such as marital conflict (Gottman 1979), violent episodes between people (Beale et al. 1998; Turner and Clarke 2009) and rape cases, in relation to physical interactions (Fossi et al. 2005) as well as verbal interactions and strategies (Lawrence et al. 2010) between attacker and victim.

Present Study

The main aim of the current research is to provide a new method, behaviour sequence analysis, for understanding and researching NVC. Previous research has shown that the accuracy rates of trained professionals are modest, in terms of detecting deception through NVC techniques (Burgoon et al. 2014; Burgoon et al. 2015; Frank and Ekman 1997; Vauch et al. 2016). However, many NVC studies lack ecological validity (Richardson et al. 2000; Scriba et al. 1999). Although participants in lab-based experiments may be motivated towards an outcome, they are not at risk of major negative consequences for being exposed, as a criminal would be in a real investigation. Indeed, the absence of such high stakes might not produce a sufficient amount of anxiety or motivation for the person in question to display valid facial cues or NVC (Miller and Stiff 1993). Therefore, the current study used real-world recorded videos of people, rather than in a laboratory. The sample chosen were criminals, and/or people of high power, who were later unequivocally exposed as being guilty at the time of making the statements of innocence that were recorded and analysed. Owing to the nature of the current study, no formal hypotheses were made. However, it is likely that previous research into NVC will be supported as elements of larger sequences. There are also likely to be differences in the chains of behaviours exhibited between deceptive and truthful statements. Based on the research by Hartwig and Bond (2011) and Sporer and Schmidt (2007), behaviours related to head movements and hand movements are likely to be different between honest and dishonest statements, and clusters of behaviours are likely to occur, but with certain individual behaviours occurring more frequently in honest and dishonest statements.

Methods

Sample

Video-recorded interview clips of politicians, criminals and people of high interest were obtained through various media sources. Content of these clips were real-life examples of deception or honesty. Each clip contained statements made by participants in response to a question, which were later proven to be a lie or the truth. Clips came from popular online video websites and included scenes from press conferences through to police interrogations. A sample of videos of 19 individuals (15 male, 4 female) between the ages of 19–48 was collected via online websites and archive footage documents, for both groups. Multiple clips were used for the same individuals, in some cases. For the truthful response group, 12 criminals were analysed (11 males, 1 female).Footnote 1 For the deception group, a total of 51 clips were used. Length of clips ranged from 4 to 58 s (M = 16.05, SD = 11.64). For the truthful group, a total of 49 clips were used. Length of clip ranged from 6 to 57 s (M = 18.6, SD = 10.2). Clips were obtained through cross-referencing well-known cases of individuals caught lying with existing recordings of incidents in which they are clearly shown on tape lying. For both groups, lying about committing murder was the most frequent video clip used (n = 10 in dishonest group, n = 6 in honest group).

Inclusion criteria for each group were that the person was unequivocally exposed as lying or telling the truth by later investigation and evidence. Everyone in the study had no indication of any medical condition that would affect their body movements. The study was approved by the University of [omitted] Research Ethics Committee.

Coding Procedure

Behaviours were recorded in relation to a question and answer pattern. A question was asked; then once the participant began to answer with a response, behaviours were recorded until the end of the response. When a new question was asked, a new sequence of behaviours was analysed. Initially, a list of possible behaviours was developed from existing literature (Burgoon et al. 2014, 2015). This provided a baseline of behaviours that might occur when viewing the clips.Footnote 2 If any additional behaviours became apparent when viewing the clips, these behaviours were added to the list. Clips were slowed down and viewed frame by frame, in order to get a more accurate analysis of the sequence of NVC. The coding categories were mutually exclusive and exhaustive, which is a prerequisite of sequence analysis (Bakeman and Quera 2011). Each clip was analysed by two separate researchers involved in the study to ensure inter-rater reliability. Finally, the coding scheme was given to an expert in the field to assess and approve. Given the nature of the coding and behaviour sequence analysis, future research can be directly added to the current data to build larger data sets for analyses.

Statistical Analysis

After videos were coded into chains of discrete categories, data were input into the statistical software R (R Core Team 2013) and analysed using a behaviour sequence analysis programme that the researchers wrote. The programme calculated frequencies of individual behaviours, transitional frequencies, chi-squared (χ 2) statistics and standardised residuals.

Results

Deception Sequences

Analyses were conducted on 55 video sequences of a particular question-answer episode in which participants were exposed to be lying. Frequencies of behaviours that participants showed were first calculated (see Table 1). The most frequently occurring behaviours were participants shaking their head (n = 66), looking down (n = 58), nodding their head (n = 53) and furrowing their eyebrows (pulling their eyebrows down) (n = 44). These results support previous findings on deception related to NVC behaviours (Mann et al. 2002) (Table 2).

Table 1 Frequencies of behaviours—deception statements
Full size table
Table 2 Frequencies of behaviours—truthful statements
Full size table

The next, main stage of sequence analysis is to calculate transition frequencies between antecedents and sequiturs. A state transition diagramFootnote 3 was made on the data (see Fig. 1). The first thing to note with the state transition diagram is that a first-order, also known as lag one, sequence analysis, was conducted. Therefore, only links between behaviour pairs are analysed.Footnote 4 These pairs then form longer chains. The correct way to read the diagram is moving from one behaviour in single steps to a following behaviour, linked via an arrow. Whereas the diagram shows a full map from start to end, which is not to say any/all participants followed the same routes entirely, numbers beside the arrows indicate likelihood and frequency of transitions. Second, all transitions in the diagram are statistically significant (p < 0.05). Klonek and colleagues suggest a cut-off criteria of transitional frequencies below 5 being omitted from analyses for large data sets (Klonek et al. 2015); however, given the smaller data set of the current sample, a frequency cut-off of only showing transitions above frequency of 3 was used, to reduce complexity and only include meaningfully high transitions.

Fig. 1
figure 1

State transition diagram of behaviours exhibited during question response for deception sequences. All transitions shown are significant. Standardised residuals are given above or beside arrows; frequencies are shown in brackets. Position of boxes does not indicate temporal order—arrows indicate sequence of behaviours

Full size image

The diagram should be read one step at a time, between behaviours. For instance, after being asked a question, furrowing the eyebrows has a standardised residual (SR) of 1.2, and six individuals made that response, in the current data set. The n values in the diagram refer to the representation of transitions across the video clips, rather than individual people. After eyebrows have been furrowed, several different behaviours followed, for different individuals. For instance, after furrowing the eyebrows, head shaking is two times more likely to occur than chance, and nine individuals showed that link. These links illustrate the pairings of behaviours, rather than long (multiple behaviour) sequences. Of the original six people who showed furrowing eyebrows, several may have shown other behaviours next, not directly moving on to head shaking. Therefore, a first-order, or lag-one state transition, diagram shows only links between behaviour pairs, not longer chains.

Figure 1 shows that overall, the first significant behaviour displayed after a question has been asked, is the furrowing of eyebrow (SR = 1.2, n = 6). This behaviour leads to several following behaviours: head tilt sideways (SR = 1.2, n = 3), slam hands down (SR = 3.2, n = 3), tilt head forward (SR = 2.0, n = 3), shake head (SR = 2.0, n = 9) and eyes look down (SR = 1.4, n = 7). Head shaking was the most frequently occurring behaviour and had a direct link to end of the sequence (i.e., end of answering) (SR = 3.6, n = 14). This link was the most frequently occurring link in the diagram. This suggests that the final gesture individuals make before finishing a lying sequence was to shake their head. The behaviour ‘eyes-looking down’ was the next most frequently occurring behaviour, but was not directly linked to the end of the answer sequence. Instead, several individuals ended their answer sequence by nodding their head (SR = 1.2, n = 7) or pressing lips together (SR = 3.1, n = 6), with slightly fewer tightening their jaw (SR = 4.1, n = 4) or simply tilting head forward (perhaps in a half nod gesture) (SR = 1.2, n = 7).

Truthful Sequences

Analyses were also conducted on truthful responses. Frequencies of behaviours exhibited by honest respondents were calculated first (see Table 1). The most frequently occurring behaviours were eyes-looks up (n = 39) and head-nods (n = 35).

The next main stage of sequence analysis, to allow a contrast between deceptive and honest movements, was to conduct a sequence analysis on the honest group. The first notable difference between the two sequences is that the honest sequence state transition diagram has fewer behaviour exhibited. Within the sequences, there is no clear path between start and end of response; however, head tiling forward (SR = 4.1, n = 6) and head tilting sideways (SR = 2.0, n = 4) were the first significant behaviours exhibited by honest respondents, which is in contrast to dishonest responses, wherein furrowing of the eyebrows preceded both behaviours (Fig. 2).

Fig. 2
figure 2

State transition diagram of behaviours exhibited during question response for truthful sequences. All transitions shown are significant. Standardised residuals are given above or beside arrows; frequencies are shown in brackets. Position of boxes does not indicate temporal order—arrows indicate sequence of behaviours

Full size image

Discussion

The aim of the current research was to show the benefits of a behaviour sequence analysis approach to understanding NVC in relation to deception. The first part of the BSA, which shows frequencies of individual behaviours, indicates support for previous findings in the literature (Mann et al. 2002). The frequency analyses also indicate that deceptive individuals repeatedly demonstrated one behaviour particular to them, aka their ‘tell’, which they tended to repeat regularly when making a lying statement; this is why head shake, for instance, occurs so frequently. In contrast, honest responses typically exhibited fewer behaviours overall, which supports previous findings in the literature (Watson et al. 2016). In this way, the frequencies in the current research are analogous to previous research that has focused on individual tells. However, as Hartwig and Bond (2011) suggest, multiple cues may be better indicators of deception and honesty. Therefore, the next stage of the research was to investigate the sequence of behaviour pairs.

The state transition diagram for the deception sequences shows that the first behaviour following being asked a question was furrowing of the eyebrows. This may be an indicator of individuals needing to stop and think about their answer (Schmidt and Cohn 2001). This is also in contrast to the honest sequence diagram, in which furrowing of the eyebrows did not occur at all. In the deception sequences, several behaviours followed, which indicate further consideration and possible reflection. For instance, the head tilt sideways linked directly back to furrowing of the eyebrows and may be the case that these behaviours show an internal conflict requiring attention and consideration (Schmidt and Cohn 2001). This shows that head tilting sideways is not, in itself, an indicator of deception.

In contrast to people shaking their heads, it may be that some individuals are aware of this tell and deliberately nodded instead, to mask their knowledge. This finding indicates that nodding or shaking of the head alone are not conclusive indicators of deceit, which supports previous literature in the area of NVC (Mann et al. 2002). The role of eye movements was not clearly supported in the current findings, which supports the research of Sporer and Schwandt (2007). It should be noted that the current sample involved people who may be aware and deliberately masking body movements. Indeed, a strength of the cluster approach and sequence analysis approach to NVC is that is capitalises on tracking and highlighting multiple behaviours individuals might accidently make or leak when concentrating too much on controlling a particular body part.

There are several limitations and areas of future research stemming from the present study. First, the current analysis used a behaviour sequence approach to understand the progression of behaviours, which did not account for time, per se. Future research could focus on time-interval sequences, which might clarify the changes between micro tells (occurring very early in the sequence) compared to deliberative body movements (occurring after a few seconds have elapsed). A further limitation was the use of behaviours above chest height (i.e., mainly the face). Also, baseline measures were not an option in the current recordings, which were clips of only the lying answer. A further limitation was that multiple clips from the same person have been used in the current data set, which could lead to over-representation of particular cues. Future research should build on the current findings with different individuals, to reduce the potential impact of this issue. Future research should also include more women and men so that gender differences can be analysed. Future research could investigate a ‘pre-question’ sequence of behaviours and analyse whether there are significant changes in behaviours, which could indicate changes in emotion or deceit. A major strength of the current research, beyond the new method introduced, was the use of real-world data. The majority of NVC and deceit research is conducted in laboratory settings; the current method allows for real-world behaviours to be analysed. This also provides an opportunity for future research to compare sequences of actual behaviours caught on film compared to lab study behaviours. The outcome of this comparison might show that lab studies show very different sequences of behaviours, in contrast to real-world cases. Finally, future research could focus on verbal alongside nonverbal communication. Research using BSA has previously focused on verbal strategies (Lawrence et al. 2010), however, not in relation to deceitfulness of the statements. Therefore, a much needed progression in the literature is to investigate the role of verbal and nonverbal lies.

Conclusions

The current research will not only help to understand what behaviours are produced when a deceptive statement is constructed, but it may also determine the sequences of production and deduce potential critical behaviours within larger sequences of nonverbal chains. The sequence analysis approach allows us to see very clearly, changes in behaviours and NVC, which are an indication of changes within the individual as a result of the question that has been asked. This approach is a novel way of statistically analysing changes in behaviour and indicating what this may mean, for example, changes in emotion or reasons for emotional changes. Investigating temporal changes in NVC could be a useful investigation tool for interviewers or interrogators to use when analysing responses. Sequences of behaviours could be used to identify changes in behaviour as a reaction to questions and therefore indicate where to focus future further questioning, to investigate the cause of the behaviours.