1 Introduction

By now, decision-making is a well-established topic of interest in the field of entrepreneurship research (e.g., Shepherd 2011; Shepherd et al. 2015). Entrepreneurs are required to make many decisions on a day-to-day basis and usually do so under conditions of high risk and uncertainty (e.g., Baron 1998). Given the inherent uncertainty involved in running a business and the decisions that come with it, entrepreneurs often rely on a set of flexible decision-making principles (Sarasvathy 2001, 2009; Dew et al. 2009). The use of heuristics is seen as a way to speed up the decision-making process (Busenitz and Barney 1997; Tversky and Kahneman 1974). However, if a decision concerns other individuals, and the entrepreneur lacks the desire or ability to retrieve more information about the opposing party, stereotyping can play a key role in shaping the entrepreneur’s judgments and decisions (Bodenhausen 1990, 1993; Greenwald and Banaji 1995). This is especially true if the decision-making process is influenced by one or more social interactions between the entrepreneur and whom it concerns (e.g., Huang et al. 2013; Loewenstein et al. 1989). Obviously, decisions that are relevant to entrepreneurs are not always made by entrepreneurs themselves (Baron and Markman 2000; Lechler 2001). Especially funding decisions by investors have also attracted a lot of attention in the extant entrepreneurship literature (e.g., Chen et al. 2009; Huang and Pearce 2015).

While many decisions in the context of entrepreneurship are made with little or no social interactions involved, others are largely based on or at least influenced by human-to-human interaction. The available evidence unequivocally suggests that behavioral cues during such interactions strongly affect the ultimate decision (Ambady and Rosenthal 1992; Bonaccio et al. 2016; McNeill 1992, 2005). This has also been demonstrated extensively for decision-making in entrepreneurial contexts that involves social interactions. In the case of entrepreneurs making the decision, most attention has been paid to hiring decisions or employment decisions more broadly (e.g., Hollandsworth et al. 1979; Koch et al. 2015). For example, Barrick et al. (2009) conclude that self-presentation tactics by applicants influence the interviewers’ perceptions of the candidate, and, in turn, whom they hire (also see Hosoda et al. 2003). In the case of decisions made by others yet affecting entrepreneurs, a vast amount of studies focuses on how business angels and venture capitalists arrive at their investment decisions (e.g., Maxwell et al. 2011; Petty and Gruber 2011). Many of those have investigated entrepreneurial pitches as a particular setting in which investors judge business ideas and note that their funding decisions are generally influenced by what is being said and done by the entrepreneurs (Ciuchta et al. 2018; Clarke 2011; Clarke et al. 2019; Pollack et al. 2012). In particular, both the verbal content of the presentation and the presentation style are considered important (Chen et al. 2009; Clark 2008). Hence, both verbal and nonverbal behavioral cues during social interactions are shown to have a major influence on individuals’ decision-making processes in the context of entrepreneurship.

Theoretical insights from both signaling theory (e.g., Busenitz et al. 2005; Connelly et al. 2011) and social exchange theory (e.g., Huang and Knight 2017) have proven helpful in understanding the impact of social signals being sent during human interactions (Ciuchta et al. 2018). Prior research has predominantly used manual techniques to analyze social signals (Vinciarelli et al. 2008). However, those methods have a number of important limitations. First, manual coding is relatively time-consuming and inherently arbitrary or, at best, ambiguous. Analysis of interactions by independent raters (e.g., Maxwell et al. 2011) is prone to cognitive biases in human decision-making (Zhang and Cueto 2017). Second, conventional methods are unable to derive characteristics from nonverbal behavior that may be important in explaining decisions. Think of micro expressions—that is, more spontaneous and subtle facial movements that occur involuntarily—which tend to reveal individuals’ genuine and underlying emotions (Ko 2018). Third, the relatively small samples—also often obtained from a specific context—limit the external validity of most studies as well as the number of relationships that can be tested empirically, which leaves many questions unanswered (Pollack et al. 2012).

In order to better understand decision-making in entrepreneurial contexts that involves social interactions, and to overcome the abovementioned limitations, we encourage entrepreneurship scholars to utilize the available state-of-the-art technologies instead. The emerging data science domain of social signal processing (henceforth SSP) encompasses a number of modern technologies using artificial intelligence (AI) that are able to automatically detect and analyze humans’ behavioral cues (Pentland 2010; Vinciarelli et al. 2009a, b, 2012). Among others, social signals are manifested through an array of nonverbal behavioral cues, such as gestures, body postures, facial expressions, eye behavior, and vocal behavior. Accurate computerized analysis of a wide range of social signals has recently become possible due to technological advancements (Vinciarelli et al. 2012). A few recent entrepreneurship studies have already applied various data science techniques to answer entrepreneurship research questions (e.g., Obschonka and Fisch 2017; Obschonka et al. 2017; Wang et al. 2017). However, the promise of data science methods for entrepreneurship research reaches much further and hence, we demonstrate how SSP techniques can advance our understanding of important decisions by entrepreneurs (i.e., entrepreneurial decisions) as well as by others yet directly affecting entrepreneurs (e.g., investment decisions).

A number of important decisions in the context of entrepreneurship are strongly affected by what happens during human-to-human interactions (e.g., board meetings, job interviews, entrepreneurial pitches). Processing the social signals sent by all parties involved unlocks valuable information for entrepreneurship researchers investigating how entrepreneurs and individuals assessing entrepreneurs (e.g., investors) arrive at their decisions. Given the dyadic nature of social interactions (Postma-Nilsenová and Postma 2013), one needs to detect and analyze social signals to and from entrepreneurs for the most comprehensive understanding. In doing so, automatic coding has clear advantages over manual coding. SSP is deemed more accurate and more efficient—thus better able to handle large amounts of data—and it can generate aspects of individuals’ nonverbal and verbal behavior that would have remained unidentified otherwise (e.g., micro expressions). Hence, we contend that contemporary technologies utilizing AI in the domain of SSP deserve more scholarly attention in the field of entrepreneurship research.

We develop a conceptual framework that provides insights into the direct and indirect relationships between behavioral and nonbehavioral cues on the one hand, and decisions that are relevant for entrepreneurs on the other hand. The framework essentially shows that such decisions are directly affected by nonbehavioral cues (e.g., demographics, personality traits), which might already be known before the interaction takes place, and by behavioral cues during the interaction, which might be nonverbal and verbal. Moreover, it is suggested that both nonverbal and verbal behaviors moderate the effects of nonbehavioral cues. In other words, nonverbal behavior like facial expressions and gesturing as well as the verbal content during the interaction can change people’s prior assessment of an individual’s characteristics and specific qualities for better or for worse, and, hence, their subsequent decision. Finally, recent evidence points at nonverbal behavior as a means to emphasize certain parts of the verbal content of speech (Clarke et al. 2019) and so, we propose a moderating effect of nonverbal behavioral cues on the relationship between verbal behavior and decisions relevant to entrepreneurs. Based on the conceptual framework, we theoretically derive a total of five propositions that we hope will be subject to thorough empirical analysis in future studies.

2 Theoretical background

2.1 Data science and entrepreneurship research

Up till now, data science and entrepreneurship have been rather distinct fields of research. Surprisingly so, as the rise of technologies collecting, storing, analyzing, and visualizing large amounts of data (Zomaya and Sakr 2017) potentially brings unprecedented opportunities for both entrepreneurs and entrepreneurship scholars. Although some have posed critical questions about the phenomenon of big data (Boyd and Crawford 2012), and some have highlighted the computational and statistical challenges that come with analyzing big data (Chen and Zhang 2014; Fan et al. 2014), scholars from different research disciplines have also started exploring its huge potential. For example, in the behavioral and social sciences at large (e.g., Harari et al. 2017; Mahmoodi et al. 2017) and in the economics (Einav and Levin 2014; Glaeser et al. 2018), management (George et al. 2014), and psychology (Gosling and Mason 2015; Kosinski et al. 2016) domains in particular. In the meantime, data science also found its way into entrepreneurship research but research at the intersection of both fields remains limited.

One has to distinguish between (1) digital technologies and (big) data providing opportunities for entrepreneurs to create new value (e.g., Nambisan 2017; Von Briel et al. 2018), and (2) technologies providing opportunities for entrepreneurship scholars to tackle conventional research questions in new ways (e.g., Uy et al. 2010; Wang et al. 2017). In the first case, research focuses on how individuals, both inside and outside the boundaries of established firms, act upon the opportunities that the data revolution and fast-paced technological development bring. For instance, Chen et al. (2017) describe the case of the German airline Lufthansa, and how they used big data to renew their business model. Moreover, Hartmann et al. (2016) derive a taxonomy of data-driven business models used by start-up firms and find six different clusters that consist of largely homogeneous sets of companies. In the second case, researchers take advantage of modern technologies, which are able to automatically collect and analyze—potentially enormous amounts of—data produced by and about humans, machines, and their interactions. New insights into entrepreneurial phenomena are then generated by means of novel research methods. For example, Obschonka (2017) highlights the potential of psychological big data for research on the antecedents and consequences of an entrepreneurial culture. This paper also belongs to the latter category, since we provide a detailed introduction to contemporary technologies utilizing artificial intelligence, which enable entrepreneurship researchers to do better research on decision-making processes that are relevant to entrepreneurs.

2.2 Decision-making in entrepreneurial contexts involving social interactions

In a generic sense, decision-making can be defined as the “entire process of choosing a course of action” (Hastie 2001: 657; McMullen and Shepherd 2006). Decisions in entrepreneurial contexts are either made by entrepreneurs themselves or by other individuals, but then, these decisions are of immediate impact on entrepreneurs. The former is often referred to as entrepreneurial decision-making, which, to date, is a rather established field of entrepreneurship research (Shepherd 2011; Shepherd et al. 2015). The latter may refer to policymakers shaping the institutional context that entrepreneurs face (e.g., Gnyawali and Fogel 1994). Another relevant stream in the entrepreneurship literature focuses on investors making decisions about whether or not to offer funding to new ventures (e.g., Busenitz et al. 2005; Franke et al. 2006; MacMillan et al. 1985; Woike et al. 2015).

Research on how entrepreneurs make their decisions has been of particular interest to entrepreneurship scholars given the extraordinary decision-making context that entrepreneurs are subject to (e.g., Shepherd et al. 2015). Entrepreneurs usually operate in complex environments and face high risk and uncertainty, emotional intensity, and time pressure, which have been shown to affect their decision-making (Baron 1998, 2008; Mullins and Forlani 2005). Moreover, entrepreneurs have to fulfill a broad range of tasks and activities, which requires them to make decisions on an almost continuous basis (McGee et al. 2009). These decisions vary from relatively minor, day-to-day issues to choices that are more fundamental for the firm’s performance. The extant literature on entrepreneurial decision-making so far mostly focused on the strategic decision-making of firms (e.g., Amason 1996; Busenitz 1999; Dean and Sharfman 1993; Forbes 2005), and how entrepreneurs’ decision-making differs from that of other entrepreneurs (e.g., Baron 2006) and non-entrepreneurs (e.g., Baron 1998; Mitchell et al. 2002). Shepherd et al. (2015) categorize the most important decision-making topics in the entrepreneurship literature and classify opportunity assessment decisions, opportunity exploitation decisions (e.g., Choi and Shepherd 2004), entrepreneurial entry decisions, and entrepreneurial exit decisions among them.

The focus of this paper, however, is on decision-making in entrepreneurial contexts that involves face-to-face interaction with one or more other individuals. A number of important entrepreneurial decisions are only made after one or more meetings with others took place, and their effect on the ultimate decision is not to be underestimated. In fact, social interaction and the emotions that come with it are said to play a key role in entrepreneurial thinking and action (Fischer and Reuber 2011; Goss 2005).

Examples of entrepreneurial decisions that clearly involve social interactions include whom to hire (e.g., Dahl and Klepper 2015), and whom to add to an entrepreneurial team (e.g., Forbes et al. 2006). Each of these decisions is of vital importance for firm survival and growth. First, employees represent the firm’s human capital on top of that of the founding entrepreneurs, which has often been linked to the growth and longevity of (new) firms (Bates 1990; Bosma et al. 2004; Colombo and Grilli 2005). Hiring the right employees is argued to be especially important in the early stages of a firm, i.e., when the number of employees is still likely to be small (Coad et al. 2017; Cooper et al. 1994; Koch et al. 2013). Second, the effectiveness of entrepreneurial teams is highly dependent on the abilities and personalities of their members (Barrick et al. 1998; Klotz et al. 2014; Mathieu et al. 2008). Indeed, the composition of entrepreneurial teams is strongly associated with the performance of the team itself and the (new) venture at large (Chandler and Lyon 2001; Francis and Sandberg 2000). Hence, it is of the utmost importance that these entrepreneurial decisions are based on information that is as timely and accurate as possible.

In the case of large organizations, the interactions do not necessarily involve members of top management teams but may also take place at lower levels of the organization. Foss et al. (2007) characterize the internal organization of the firm as a nested hierarchy of judgment (Foss and Klein 2012). Here, entrepreneurs are the source of primary or original judgment that may delegate entrepreneurial decisions to their subordinates, referred to as derived judgment. Differences in firm performance are partly explained by the entrepreneurs’ ability to exercise original judgment and to delegate derived judgment. Entrepreneurial activities may also be performed by the so-called proxy entrepreneurs based on delegated decision rights (Foss et al. 2007). In the case of a hiring decision, it may be one or more members of the HR department that have the ultimate say in whether or not to make a job offer to an applicant. Furthermore, each middle manager within a large organization is usually allowed to decide on how to form corporate entrepreneurial teams. In smaller businesses, there usually is no such distinction between primary and derived judgment. Rather, all entrepreneurial decisions are taken by the original entrepreneur(s). Hiring someone new may in practice even be the same act as adding a new entrepreneurial team member.

Funding decisions by investors are almost by definition made after some form of social interaction with the entrepreneurs or team of entrepreneurs took place. One particular setting that facilitates interaction between new venture teams and prospective investors is where entrepreneurs have the opportunity to convey their business idea in an entrepreneurial pitch (Ciuchta et al. 2018; Pollack et al. 2012). Most often, investment decisions are made by bankers (e.g., Mason and Stark 2004), business angels (e.g., Clark 2008), or venture capitalists (e.g., Chen et al. 2009; Shepherd et al. 2003; Zacharakis and Shepherd 2001). An often-mentioned alternative form of entrepreneurial financing is crowdfunding (e.g., Belleflamme et al. 2014). However, by construct, crowdfunding decisions do not require prior face-to-face interaction, as it is virtually impossible for entrepreneurs to face all crowdfunders (Bi et al. 2017).

2.3 Behavioral cues during social interactions

Most often, the aforementioned examples of decisions in entrepreneurial contexts are not solely based on a defined set of prior information (for example, derived from an applicant’s resume), but also on what is being said and done during a real-life meeting (for example, a job interview). During those meetings, individuals tend to behave in ways that increase their chances for a favorable decision by the entrepreneur, both verbally and nonverbally (Gilmore et al. 1999). Also, entrepreneurs themselves often try to persuade others, for example to receive (additional) funding, both through verbal and nonverbal behavior (e.g., Chen et al. 2009; Clark 2008) and through the use of visual symbols or gestures (Clarke 2011; Clarke et al. 2019). Impression management behaviors are argued to increase others’ perceptions of the legitimacy of the firm (Fisher et al. 2017; Nagy et al. 2012), which, in turn, may lead to increased funding (Pollack et al. 2012).

It has indeed been shown that certain behavior during interpersonal interactions can have a major influence on decisions made by the counterparty. For example, a meta-analysis by Higgins et al. (2003) reveals that self-promotion and other impression management practices positively affect the evaluation by job interviewers. Barrick et al. (2009) confirm that self-presentation tactics positively influence ratings by job interviewers. However, they conclude to say that what interviewers see may not be what the firm ultimately gets, i.e., after appointment. Especially unstructured job interviews appear to be susceptible to influence tactics by applicants. Based on another meta-analysis, Hosoda et al. (2003) suggest that physically attractive employees obtain more positive job-related outcomes than those who are less attractive, including performance appraisal, compensation levels, and even the choice for a business partner. Decades ago, Dion et al. (1972) already provided evidence for the existence of a so-called halo effect, in this particular case the tendency of people to think that what is beautiful is also good (also see Eagly et al. 1991; Morrow 1990). A physically attractive person is also sometimes said to possess beauty capital, but in any case, research consistently finds a positive link with success (Pfann et al. 2000; Price 2008).

Observers are known to readily attribute all kinds of personality traits, abilities, and emotions to others based on limited sets of information, that is, not only from someone’s physical appearance but also on, for example, the speech style of the observed party (Warner and Sugarman 1986). Stereotypes are known to influence social perceptions and, in turn, subconsciously shape individuals’ judgments and decisions (e.g., Greenwald and Banaji 1995). Vinciarelli et al. (2009a) rather talk about somatotypes, i.e., judgments typically influenced by stereotypes that do not necessarily correspond to reality. Still, they strongly influence social perceptions and cause people to attribute certain personality traits to others without knowing them personally. The heuristic view of stereotyping states that “people use their oversimplified beliefs about social groups as a basis for responding to the members of those groups whenever they lack the desire or the ability to engage in more extensive thought about the individuals” (Bodenhausen 1993). Hence, stereotypes can be seen as judgmental heuristics, and a way to easily process the available information in order to make decisions (Bodenhausen 1990; Bodenhausen and Wyer 1985).

Also, entrepreneurs are said to make use of heuristics in their decision-making as a way to deal with conditions of uncertainty (e.g., Busenitz and Barney 1997; Tversky and Kahneman 1974). Hence, readily observable behavioral cues during interactions with others are likely to affect entrepreneurial decisions. It is especially interesting to know how nonverbal behavioral cues enter the entrepreneurial decision-making process, as the conclusions drawn from them not necessarily correspond with reality. Likewise, investors often rely on their gut feel when opposing nascent entrepreneurs and, hence, may also be susceptible to the consequences of using heuristics or their intuition more broadly (Huang and Pearce 2015).

2.4 Social signal processing and research on decision-making

SSP refers to the digital processing and analysis of the social signals emitted by humans, particularly in a nonverbal way (Pentland 2004, 2007, 2010; Vinciarelli et al. 2008, 2009a). For instance, the face emits facial expressions that signal interest, determination, friendliness, boredom, or other attitudes (Pentland 2007). On a more abstract level, the expression of social signals relates to categories like activity, engagement, emphasis, and mirroring, which can then be used in predicting behavior and decisions (Pentland 2007). SSP has been successfully applied in assessing and predicting people’s interest in products, in analyzing business negotiations, in predicting the outcome of elevator pitches, and in predicting the results of job interviews, albeit in a more general context (Pentland 2004, 2007).

In many previous studies, coders or raters are used to assess and manually categorize the social signals sent by all individuals involved (e.g., Clarke et al. 2019; Maxwell et al. 2011). SSP has a number of advantages as compared with those more traditional data processing methods. First, SSP is more accurate, since automated analysis is less prone to errors that humans do tend to make, for example because of fatigue or boredom. As a result, SSP generates less biased data, if not unbiased. Second, SSP can easily process large volumes of data through machine learning algorithms. Given the digital nature of the process, scaling up the sample size will not increase the efforts and costs of data analysis tremendously, and, hence, is much more efficient. Third, SSP might identify characteristics that cannot be observed as easily by human observers, if at all. Software used to analyze humans’ behavior generally detects many different cues simultaneously, which are sometimes based on very subtle expressions or movements. It could well be that these characteristics turn out to be crucial in decision-making processes that strongly affect entrepreneurial performance.

Vinciarelli et al. (2009a) present a taxonomy of nonverbal behavioral cues associated with important social signals (also see Vinciarelli et al. 2008, 2009b, 2012). The behavioral cues are classified into the following five categories:

  1. 1.

    Physical appearance (e.g., height, attractiveness, and body shape)

  2. 2.

    Gestures and posture (e.g., hand gestures, posture, and walking)

  3. 3.

    Face and eye behavior (e.g., facial expressions, gaze behavior, and focus of attention)

  4. 4.

    Vocal behavior (e.g., prosody, turn taking, vocal outbursts, and silence)

  5. 5.

    Space and environment (e.g., distance and seating arrangement)

Computer vision is the technology that is used for the automatic detection of the behavioral cues belonging to the first three categories and the fifth category. To detect the cues belonging to the first two categories as well as facial expressions, one may also use biometry. Finally, speech analysis is used to detect all vocal behaviors (i.e., the fourth category).

Many of the aforementioned behavioral cues may fulfill certain functions, such as impression management and emotion expression (Vinciarelli et al. 2012). Moreover, these cues may reveal, for example, deceit and deception. Overall, SSP addresses both the psychological level, i.e., how, when, and why social signals are expressed (via nonverbal behavioral cues, as outlined above), and the computational level, where social signals are analyzed via automatic detection and interpretation methods and/or synthesized via the automatic generation of cues and user interaction (Vinciarelli et al. 2012).

SSP has many applications, ranging from enhancing virtual meetings (i.e., via smartphones or other mobile scenarios) to marketing, negotiations, and coaching (Vinciarelli et al. 2012; Salah et al. 2011). Particularly for the latter areas, effective nonverbal communication is crucial, since the perception of a person’s personality, motivations, and trustworthiness considerably influences the reception and respective decisions by the participants. The importance of effective nonverbal communication is more pronounced for entrepreneurs and investors due to the extraordinary decision-making context with high levels of uncertainty that they face (Shepherd et al. 2015). This relates to significant biases in entrepreneurs’ encounters with other individuals, and SSP can be applied for observing, modeling, and/or controlling for such biases and the according behavior. In coaching, for example, the motivation level can be tuned to the expressed social signals of the coaches, while, for marketing, the social signals of the target group need to be considered and assessed similarly in order to enable a useful and effective approach.

SSP is enabled using various technologies, e.g., by video capture, images, or capture of speech. Furthermore, specialized sensor devices can be used, such as the sociometric badge (Wu et al. 2008), for measuring movement, space, environment, and vocal behavior. A further relatively non-intrusive wearable device is the OpenBeacon sensor provided by the SocioPatterns consortium (Barrat et al. 2010). Using that, movement, space, and environment can be captured, which provides a simple yet effective method for observing physical social signals and analysis of those signals (e.g., Atzmueller 2014).

3 A conceptual framework on decision-making involving social interactions

The conceptual framework in Fig. 1 aims to further clarify how individuals make decisions that are highly relevant to entrepreneurs and that involve social interaction.

Fig. 1
figure 1

Conceptual framework

Full size image

In general, one needs complete and truthful information in order to be able to make the best decision. However, the available information is often incomplete—people tend to omit information that can harm their image (Gilmore et al. 1999)—and hardly ever (fully) reliable (Barrick et al. 2009). Impression management behaviors are widely applied (Nagy et al. 2012), and some individuals come better prepared than others (Chen et al. 2009; Pollack et al. 2012). Nevertheless, some of the social signals cannot be controlled and are sent unconsciously. Even the best-prepared individual has involuntary and subtle facial expressions for a short period of time (Ko 2018). Besides, consciously sent signals can also reveal underlying yet genuine emotions that are important for decision-making. Some of these may have been omitted in existing studies explaining decisions that are important for entrepreneurial performance.

The conceptual framework shows that decisions in entrepreneurial contexts that involve social interactions are not merely based on nonbehavioral cues. Among others, nonbehavioral cues include demographics and the traits and competencies of whom it concerns (e.g., Mitchelmore and Rowley 2010). In turn, demographics are for example the individual’s age, gender, and educational level. Entrepreneurial traits may include an individual’s need for achievement, proactive personality, and entrepreneurial self-efficacy. A good example of an entrepreneurial competency is someone’s network connectivity (Björk and Magnusson 2009). Even though most of the nonbehavioral cues can also be observed during the interaction—at least, to a certain extent (think of estimating someone else’s age)—usually they are already known beforehand. Prior information on nonbehavioral cues can be obtained by the decision maker, either through a shared history—as is the case with family, friends, (former) colleagues, and other acquaintances—or just by having received this information from the other individual. Think of an applicant sharing his demographics and signaling his traits and competencies by means of a cover letter and resume. Next to nonbehavioral cues, the decision maker also takes into account nonverbal and verbal behavior throughout the duration of the interaction.

First, nonverbal behavior may have a direct influence on the outcome of decisions by entrepreneurs or investors. Nonverbal behavior consists of all that is being done, next to the physical appearance.Footnote 1 This also includes nonverbal vocal behavior, i.e., expressive non-content characteristics of speech, such as pitch, tone, and tempo, and how one uses the space and environment (e.g., how one gets seated, if at all). Grahe and Bernieri (1999) emphasize the importance of nonverbal behavioral cues for decision-making. Especially people’s facial behavior plays a major role in shaping perceptions during social interactions. Not only age and gender but also individuals’ personality and attractiveness can be derived from a human’s face (e.g., Ambady and Rosenthal 1992). Accordingly, analysis of facial behavior can approve (or disprove) individuals’ nonbehavioral cues. There is growing evidence in the entrepreneurship and broader management literature for the importance of nonverbal behavior for decision-making. For example, self-promotion and other impression management tactics positively influence the evaluation by job interviewers (Higgins et al. 2003; Barrick et al. 2009). Hand gestures and body postures have been shown to influence the perception of the employer, since it might signal confidence and dominance (Sterrett 1978). Also, De Groot and Motowidlo (1999) have found that physical attractiveness, hand gestures, postures, and gaze behavior influence interviewers’ judgements.

Concerning investors’ decisions, research on entrepreneurial passion has suggested that the demonstrated passion of an entrepreneur in a pitch directly influences investor decision-making (Cardon et al. 2009; Chen et al. 2009; Pollack et al. 2012). Hence, nonverbal behavioral cues such as using animated facial expressions, a lot of gestures, and talking with varied tone and pitch could send signals of passion to investors in an entrepreneurial pitch (Nagy et al. 2012; Ciuchta et al. 2018). In addition, research and practice of entrepreneurship would suggest that coachability is very important for investors and, for example, an open hand gesture, mimicry, and attention through eye contact could influence investors’ judgment of entrepreneurs’ degree of coachability (Ciuchta et al. 2018).

Hence, we put forward the following proposition:

  • Proposition 1 (P1): Nonverbal behavioral cues have a direct relationship with decision-making in entrepreneurial contexts.

Second, verbal behavior may have a direct influence on the outcome of decisions by entrepreneurs or investors. Verbal behavior refers to all that is being said (or not, e.g., in the case of conscious non-disclosure of information), and, hence, is about the contents of speech. Appropriateness of the content has been shown to be one of the most important determinants of employment decisions within and outside the entrepreneurship context (Barrick et al. 2009; Hollandsworth et al. 1979). Candidates who respond concisely, cooperate fully in answering questions, and state personal opinions (when relevant) have higher chances to receive a job offer. These findings suggest that the verbal content of the interview may have a major influence on hiring decisions.

Preparedness of the entrepreneur is an important determinant of investors’ decisions that is based on the content of the pitch (Clark 2008; Pollack et al. 2012). Clarity of the pitch content and the pitch structure would reflect preparedness and has a direct positive influence on investors’ decisions. Entrepreneurs’ ability to answer the questions of the potential investors’ sufficiently, such as their calculative questions and questions on business assumptions, could majorly determine the outcome of the pitch (Pollack et al. 2012; Ciuchta et al. 2018).

Hence, we put forward the following proposition:

  • Proposition 2 (P2): Verbal behavioral cues have a direct relationship with decision-making in entrepreneurial contexts.

Moreover, we contend that both nonverbal and verbal behavioral cues moderate the effects of nonbehavioral cues on decisions in entrepreneurial contexts. Focusing on nonverbal behavioral cues first, the decision maker’s (prior) perception of the characteristics and qualities of the opposing individual may change for the better or the worse due to this individual’s nonverbal behavior during the interaction. This may eventually lead to another decision. For example, during a job interview, the entrepreneur’s perception of a job candidate’s qualifications may change depending on the candidate’s appearance, facial expression, body postures, and hand gestures. In particular, we suggest that an initial judgment of the qualifications of the candidate, for example based upon the CV, is moderated by their physical appearance (e.g., formal or casual dress), eye behavior (e.g., gaze, rolling eye), and pitch voice (e.g., excitement, seriousness). Then, the job interview could strengthen or weaken the decision maker’s assessment of the nonbehavioral cues.

Entrepreneurs’ nonverbal behavior also moderates the relationship between their nonbehavioral cues and investment decisions. Clarke et al. (2019), for example, have found that using hand gestures to depict and symbolize business ideas had strong positive effects on the propensity to invest. They found that the skilled use of gestures by entrepreneurs helps investors to imagine aspects of the products and businesses that were conveyed partially through the pitch content. Entrepreneurs’ nonverbal behavioral cues such as the pitch voice, their physical appearance, proper body postures, and gestures could convey the message of the entrepreneurs more clearly (or confuse the investor alternatively). If, for example, entrepreneurs are talking about a radically innovative idea while their voice is monotonous, their eyes are rolling, and they cross their arms, investors are less likely to perceive the idea as innovative (Clarke et al. 2019).

Hence, we put forward the following proposition:

  • Proposition 3 (P3): Nonverbal behavioral cues moderate the direct relationship between nonbehavioral cues and decision-making in entrepreneurial contexts.

Fourth, verbal behavior moderates the effects of nonbehavioral cues on decisions in entrepreneurial contexts. As a matter of fact, verbal behavior is closely linked to nonbehavioral cues, because the information that is exchanged during the human interaction very much influences the view of the decision maker. In the case of decision-making by entrepreneurs, given that entrepreneurs are generally more intuitive in making decisions than non-entrepreneurs (Blume and Covin 2011), they may extensively rely on their interactions with others. For example in the case of job interviews, proper answers, relevant personal opinions, signs of preparation in terms of interesting questions, and information about the background of the venture and the entrepreneur could extensively help interviewees to receive a job offer. Given the informality of the hiring process in a new venture, entrepreneurs are more likely to put heavier weight on exchanged information in an interview as compared to more established large companies.

Similarly, verbal behavior influences the perception of entrepreneurs’ qualifications among investors. If entrepreneurs use appropriate words and structure their pitches well, they send the signal that their idea is legit and they have the capability and knowledge stated in their business plan (Pollack et al. 2012). Chen et al. (2009), in an influential study, divided the passion construct into affective and cognitive dimensions. Cognitive passion, termed as “preparedness,” refers to the substance and understanding of the business idea that entrepreneurs communicate through the content of their pitch. They found that preparedness has a positive influence on investors’ decisions. Moreover, proper answers to investors’ questions after the pitch and in the follow-up meetings are crucial to strengthen (or weaken) investors’ opinions of the idea and the entrepreneur (Ciuchta et al. 2018). Hence, clarity and precision of the pitch and in later meetings would legitimize the claims that entrepreneurs make in their business plan or the pitch decks.

Hence, we put forward the following proposition:

  • Proposition 4 (P4): Verbal behavioral cues moderate the direct relationship between nonbehavioral cues and decision-making in entrepreneurial contexts.

Finally, we also put forward the following proposition (Clarke et al. 2019; Cornelissen et al. 2012):

  • Proposition 5 (P5): Nonverbal behavioral cues moderate the relationship between verbal behavioral cues and decisions in entrepreneurial contexts.

The two times two counterclockwise arrows together forming a circle indicate that the nonverbal and verbal behaviors by both parties involved in the interaction are mutually dependent. Obviously, verbal behavior by the one depends on that of the other, and vice versa, just like any normal conversation unfolds. This principle, however, is also applicable to most of the nonverbal behavioral cues. For example, there is some evidence for speakers’ tendency to imitate the vocal pitch of their conversation partner (Postma-Nilsenová and Postma 2013). It clearly does not hold regarding the natural characteristics of the individuals’ physical appearance, since it is predetermined and cannot be changed once the interaction started.

4 A detailed introduction to social signal processing

4.1 Social signal classification

With the emergence of powerful image and signal processing methods—that is, deep learning (LeCun et al. 2015)—the automatic detection of social signals is becoming feasible. Deep learning refers to artificial neural networks, multilayered algorithms that process inputs in multiple stages. Each stage (or layer) is associated with adaptive parameters that contribute to the transformation of the input (e.g., an image of a face) to a desired output (e.g., the gender of the face). Typically, deep learning algorithms are trained by huge collections of data, consisting of “labeled examples,” i.e., an example and its associated output. In our example, this would be an image of a face and a label specifying the gender of the face (“male” or “female”). During training, the parameters (the so-called connection weights) of deep learning algorithms are automatically adjusted according to a procedure that ensures that the adjustments are improving the quality of the transformation from input to desired output. For instance, in the case of a facial gender classification task, the individual connection weights of the network are adjusted to lower the occurrence of misclassified faces. A typical deep learning algorithm for image classification consists of millions of adjustable weights and requires about several million examples in order to be able to recognize previously unseen images with an accuracy of over 90% correct classification. Deep learning networks can also be successfully applied on one-dimensional inputs, e.g., acoustic signals such as speech. In addition, deep learning variants called recurrent neural networks can be successfully trained on classifying sequences of data, such as text or time series. The huge datasets required for training deep learning algorithms provide an obstacle for applications in domains in which data is sparse and data collection is costly. Fortunately, there is a growing number of pre-trained networks. The weights in such networks are optimized for a given task (e.g., facial gender recognition). Application to a closely related task (e.g., estimating the age of the depicted face) requires a relatively small amount of data for training.

What can deep learning detect in social signals? In what follows, following the taxonomy of Vinciarelli et al. (2009a), we review the application of SSP to three sources of social signals: (1) the body, (2) the face, and (3) the voice. Subsequently, we review approaches to measure the interplay of social signals in dyadic interactions.

4.1.1 Gestures and posture

Gestural and postural social signals have been argued to transmit a major part of the affective information. Aviezer et al. (2012) showed that the body is more important than the face in communicating social signals about the affective state (i.e., extreme positive and negative emotions). The automatic recognition of body posture and gestures of individuals can be performed quite reliably with deep learning algorithms. State-of-the-art algorithms can infer the body posture and gestures directly from videos of, for example, pitches to resource owners. Two recent examples are Open Pose (Cao et al. 2017) and Dense Pose.Footnote 2 Both Open Pose and Dense Pose can recognize individual body parts and postures. Dense Pose is more precise and robust than Open Pose though.

4.1.2 Face and eye behavior

The automatic classification of facial social signals is an active research area. In recent years, convolutional neural networks have been applied to facial appearance classification, facial expression recognition, and video-based eye gaze detection. Fisher et al. (2017) suggested monitoring of eye movements of financial resource providers when evaluating a new venture description or during the entrepreneurial pitch. Such tracking of eye behavior is an innovative way to identify and isolate what investors pay attention to. In this respect, collecting and analyzing video-based eye gaze detection may be informative.

In a recent survey of “deep” facial expression recognition, Li and Deng (2018) list databases of still images and video sequences and the deep learning algorithms trained on these databases. Current software is able to detect facial expressions and facial action units (i.e., the building blocks of facial expressions). Emotional expressions (e.g., entrepreneurial passion, see Cardon et al. 2009) are a challenge, mainly due to the lack of natural labeled data although that is about to change (Li and Deng 2018). Most datasets contain posed expressions or evoked expressions. Typically, in these datasets, the faces are presented in frontal view and in suitable lighting conditions. Automatically recognizing facial expressions in the wild is a more challenging task.

Facial expression analysis can be performed from still images, video sequences, or depth information (e.g., acquired with a Kinect device). The iPhone X projects over 30,000 infrared dots on the face to create a depth map. Currently, the depth map is being used for face identification but in the near future, it may serve facial expression recognition as well.

4.1.3 Vocal behavior

Vocal social signals constitute both verbal and nonverbal components. The automatic recognition of verbal speech has made large strides recently (due to deep learning; Hinton et al. 2012) but has not yet reached the level of human recognition. Vocal pitch supports verbal communication (e.g., statements versus questions) but also plays a fundamental role as a social signal. A well-known result is the effect of pitch on leadership perception. Humans with low-pitched voices are generally perceived as better leaders than those with high-pitched voices, although there is no evidence that humans with low-pitched voices are actually better leaders (Klofstad and Anderson 2018). We know that leadership skills are important for the survival and growth of new ventures. Hence, through the detection of pitch voices, it could be studied whether or not in reality entrepreneurs with low-pitched voices are more successful in attracting investment, and, later on, in growing their venture. Traditional pitch determination algorithms rely on heuristic algorithms that extract the vocal pitch from the speech signal. Recent work showed the superiority of deep learning approaches in pitch determination (Han and Wang 2014). Deep learning algorithms have also been applied to automatic affective speech classification, i.e., estimating the emotional state of persons from their vocal patterns, and achieving very good results (Trigeorgis et al. 2016).

At present, depending on the circumstances (e.g., not too much noise in the background, not too much overlap of different voices), deep learning algorithms can reliably estimate vocal pitch and prosodic patterns associated with emotions. The main limitation of current systems is the scarcity of labeled data on vocal expressions in the wild. Suitable large-scale databases are likely to become available in the coming years (for a recent overview of available databases for speech emotion recognition, see Swain et al. 2018).

4.2 Dyadic interactions

All three types of social signals are often studied for individuals in isolation. Per definition, social signals are associated with interacting humans. Hence, the study of dyadic interactions is of prime interest to understand the social dynamics of interlocutors. A range of studies has focused on measuring the dyadic interactions of social signals.

Studies of vocal interactions mainly focus on the vocal pitch (F0) of the interlocutors. In dyadic conversations, speakers tend to imitate the vocal pitch of their conversation partner (see e.g., Postma-Nilsenová and Postma 2013). Such speech imitation signals social affiliation and is attenuated by other social signals such as eye gaze (Postma-Nilsenová et al. 2013). Other forms of pitch adaptation signal differences in social status. Gregory and Webster (1996) studied the adaptation of vocal pitch of Larry King and his guests that appeared on the Larry King Live TV show. Mr. King adapted the pitch of his voice toward that of his guests with a higher social status than himself, whereas guest with a lower social status than Mr. King adapted their pitch toward that of Mr. King. A similar study can be conducted for entrepreneurs who pitch their business ideas to investors. Given that, in most circumstances, investors are perceived to have higher social status, it could be hypothesized that entrepreneurs would tend to imitate the vocal pitch of potential investors. Moreover, investors could also send social signals to high-potential entrepreneurs in order to attract their attention and distinguish themselves from other investors.

Similar interactions may exist in facial interactions. Facial expression mimicry is observed in many studies and can even be generated by virtual agents interacting with humans (Mattheij et al. 2015). Preliminary studies point at the complex dynamics of the coupled facial expression dynamics of conversation partners. With data-science methods, it may be possible to determine who is leading and who is following the facial expressions of the other (Postma and Nilsenová 2016).

Gestural mimicry can be studied directly from video using frame differencing or by relying on more sophisticated methods such as Open Pose and Dense Pose. Paxton and Dale (2013) described how frame differencing (i.e., subtracting subsequent frames of a video sequence to detect movement) can be used to detect bodily synchrony in conversations. Bodily synchrony is another sign of social affiliation or bonding. Related to that, mimicry and synchronization might be vital mechanisms through which entrepreneurial passion might become contagious. Hence, an interesting research question could be whether mimicry and synchronization as mechanisms facilitating contagiousness of passion positively affect investment decisions in entrepreneurial pitches. Video data that displays both entrepreneurs and investors at the same time is the input for digital processing and analysis of social signals to detect the level of produced and perceived passion in a pitch.

Finally, it is worth mentioning the domain of social sensing (e.g., Eagle and Pentland 2006; Atzmueller et al. 2014; Liu et al. 2015; Mashhadi et al. 2016). Social sensing refers to the collection of social data at a large scale (Atzmueller et al. 2014). Examples of sources of social data are online and social media and physical behavior. Analyzing dyadic interactions with respect to physical behavior in the context of face-to-face interactions has been performed, for example, at conferences using wearable sensors (Macek et al. 2012; Atzmueller 2014; Kibanov et al. 2014). Here, specific roles regarding face-to-face interactions and communication behavior as well as influence factors for long and short conversations have been analyzed (e.g., Macek et al. 2012), also regarding the dynamics and community building aspects (Kibanov et al. 2014). Furthermore, Kibanov et al. (2015) investigate the relationship between physical social interactions and online social interactions in order to check whether online social interactions can serve as a proxy for the physical ones. In the context of their experiments, there is evidence for behavioral online proxies especially for “strong” relationships. Using these, it may be possible to infer physical face-to-face interaction (options) given social media. Finally, behavioral interactions have also been analyzed for predicting face-to-face interactions at specific events, for example, in the context of teams and/or groups (e.g., Atzmueller 2014), with high accuracy. Predicting the outcome of specific situations has been investigated as well, for example relating to salary negotiations, dating, friendship, and business preferences. Here, accuracy is comparable with that of human experts in analogous situations (Pentland 2004).

5 Concluding remarks and discussion

The aim of this conceptual paper is to introduce contemporary research methods to entrepreneurship researchers that can advance our understanding of the decision-making processes of entrepreneurs as well as others yet directly affecting entrepreneurs, with a special focus on decision-making processes that are influenced by social interaction. In particular, we discuss and illustrate the relevance and potential value of the emerging domain of social signal processing. SSP refers to a number of modern technologies utilizing artificial intelligence that can automatically detect and analyze social signals being sent during human-to-human interactions. We claim that SSP techniques are more accurate and more efficient than conventional methods (i.e., manual coding and/or rating). Moreover, it may identify important characteristics that hitherto have been absent in research explaining how entrepreneurs and investors arrive at their decisions.

Several important aspects should be carefully considered when using SSP for research on decision-making. First, we acknowledge that algorithms may be biased as well (Hajian et al. 2016; Pournaras et al. 2017). This is mainly because algorithms collect and process data and use tags that are created in subjective ways by humans with certain perspectives on matters. Thus, the use of technologies will not lead, by any means, to pure facts and fully objective information. They rather help with providing further insights into decisions involving social interactions and, hence, have to be seen as a step into the right direction. Second, decision-making can be either individual or team-based. Dynamics of individual decision-making is substantially different from those of team-based decision-making, since the latter includes interpersonal tensions and other people’s problems (Wasserman 2013). This once more suggests the importance of a multilevel perspective in future studies on entrepreneurial decision-making (Shepherd 2011). Third, while we have mentioned heuristics, intuitive, and adaptive decisions a few times before, we do not necessarily consider them as errors (Tversky and Kahneman 1974) or drawbacks in decision-making processes. Some other studies in the fields of psychology and behavioral economics (e.g., Tetlock and Mellers 2002; Zhang and Cueto 2017) refer to this as “the great rationality debate”, and argue there may actually be some value added in entrepreneurs using heuristics and intuitive decisions, as is also discussed in the effectuation literature (Sarasvathy 2001). In addition, there is evidence showing that entrepreneurs using such decision-making principles succeeded in venture creation and growth (Koellinger et al. 2007; Navis and Ozbek 2016). This discussion resembles the System 1 and System 2 thinking metaphor (Kahneman 2003) and whether, when, and to what extent System 2 thinking should replace System 1 thinking. According to Milkman et al. (2009), “a key task for psychologists is to identify how and in what situations people should try to move from intuitively compelling System 1 thinking to more deliberative System 2 thinking, and to design situations that make System 1 thinking work in the decision maker’s favor” (p. 382). Finally, the use of SSP should not necessarily be confined to research on decision-making by entrepreneurs or investors. In fact, SSP can also be applied in other inter- or intrapersonal contexts, such as negotiations. It may provide individual feedback on, for example, emotions involved in the negotiations.

Future studies are encouraged to implement what has been proposed in this paper. In particular, these studies might want to look into interactions that entrepreneurs have with other relevant actors, such as customers, suppliers, and advisors. Entrepreneurs have their own unique ways to interact with their stakeholders that can be explored in future studies. Although the domain of SSP is rapidly emerging, entrepreneurship scholars should be aware about what exactly can be obtained at this stage of development of the SSP field. For example, it is still a challenge to derive emotions and personality traits from social signals. Unlike computerized text analysis with developed and tested tools such as Receptiviti (Obschonka et al. 2017), SSP, at this stage, has not yet developed a product ready to be used, neither by entrepreneurs nor by entrepreneurship scholars. Thus, in the meantime, entrepreneurship scholars most likely need to rely on data scientists to collect and analyze data on social signals.