4.1 Introduction to a Jokes-Based Survey Method

One of the main criticisms on consultants is their lack of expertise. Still, consultants are hired for their expertise, which appears paradoxical. When consultant expertise is lacking, the consequences for clients can be severe. In newspapers, bankruptcies have been related to the flawed, utopian or one-sided advice of consultants, absence of proper evidence, lacking guidelines for execution and recommendations that signal wishful thinking. The problem is acknowledged by consultants who list “being qualified” in their codes of conduct as first key value and important condition for their work. Being qualified is a virtue they identify with. However, as they often cannot sufficiently keep up with these standards, several jokes have picked up on the expertise issue.

When it comes to an assessment of consultant expertise, opinions may differ. For one group consultant expertise can be fine, while another group may be less convinced. This chapter starts here and seeks to find out how a survey method using cartoon-based rating questions on consultant expertise can provide insights in stakeholder opinions. Results are compared to those found with traditional statement-based rating questions on the same topic. Stakeholder opinions studied are those of clients, their employees, consultants themselves, academics studying consultants, and outsiders who are only informed by public sources. Each group has to rate traditional statements, and statements expressed in cartoons that criticize consultant expertise.

For philosophers and social scientists, it is relevant to study stakeholder opinions and their interests related to ethical issues, in this case lacking expertise. As the stakeholder configuration is quite complex for consultants with their own clients, client employees, the clients of clients etc., a method to study different perceptions of stakeholder groups regarding ethical issues helps to better understand for whom value is created or destroyed as suggested by Freeman (2010) and Krehmeyer and Freeman (2012). As with the jokes-based interview approach, the jokes-based survey method may help to get better access to the empirical material scholars in ethics need for their ethics reflections, and the empirical material will be of better quality, than when only relying on journalistic accounts, traditional methods, or thought experiments.

The next sections will first review management literature on the paradox related to consultant expertise, second the design of our jokes-based survey method will be presented including some key outcomes this method has generated, and the chapter concludes with discussing possibilities and limitations of the jokes-based survey method for research in business ethics, including possible reviewer concerns.

4.2 The Issue—Opinions on Consultants’ Lack of Expertise

Including cartoon-based survey questions to study opinions on professional ethics is a new mixed-method approach, here applied to the issue of consultant expertise and its flaws. Unlike law or medicine, consultancy as a profession does not have entry requirements or a generally accepted, common body of knowledge (Exton, 1982; Glückler & Armbrüster, 2003). This can create an awkward situation where few differences exist between the expertise of consultants and the knowledge of their clients. Exton (1982, p. 212) argues: “One might well expect that those whose profession it is to counsel others should possess some knowledge not common to those counselled”. Having sufficient expertise is therefore a key value in consultants’ codes of conduct but also a common critique.

Still, expertise is reported to be one of the key reasons for hiring consultants (Poulfelt & Payne, 1994; Wood, 2002). Consultants but also several researchers, therefore, emphasize their skilled and qualified nature (Greiner & Metzger, 1983; Kubr, 2002). Schein (1990) and Saxton (1995) describe consultants’ expert role as their most common role. Expert consultants intend to provide “helpful information relevant to the client’s problem” (Saxton, 1995, p. 59). This infers an image of consultants as aiding clients who lack the expertise to arrive at the desired solutions on their own (Sturdy et al., 2009, p. 247). Given this importance, lack of expertise may imply huge consequences for clients, like letting them decide for a wrong alternative, or reorganise at the cost of too many employees.

Therefore, lack of necessary expertise does figure prominently in the list of common vices in consulting (see Chap. 2, Table 2.2), and prevention is therefore highly prominent in consultants’ codes of conduct. The kind of problem addressed in the literature is that “consultancy means the absence of deeper knowledge, shallowness partly associated with fashions and fads as well as overpayment and an almost immoral attitude” (Alvesson & Johansson, 2002, p. 229). Related critiques point at the ambiguous and vague results consultants produce (Clark, 1995; Mitchell, 1994; Sturdy, 1997); the overly frequent use of standardized models including reusing old or repackaged ideas (Redekop & Heath, 2007; Sturdy, 2009; Whittle, 2006), and lack of industry knowledge (Sturdy, 2009; Sturdy et al., 2008). Critics also suggest consultants cover up through the use of jargon, storytelling, glossy brochures, slick PowerPoint presentations, and expensive suits (Alvesson & Johansson, 2002; Ashford, 1998; Clark & Salaman, 1998; Pinault, 2000; Whittle, 2006).

It is not unusual for professions to be both highly respected and widely criticized at the same time. Doctors, lawyers and politicians—like consultants—continue to be the butt of many critical jokes and the target of satire. Galanter (2005, p. 19), in his study about lawyer jokes, suggests that critical jokes “reveal that the qualities and actions for which the experts are despised are closely related to the things for which they are esteemed”.

The paradoxical findings pose the question how different stakeholders feel about consultant expertise: are clients more critical than consultants themselves? What is the view of academics that study consultants and how do employees experience the expertise of consultants? The different groups may have different opinions. Moreover, such ambiguities might be difficult to capture with direct, unambiguous survey questions only. For that reason, we decided to include cartoons on consultant expertise in our study design, to allow for more ambiguous understandings of consultant expertise, and to better introduce the topic with its implied paradoxes.

The next section discusses a mixed-method survey approach that links stakeholder opinions to two cartoons criticizing consultants’ lack of expertise, as applied in Bouwmeester and Stiekema (2015). By triggering respondents’ memories and experiences in a different way by asking questions about critical expertise cartoons, respondents might also be more willing to answer the set of traditional questions on consultant expertise. While the method is applied to stakeholder opinions on one particular professional vice, this only is an example case. The method can be used to study stakeholder opinions on any professional vice that has been joked about, and where there might be some controversy.

4.3 Application—Using Expertise Cartoons to Study Opinions

4.3.1 Jokes-Based Survey Method: Rating Cartoons Next to Traditional Statements

For this quantitative study, stakeholders could rate critical cartoons next to traditional statements about consultant expertise. That makes this a mixed-method study that combines rating questions related to cartoons with more traditional, literal rating questions. The method assumes respondents have the interpretive abilities to understand cartoons as Doherty (2011) argues. Likewise, Sturdy et al. (2008) and Bouwmeester (2013) consider humorous discourse an attractive source to study a profession because people only laugh about things they recognize, feel or care about (Cohen, 1999; Veatch, 1998). Still, humorous representation is somewhat distorted, exaggerated, sometimes ironical, and thus in need of adequate interpretation. To check if we can assume valid interpretation, traditional survey questions next to cartoon-based questions help to see if interpretations are consistent across methods, and within respondent groups.

Stakeholder groups are defined as “groups who can affect or are affected by a corporation” (Freeman, 2010, p. 1). In the Bouwmeester and Stiekema (2015) study, we report opinions of five groups. For groups like clients of consultants or client employees, this affect is obvious, given that consultants directly influence them with their assignments. Academics can be affected by consultants as well when being competitors or rivals in doing contract research, and academics are also an interested party when consultants are the subject of their studies. We consider as outsiders those who base their image of consultants only on what they read in newspapers or hear about them from friends. Their knowledge is not based on their own direct experience, and their stakes must be relatively low. In contrast, consultants themselves as the fifth group are strongly affected by the work of other consultants in that they strive to maintain, change, but sometimes also undermine their shared professional image and reputation, especially when expertise is lacking.

We have asked these five different stakeholder groups the same questions on consultants’ expert image: first questions related to two critical cartoons and then as follow-up traditional questions about consultants’ expertise. We selected two cartoons found on the Internet, the first criticizing expertise as being superficial, the second criticizing consultants’ empty rhetoric. The text of the two cartoons and a short description of the image can be found in Table 4.1.

Table 4.1 Cartoons used in survey on stakeholder views on consultant expertise
Full size table

Figure 4.1 shows the first cartoon to illustrate how the criticism is visualized that consultants lack expertise.

Fig. 4.1
A cartoon of a man at his desk communicating with someone on the telephone. Text reads, I know nothing about the subject, but I am happy to give you my expert opinion.

Cartoon 1 illustrating lack of expertise. © 2007 Tribune Content Agency, LLC All rights reserved

Full size image

The underlying assumption in the cartoon is that it does not matter what you know as a consultant, it is enough to merely claim you are an expert. This corresponds to a criticized absence of deeper knowledge (e.g. Alvesson & Johansson, 2002; Sturdy, 1997).

The second cartoon reflects the criticism that consultants’ rhetoric can be empty, by concealing lack of expertise behind buzzwords and storytelling. Dogbert’s engagement manager (Ratbert) applies such techniques. He is the consultant who is responsible for managing the relation with the client. The rhetoric may impress clients, and trick them into believing consultants on the team have sufficient expertise (e.g. Alvesson & Johansson, 2002; Clark & Salaman, 1998).

Stakeholder groups were initially approached by spreading a link to the survey via LinkedIn groups aimed at consultants and managers, and via the authors’ networks. Respondents were asked various questions in order to help identify their stakeholder group, before asking them to rate cartoons. This initial strategy sufficiently motivated most stakeholder groups to answer the questions, but academics where not reached well. Therefore, we added a second round of outreach with direct emails to academics known for their publications on consultants. This ensured sufficient group size. The final sample size consisted of 216 respondents (consultants 22%, clients 26%, client employees 19%, academics 17%, outsiders 16%). The survey was opened for response for one month.

Since this research compares multiple groups, we used one-way ANOVA tests combined with a post hoc Tukey–Kramer test to identify specific differences between groups. Subsequently, the results were controlled with a Games-Howell test due to the fact that sample sizes were unequal. When performing ANOVA tests, the data should ideally comply with two assumptions (Field, 2009). First is the assumption concerning the normality of the distribution. This research complies with this criterion due to the sample sizes used: “The central limit theorem shows that for sample sizes greater than 5 or 10 per group the means are approximately normally distributed regardless of the original distribution” (Norman, 2010, p. 628). Second, we assume homogeneity of variance, which was tested with a Levene test where significance levels higher than 0.05 indicate homogeneous data. While in some cases the data lacked such homogeneity, we still choose to conduct ANOVA tests because ANOVA remains a robust test for such unequal variances (Glass et al., 1972; Lix et al., 1996).

4.3.2 Survey Results: Cartoon-Based and Statement-Based Responses

To study stakeholders’ opinions on consultants’ expertise we asked respondents to assess the two cartoons discussed in the methods section. The respondents were first asked to consider how funny they thought the cartoons were by assigning them stars from 1 (not funny) to 5 (very funny).

Next, they were asked about the extent of truth portrayed by these cartoons, and finally whether or not they reflected their own perception of consultants. These responses were given on a five-point scale (strongly disagree, disagree, neither agree nor disagree, agree, strongly agree). Table 4.2 presents a summary of the results.

Table 4.2 Stakeholder views on consultant expertise: cartoon-based questions
Full size table

Table 4.2 illustrates that all stakeholder groups consider the cartoons funny. They give the two cartoons on average a rating of 3 stars or a bit more, and without significant differences between the groups. Groups somewhat agree with the statement that the cartoons have a kernel of truth (average value 3.2), and again without significant differences between groups.

When answering the question of whether respondents had similar perceptions of consultants’ image, as indicated by the two cartoons, the group of client employees agreed significantly more (3.28 first cartoon and 3.45 s cartoon) than clients (2.64 and 2.73) and consultants (2.38 and 2.55). Opinions are not very strong, but clients and consultants tended to disagree with the image expressed by the cartoons while client employees agreed, suggesting that the humour displayed by the cartoons contained an element of bitterness for client employees. Academics and outsiders held an intermediate position with a rather neutral opinion.

Table 4.3 summarizes the responses gathered with traditional survey questions about consultants’ expertise. The differences in reactions displayed between the five stakeholder groups are in line with responses to the cartoon questions summarized in Table 4.2.

Table 4.3 Stakeholder opinions on consultant expertise: traditional questions
Full size table

All stakeholder groups tend to agree that consultants are good at understanding a company’s specific problems (average 3.58) and are able to structure complex situations (average 3.56). It can, therefore, be concluded that criticisms about consultants’ expertise do not focus directly on their diagnostic and analytical abilities. All stakeholder groups grant them their skills in this respect.

Also the average perception about the relevance of consultant knowledge is moderately positive, however, there are significant differences displayed among groups. Client employees, academics and outsiders are rather neutral, seeing less relevance of consultant knowledge than consultants (average 3.60) and clients even more (average 3.80).

When asked about the superficiality of methods used by consultants, client employees (3.57) and academics (3.37) tend to agree. On the other hand, clients and consultants tend to disagree with values somewhat below 3. The same logic applied to questions regarding the claim that consultancy expertise is flawed: clients and consultants slightly disagree with this claim while client employees agree significantly more with this criticism (3.45).

Tables 4.2 and 4.3 both demonstrate that client employees are the most negative stakeholder group when it comes to consultants’ expertise, not seeing much relevance of consultants’ knowledge and criticizing the superficiality of their methods and their flawed expertise. Clients and consultants had the most positive opinion on the relevance of consultants’ knowledge, the value of their methods and the quality of their expertise, indicating an overall belief in consultants’ competences, which explains there is a market for consultant services. Outsiders and academics stay close to the average in their judgements, not having a strong opinion on consultant expertise.

The results in this mixed-method design are consistent, no matter if the questions are asked in a traditional way, or based on an expertise cartoon. All groups agree on the relatively strong analytical and diagnostic abilities of consultants. This is a positive aspect that gets lost in critical cartoons. Cartoons in general might be less suitable to point at strengths. A the positive side, they add to the validity of the survey, when used as control questions, and they motivate respondents to complete the surveys. These benefits can be profited from in business ethics research with its social desirability issues but also in research on other topics that can invite a critical laugh, like fashion, tradition, culture, etc.

4.3.3 Jokes-Based Survey Method and Contributions to Ethical Research Questions

We found that respondents were very able to interpret implied messages of the selected cartoons, as results were consistent with answers on traditional rating questions. Therefore, we can also consider the cartoon-based questions valid. The two cartoons have a kernel of truth according to all groups and both cartoons receive a rating of three stars for funniness. That means the normality condition for humour was met, and sufficient respondents recognized the issue.

When asking if the cartoons also adequately reflect the expert image of consultants, the groups start disagreeing. The same kind of disagreement appears when asking about expertise based on traditional statements regarding the relevance of consultant knowledge, their superficial methods and their flawed expertise. Client employees agree more with the virtue ethical critiques, clients and consultants do laugh but also tend to disagreement.

As consultants and clients are rather positive about consultants’ knowledge and expertise, they do not seem to observe negative ethical implications due to any lack of consultant expertise. Clients and consultants must feel that on average, all assignments considered, they can satisfy their own ethical standards and the standards as expressed in their codes of conduct regarding expertise and competence. No strong deontological critiques here. Client employees recognize consultants’ incompetence the most. For them, implications of consultants’ lack of expertise will be more negative on average.

The used cartoons do not indicate consequentialist critiques. Still, implied consequences of lack of expertise must fuel the more negative average opinion of client employees, who are always at the receiving end of what clients and consultants plan for them. Consequences of incompetence are mostly experienced by them, and may thus strongly influence their view of consultants’ professional image.

With the jokes-based survey method, we could contribute to debates in business ethics related to consultants’ expertise paradox. Usually, the client is pictured as victim of consultants in the literature, but it seems more an ally here. Client employees and maybe other stakeholders in the client system are the ones who suffer most from consultants’ incompetence, and they pay the price. These realized contributions show the method’s value as addition to the sparse research methods in business ethics. The method contributions are mainly descriptive, by quantifying and comparing differing opinions on consultants’ expert virtues. The jokes-based survey method can thus examine research questions like the extent to which different stakeholders’ perceptions of ethical transgressions are aligned on various dimensions.

4.4 Possibilities and Limitations of a Jokes-Based Survey Method

4.4.1 How to Make Use of Cartoon-Based Rating Questions in a Survey

We can conclude that cartoon-based questions in a survey related to unethical business behaviour give good results, just like traditional statement-based questions that respondents have to rate on a five-point Likert scale. Results are consistent across these different forms of questioning, and so for the different stakeholder groups. Cartoons visualize ethical transgression, and leave no doubt on the implied judgement. Still, the method requires proper cartoon selection, and careful selection of respondents, who should have the ability to interpret the cartoons well. Then, adding jokes-based question to a survey can have several advantages when studying business ethical transgressions.

How do I select cartoons? An important condition is that the researcher has sufficient context knowledge and is able to interpret the jokes well. Designing a survey depends very much on the researcher. The selected cartoons need to be sufficiently on topic and understandable by experts as well as respondents. To check this out, pretesting the survey is needed. As an alternative for cartoons short text jokes could be selected if they relate better to the topic under investigation, but cartoons usually have a stronger expression.

How do I select respondents? Respondent selection is as important, as they need to be able to interpret the jokes. Therefore, send the survey to places where you can assume respondents have the context knowledge that is needed to understand the cartoons. A lawyer better understands lawyer jokes, and a consultant better understands consultant jokes. The same applies to their stakeholders. Understanding business jokes is a bit more challenging than understanding literal questions on the same topic, due to aspects like fiction, irony, abstraction, exaggeration and distortion.

How can cartoons increase response rates? Response rates benefit from the humour touch in a survey when cartoons are included. It gives respondents a laugh, while filling in just another survey. If they know this up front as an introduction to the link, it can be used as motivation to open the form and fill it in. In addition, the completion rate may benefit as well, as seeing some cartoons during the survey works like a reward. It should not be overkill though, as the length of the survey can undermine these positive effects. Also a cartoon you do not understand, or that is off topic, will not work motivational. Still, because people are different in how they process information, adding cartoon-based questions is attractive. Some respondents will have a more visual mind, and may drop out earlier when they have to process text only.

How can cartoon-based questions in a survey increase validity? Cartoon-based questions can be used to address sensitive ethical topics, and topics that invite a laugh or show ambiguity. A good cartoon adds clarity to a rating question, by its visualizations. For instance, lack of expertise may sound vague. How little expertise would that be? When looking at the expertise cartoon (Fig. 4.1), it is stated that any expertise is missing related to the question the client asks. In consulting your expertise can always be better, you learn on the job, but such absence of expertise is not meant in the cartoon. It is a more extreme version, which is discussed as part of the ethics top ten under ‘underperformance’ as one of its consequences. By using the cartoon, it becomes very clear what kind of incompetence is meant. Complete lack of knowledge on the subject is no average situation, but still one that happens enough to be common. Indeed, to be perceived as funny, cartoons have to meet the normality condition of humour (Veatch, 1998). To further increase validity, we used a mixed-method design and asked literal question and the cartoon-based questions on the same topic. These two types of questions work as a control and enable you to triangulate findings.

How can cartoons reduce social desirability bias? Based on what happened during cartoon-based interviews, the use of humour and visuals can make the respondent more open, gives them better access to memories, and reduces some internal barriers about what can be said and what not. When this happens, also the answers on traditional questions may benefit from using a mixed method design. By using humour in the survey design, we can get closer to the respondents’ views on professional vices.

4.4.2 Method Limitations of Jokes-Based Rating Questions in a Survey

A first limitation of using cartoons in a jokes-based survey is that there is little possibility to go beyond the cartoon. Rating questions as common in quantitative designs have a very closed way of asking. A respondent can agree or disagree in degrees, but can add nothing different, or cannot indicate agreement with a slightly modified version of the question or in this case the joke. While open or semistructured interview questions give more room to respondents, rating questions cannot explore the associations triggered by cartoons. Rating questions also very much depend on the available jokes and their topics and statements. Open questions could be added to surveys as a remedy, but good opportunities to further explore what respondents have in mind are absent. The interpretation of responses to such open questions is also difficult, due to lack of context knowledge that is relevant to the respondent, but unknown to the researcher, and lack of opportunity to ask follow-up questions. It is also difficult to see if the respondent gets the joke, and if there is interpretation bias. That makes the mixed method design important, as it allows for control.

Second, there is humour bias. Cartoons focus on the negative, the imperfect, the abnormal etc. While traditional rating questions in a survey can relate to all kinds of statements ranging from positive to negative, cartoon statements tend to be critical. That makes the use of cartoon-based rating questions limited. When cartoons relate to ethics or business ethics, the focus is mainly on transgressions, as having virtues or good intentions is less funny (except maybe in more criminal, streetwise environments). In addition, jokes make fun of not meeting moral expectations that happen somewhat regularly. While they need to be emotionally absurd, transgressions should also be sufficiently common, or somewhat normal (cf. Veatch, 1998). In addition, transgressions cannot be too painful or too bad, as then the fun factor disappears. That limits the range of topics cartoons can address.

A third limitation linked to the topic of ethical transgressions pertains to how visible unethical behaviour is. Visibility is a condition for making cartoons. It is impossible to show something as funny that cannot be visualized, even if the transgression is sufficiently common. What seems to be visible enough though is character, and behavioural consequences, but intentions, identification with values or principles is much more difficult to visualize. This might explain why cartoons with ethical criticism focus predominantly on vices and problematic consequences. Transgressions without visible consequences may stay hidden, or require other forms of humorous expression.

4.4.3 Reviewer Perspectives on Jokes-Based Surveys and Possible Responses

The method using cartoon-based survey questions has been applied in an article submitted to the section of non-traditional research in the journal Management Decision. Bouwmeester and Stiekema (2015) got accepted after two rounds of reviews. It was the first submission, so no other journals have reviewed the method.

Method concerns raised by the reviewers were about better explaining how the methodology was a mixed-method approach, and how it draws on an interpretive epistemology for the questions based on cartoons (Czarniawska, 1997). For the traditional survey questions, interpretation was also needed, but not more than normal in a survey, and so it needed no further explanation.

One reviewer commented on the method with a statement like: “I actually liked the methods—a bit simple, but clever enough to give the reader a reason to continue” and after the first revision: “I think the humour method is intriguing”. Concerns related to our group comparisons, that gave very descriptive results initially. The main challenge was to tell a good theoretical story, and to find an angle to better frame the results. We responded by interpreting the results from a rhetorical point of view, by looking at audiences and their interests.

When applying the method to other vices than lack of expertise, or linked to other professions, the mixed-method survey design should be applicable as well. Critical cartoons that address the issue need to be available of course, and for a stakeholder comparison, some controversy is needed as well. Alternatively, opinions on values that change over time can be studied when criticisms in cartoons change, or when perceived funniness of cartoons decreases over time, due to what is or isn’t a mild violation of norms anymore.