1 The Great and Terrible Humbug

At the end of The Wonderful Wizard of Oz, Dorothy, and her friends arrive in Emerald City with concrete requests before the Wizard. Scarecrow wanted a brain, Tin Woodman wanted a heart, the Cowardly Lion wanted to be courageous while Dorothy just wanted to go back home. The Wizard, who appeared as a formidable creature with a thunderous voice and a giant head, was prepared to grant all their wishes in exchange for a magic broomstick. But when Toto (the dog) tipped over a screen, it was revealed that the terrifying Wizard was nothing but a puny old man creating audio-visual effects from behind a curtain. “I think you are a very bad man,” exclaimed Dorothy. In response, the Wizard told a heartwarming story about the circumstances that had forced him to engage in this game of deceit. As a balloonist, the Wizard had been swept away by a storm and landed in a faraway place where, by accident, people thought he was a great wizard. He was forced to keep the lie going for fear of his safety. When Dorothy insists that the Wizard is a bad man, the Wizard famously replies: “I am not a bad man. I’m just a very bad wizard.” She would not relent, referring to him as ‘The Great and Terrible Humbug’ until she finally returned home.

The unmasking of the Wizard leads to a discussion of whether he was justified in deceiving visitors as a means of self-defense. The mechanism of deceit itself does not receive further examination. In taking this course, Dorothy misses the chance to understand how she and her friends (Toto excluded) were so easily fooled. What was it that made the deceit so effective? What can we learn from the Wizard’s tactics? How are we to unmask similar deceivers in the future? These are essential questions, yet they are seamlessly ignored. One can hardly applaud the group’s critical skills since none contributed to revealing the truth. If Toto hadn’t pulled the curtain, Dorothy and her friends might have never discovered the truth about the Wizard’s bogus powers. This raises the following methodological question: Seeing that the group and the Wizard are in a state of disagreement regarding the group’s wishes and the conditions for granting these, can it be said that the Wizard’s deceit is a species of fallaciousness?

At first glance, it appears that the Wizard’s deceit – a game of lights, thunderous sounds, and green curtains – falls outside the scope of traditional fallacy theory. Scholars have primarily identified fallacies with rule violations that occur through verbal interaction in the context of disagreement (Tindale 2007; van Eemeren et al. 2009; Walton 1987; Woods and Walton 1982). The Wizard’s non-verbal actions can hardly appear on such a disciplinary radar. One might insist that the Wizard’s behavior is some form of implicit argumentation and that the Wizard is, in fact, defending some propositional content, e.g., “If I have a big head, you should be scared and accept my offer” (see van Eemeren and Grootendorst 1984: 49), but this seems forced. There are, in any case, methodological limits to such mechanical translations of non-verbal acts into speech (see Bateman 2018). The wizard patently did not deceive by saying something but by fabricating visual and auditory evidence. His head was not described as big and scary but was made “out of many thicknesses of paper, and with a carefully painted face.” He deceived, therefore, by fabricating evidence.

In this paper, we argue that the notion of fallaciousness can and should be extended to include such cases of convincing through deceit. To this end, we introduce, define, and illustrate the concepts of evidentiary convincing and evidentiary fallacies. We begin by introducing a distinction between two ways in which points of view can be changed: through argument (argumentative convincing) and through evidence (evidentiary convincing). We then introduce the notion of evidentiary fallacies as fallacies that occur in a process of evidentiary convincing when the procedure is mis-executed through fabrication or suppression of evidence. We then provide a first theoretical characterization based on a constructed example. With this as a theoretical basis, we discuss two real-life instances of evidentiary fallaciousness. Finally, we discuss the benefits and risks of such a theoretical extension of fallacy theory.

We bring to the fore that being convinced by understanding arguments has a counterpart in being convinced by examining evidence. If that holds, then fallaciousness in the process of discussing the acceptability of a claim could have a counterpart in the process of examining evidence. Reflection on evidence tampering is nowadays condemned to fragmentation into ever more specialized discussions in fields such as forensic sciences, policy, and, perhaps most famously, scientific methodology. Normatively inclined scholars are constantly concerned with the phenomenon of fraud and forgery within their own fields. But if a unitary theoretical framework is possible, then fallacy theory can provide a home where evidence tampering can be seen as the mis-execution of an interactional procedure aimed at convincing.

2 Two Pathways Toward Convincing

How can one’s interaction with others result in commitment change relative to a point of view? The question is central to argumentation theory, informal logic, discourse analysis, and many others fields (van Eemeren et al. 2014). It plays a role in descriptive studies of how individuals are convinced (de Oliveira Fernandes and Oswald 2022; Stiff and Mongeau 2016) as well as normative studies of ‘rational persuasion’ (Dutilh Novaes 2021), ‘critical thinking’ (Dwyer 2017), ‘good reasoning’ (Groarke and Tindale 2013) and ‘norms of (public) argumentation’ (Zenker et al. 2023). Yet the phenomenon of convincing is not just a topic of academic research. The question plays an important, if implicit, role in any endeavor aimed at gaining consent in political debates, business interactions, legal proceedings, and other contexts. Whether implicit or explicit, speakers must work with some notion of ‘what works’ in convincing an audience or interlocutor.

Despite the question’s centrality to various disciplines and activities, scholars and practitioners are still primarily focused on one of the two possible pathways toward convincing. The choice of path appears in different terminologies and through a broad diversity of theoretical lenses but could be described as follows: when an individual changes another individual’s point of view, this change is an instance of convincing if the pragmatic work is primarily carried out by arguments. Of course, scholars will admit that there are other ways in which the change of commitment can occur: one could be seduced, hypnotized, or medicated into new commitments, but these will not count as instances of convincing. If the argument is not the ‘active ingredient,’ then the process will not deserve the label of ‘convincing.’ In short, convincing is done by arguments. Arguments might indeed be implicit, ‘hidden’ as it were, among other forms of communicative behavior that make up the total of the interaction, but the perlocutionary act of convincing is fundamentally linked with the speech act of argumentation (Jacobs 1989; van Eemeren and Grootendorst 1992; Eemeren and Grootendorst 2004).

The scholarship that connects the phenomenon of convincing with the speech act of argumentation has successfully uncovered the many complexities and puzzles of our argumentative reality. We have an excellent theoretical grip on what it means to convince through speech acts during an argumentative interaction. However, from experience and specific institutionalized interactions, we know that arguments are not the only products that lead to convincing. Under particular conditions of agreement on procedural matters, evidence can have the same effect (Dove 2012; Popa 2022). We can be convinced by argument, but we can also be convinced by what we see or, more generally, by the evidence we examine.

To clarify this distinction between the two pathways, let us introduce the following imaginary (but realistic) scenario. Two friends, Mary and John, are talking to each other at a party. According to Mary, the party is a great success. According to John, the party is not a great success. Traditionally, convincing would occur in this situation when, e.g., Mary advances an argument that John accepts and, as a result, John gives up his standpoint that the party is not a great success. Mary could say: “Of course this party is a great success! The location and the music are awesome.” John might accept this argument and give up his claim that the party is unsuccessful.

But the two could also take a different path. Even if the exchange of arguments fails to produce a convincing argument, the two might realize that their disagreement hinges upon a mutually recognized criterion for evaluating the success of a party. We will assume that John and Mary discover, through an exchange that includes but is not restricted to the speech act of argumentation, that they agree on the following: If there are ten or more guests at the party, the party is a great success – otherwise, it is not. Having discovered this common ground, John and Mary can always continue their argumentative interaction, but examining the available evidence by doing a simple headcount seems more practical. We will assume that they agree on this. If there are ten or more guests at the party, Mary is right – the party is a success; if not, John is right – the party is not a success. Next, the two count the number of guests present at the party and they discover that indeed more than ten guests are present. At this point, John could perhaps come up with complaints about the counting procedure (Should the process occur now or later? Do the host’s family members count as guests? etc.) Such problems could be worked out, and the counting could be repeated under a new agreement. Eventually, if Mary and John operationalize their criterion satisfactorily, the evidence will compel one or the other to abandon their initial standpoint. If the examination runs smoothly and the two count more than ten (valid) guests present at the party, then John is convinced that he was wrong, and Mary is convinced she was right.

We will introduce the following terminology to analyze this case and its consequences further.

  1. 1.

    Evidence. We will refer to the objects or events based on which Mary and John agree to resolve their disagreement as “evidence.” In the given case, the guests present at the party – or more specifically, the total number of guests present at the party at a given moment – count for Mary and John as evidence. Avoiding essentialist definitions of evidence is easy because we follow the discussants’ agreements. Whatever they agree to consider as evidence will count as evidence. A normative approach might judge whether ‘number of guests’ is indeed the right type of evidence to determine a party’s success and even question whether the evidence can be gathered adequately according to the standards of the institution in which the discussion takes place. Still, such evaluative steps are irrelevant for the definition. Evidence is whatever object or event that the discussants agree to consider as pertinent to the resolution of the disagreement.

  2. 2.

    Evidentiary procedure. We will refer to collecting and examining evidence as an “evidentiary procedure.” In the case at hand, Mary and John engage in the evidentiary procedure of examining each room of the house (and perhaps other ancillary buildings) in order to determine how many guests are present at the party. Here, too, we employ the participants’ observable agreements as a basis. The experimentum crucis Mary and John agree to perform in order to resolve their disagreement by means of evidence will count as an evidentiary procedure.

  3. 3.

    Evidentiary convincing. We will refer to John’s change of commitment as a consequence of the evidentiary procedure as “evidentiary convincing.” Evidentiary convincing is contrasted with argumentative convincing in that the former occurs based on observing evidence while the latter occurs based on understanding arguments. John is evidentiary convinced in that he is convinced by examining the evidence (i.e., by seeing the tenth guest). John would have been argumentatively convinced if he had been convinced by one of Mary’s arguments.

In what follows, we wish to discuss the theoretical implications of these three definitions further.

First, let us point out that the distinction between argument and evidence bears some similarity with an Aristotelian distinction between artistic means of persuasion, those created by the speaker, and non-artistic means of persuasion, those that already exist, created by man or nature, and need to be discovered (Aristotle 2007, Rhetoric, 1355b35-40). Yet, Aristotle’s discussion of non-artistic means of persuasion is confined strictly to judicial rhetoric, elaborating on the employment of laws, witnesses, contracts, oaths, and evidence obtained from slaves under torture (1375a20-25). The artistic/non-artistic distinction is not what we aim for here. Evidence can occur both inside and outside the judicial context whenever two parties agree to a certain evidentiary procedure. Furthermore, evidence as defined here can be both artistic (in that it is created by the speaker for the purposes of the evidentiary procedure) and non-artistic (in that it pre-exists and awaits to be discovered by the participants in the evidentiary procedure). For example, if Mary and John disagree on John’s ability to do fifty push-ups, then John would be producing the evidence himself (‘artistically’) by doing the fifty push-ups in front of Mary. Integrative concepts of this kind, including both artistic and non-artistic means, can be observed nowadays in more recent works where it is generally assumed that convincing attempts can occur across different modalities (e.g., Bateman 2018; Tseronis 2018; Tindale 2021).Footnote 1 The distinction between argument and evidence seems both closer to the procedural distinctions already employed in various institutional contexts such as a court of law or science and avoids terminological shortfalls when applied to borderline cases. When both parties agree to examine an object or event as evidence, then regardless of whether that object or event is propositional or not, visual or not, whether it contains text or not and whether it is distinguishable by some other trait – the two are engaged in an evidentiary procedure. If the evidence is decisive, one of them will be evidentiary convinced.

Second, it is necessary to further explain the relationship between evidence and argument in the given example. We note that Mary and John come to establish the details of the evidentiary procedure using speech acts. Indeed, the failed attempt to convince through arguments brings the two before the realization that they agree on a usable criterion of party success. Furthermore, during the evidentiary procedure, the two might use speech acts to coordinate and exchange information (“Let’s go here!” “Is that a room?” etc.). Yet this does not interfere with the distinction we propose here. The use of verbal means to establish and execute the evidentiary procedure does not alter the fact that John is evidentiary convinced since his change of commitment is triggered by observing the tenth guest during the procedure, not by some act of implicit argumentation advanced by Mary. The reconstruction of evidentiary procedures as including implicit argumentation has been proposed in the past. For example, van Eemeren and Grootendorst reconstruct a similar evidentiary procedure as implicit argumentation:

A person showing another person a set of fingerprints in order to justify, to that person’s satisfaction, the expressed opinion that a certain person is guilty of some crime, will in principle be attempting to convince the other person with the (unexpressed) argumentation ‘These fingerprints were left by the culprit, they are X’s fingerprints, therefore X is the culprit’ (or a variant of this). Showing the fingerprints is not, in itself, sufficient to convince the other person. (van Eemeren and Grootendorst 1984: 49)

We could apply this to our case. We could reconstruct a speech act that corresponds with Mary’s behavior in our case, e.g., ‘Here is the tenth guest, the tenth guest demonstrates success as per the agreement, therefore this party is a success.’ With this act ‘interposed’ as it were between the evidence and John’s change of commitment, the convincing becomes argumentative again and the very distinction between evidence and argument becomes superfluous. Although surely plausible at first sight, this reconstruction cannot be undertaken consistently because it ignores that change of commitment occurs before and without the intervention of the other party. If a case depends on whether some fingerprints belong to a suspect and you observe the (visual) evidence of a DNA match, then it is that evidence before and independently of anyone’s pointing or arguing that convinces you. The same goes for our case. John is convinced by seeing the tenth guest, which he can do before and independently of Mary seeing the same guest or saying something about it. So, we agree with the authors when they insist that showing the fingerprints is not “in itself” sufficient to convince – but everything else that is needed can be agreed upon by the parties involved in establishing the rules for the evidentiary procedure. We conclude that even though the evidentiary procedure depends on communicative acts (argumentative or otherwise) for its establishment and execution, evidentiary convincing cannot be reduced to argumentative convincing.Footnote 2

Third, we could alternatively see the evidentiary procedure as an interruption from the actual resolution process. The evidentiary procedure would, in this case, be a mere detour meant to establish the acceptability of the premise that states the number of guests present at the party. When they are done with the head counting and the premise is established, the ‘actual’ discussion resumes and John is once more convinced argumentatively. The problem with this counter-analysis is that it assumes that the proposition tested through the evidentiary procedure, i.e., “There are ten or more guests at the party,” is nothing but a premise in the given discussion. Along these lines, the premise would appear at the bottom of the argumentation structure, with “The party is a great success” appearing at the top. Yet we would like to challenge this analysis. The proposition “There are ten or more guests at the party” is not a low-level premise but, in fact, the main, indeed the only, point of disagreement between the two. Even though Mary and John start their discussion by expressing opposing commitments relative to a different proposition, namely, “This party is a great success”, they quickly discover that their disagreement on this initial proposition is only a consequence of their underlying disagreement on the number of guests. The status of the proposition tested through evidence is relevant for the present purposes, for if the evidentiary procedure deals with a proposition that constitutes not some low-level premise but the main object of disagreement, the procedure cannot constitute an interruption of the resolution process but the resolution process itself.

The distinction between evidentiary convincing and argumentative convincing, as well as the corresponding distinction between argument and evidence, will allow for further considerations, but we have only sought to establish it prima facie with some degree of plausibility. Evidentiary procedures occur whenever parties agree to turn away from the argumentative procedure as a means of resolving their dispute and undertake to examine the evidence. This is perhaps just a roundabout way at arriving at the familiar idea that there are two ways to be convinced of the existence of the pudding: understanding and accepting arguments to that effect or eating the pudding. Both are valid, but they are not the same.

3 How to Commit an Evidentiary Fallacy

We assume it is meaningful to think normatively about the conditions under which individuals change their points of view. In the past, scholars in argumentation theory, informal logic, rhetoric, and discourse analysis have exclusively focused on argumentative convincing, building on a tradition that goes back to the Platonic concern for the questionable dialogical techniques employed by the sophists (Hamblin 1970; van Eemeren and Grootendorst 1992; van Eemeren et al. 2014; Walton 1987). We are addressing two questions regarding a potential extension of the notion of fallacies in the realm of mis-executed evidentiary procedures. First, there is a conceptual question: Does it make sense to speak of fallacies as being committed when evidentiary procedures are mis-executed? Second, there is a practical question: Is it helpful to see such mis-executions as a type – or ‘mode’ – of fallaciousness? For both these questions, we need to single out a definition of ‘fallacy’ as a baseline in relation to which the proposed extension can be discussed. The pragma-dialectical notion of ‘fallacy’ as the violation of the dialectical rules for resolving a difference of opinion can, for the present purposes, provide such a baseline (van Eemeren and Grootendorst 2004). Other definitions with an equivalent focus on argumentative convincing could have worked equally well since we are only interested in discussing a conceptual expansion from the argumentative to the evidentiary.

The phenomenon of tampering with evidence by fabrication or suppression is already under the attention of scholars and practitioners in many institutional contexts. In the legal domain, rulings are often based on, or at least influenced by, evidence brought to court (Green 2006; Miller 2021; Reurink 2018). In the political domain, proponents of evidence-based policy are well aware of the dangers of fabricating or suppressing evidence (Marston and Watts 2003; Oliver et al. 2014). In the academic domain, various types of misconduct pertaining to data have been the bread and butter of websites such as www.retractionwatch.com as well as studies of scientific misconduct (George 2016; LaFollette 1992; Schachman 1993). Even with the many valuable contributions in such fields, scholarship remains bound to its specific context without seeking to provide a more fundamental understanding of evidence tampering as part of an interactional process aimed at convincing. As a result, inter- and trans-disciplinary communication on this topic is relatively scarce. It is worthwhile to go back to the drawing table and ask how evidentiary fallacies function and whether already-existing fallacy scholarship can provide any means for understanding and detecting these phenomena.

Let us modify the imaginary scenario developed in the previous section to include some form of tampering with evidence. In this new version, John skillfully and intentionally misdirects Mary’s attention so that she fails to observe a door in the corner of the house that leads to an additional room. In that room, we assume, there are four more guests. Mary is thus “kept in the dark” concerning vital evidence (Carson 2010: 54). The headcount carried out in this fashion gives the result of eight present guests, which means, following their agreed-upon criterion, that John is correct in believing the party is not a success. Had the additional guests been considered, Mary’s standpoint would have prevailed.

Although John intentionally mis-executes the evidentiary procedure in this new version, his intentions might not be all that relevant for the evaluation. The question of fallaciousness does not seem to hinge on any party’s intention.Footnote 3 A corollary of this is that both Mary and John can be held responsible for failing to safeguard the accuracy of the procedure, at least as far as preventable mis-executions are concerned. We should limit ourselves to ‘preventable’ mis-executions because situations could arise where the procedural derailment is far beyond what either Mary or John can reasonably be expected to prevent (e.g., the mentioned door might be hidden entirely). In such exceptional situations, fallaciousness could still be identified even though the two would be absolved of the associated responsibility. Various borderline cases can be created by further modifying the scenario. For reasons of space, we will restrict ourselves to one. Imagine that there are no guests in the unexamined room. Can we still speak of a mis-executed evidentiary procedure? In one sense, the procedure was executed correctly in that they achieved their goal of counting all the guests at the party (even though they were not justified in claiming the achievement). In another sense, the procedure was not correctly executed because they did not follow the agreed-upon method of examining the entire house. Put differently, if the two are operating with a faulty notion of ‘the house’ – that is, a notion that we as evaluators can recognize to be defective – then the question of fallaciousness would have to be answered based on the details of their agreed-upon evidentiary procedure. In each case, however, we suspect the evaluator must operate with some general notion of reasonable oversight, given that procedures can go astray in many ways. Footnote 4

Based on the considerations above, we would like to formulate the following definition of an evidentiary fallacy:

When two or more parties agree on an evidentiary procedure for resolving their disagreement, an evidentiary fallacy is committed when evidence is either intentionally or unintentionally fabricated (i.e., non-evidence is taken into consideration) or suppressed (i.e., evidence is not taken into consideration) to the effect that one or more of the agreed-upon rules of the evidentiary procedure are violated.

Based on this definition we can see that there are only two categories of evidentiary fallacies – described, at least, at the highest level of generality. One can either fabricate evidence by adding or allowing evidence that is not in accordance with the agreed-upon procedure and the standpoint being tested or suppress evidence by blocking or disallowing evidence that would be in accordance with the agreed-upon procedure and the standpoint being tested. It is possible however that future research on this topic could undertake a more fine-grained distinction between these two categories, as for example distinguishing between fabricating completely new evidence and fabricating modifications of the existing evidence. We will want to suggest later that these major categories can be divided further, but for the illustration purposes of the next section this basic division will suffice.

A historical detour is required before proceeding further. The concept of ‘evidence’ has certainly not been absent from the vocabulary of scholars interested in the normativity of argumentative discourse, particularly those influenced by the judicial tradition. But we believe that the notion closest to ours is J. S. Mill’s notion of fallacy from evidence distinctly conceived, with its subcategories of fallacy of mal-observation and fallacy of non-observation (Mill 1974, System of Logic, V, ii, § 2, and iv, § 1–5). The phrase “incorrect performance of the proving process” is close, in letter but also in spirit, to what we describe as the mis-execution of the evidentiary procedure. Nonetheless, we are with Mill still within the realm of “speaking of instances” (V iv § 3, italics added) as opposed to the cases described here where the parties decide to resolve their disagreement by examining those instances directly. Mill’s use is however not consistent for when he later speaks of “non-observation of circumstances” (V iv § 4–6), the cases presented are both those of direct observation of evidence and of citing (“speaking of”) evidence. The ambiguity is carried over to the notion of “facts” (V, ii, § 2) which cover both the actual events to be observed and the statements describing events that are not directly observed. Such ambiguity is perhaps a continuation of how terms such as ‘evidence’, ‘fact’ or ‘data’ are used in everyday speech, but it conceals the different skills that are needed to identify and avoid evidentiary fallacies, as opposed to argumentative ones. Moreover, tributary to an argumentative understanding of fallacies, his focus, when discussing facts that are evidentiary for other facts, the cause of error relies for Mill in what could be labeled the probative force of the fact, the inference from facts, rather than in how the facts are established – which would be the focus of the evidentiary procedure. This is why we should insist on maintaining a distinction between argumentative and evidentiary convincing. In an argumentative procedure, avoiding fallacies is a discursive prowess consisting of asking the right critical questions at the right time, and eventually answering them, rejecting unwarranted premises, advancing counterarguments, recognizing subtle changes in the meaning of terms etc. In an evidentiary procedure, recognizing and avoiding fallacies is a practical prowess closer to Aristotle’s technê (τέχνη) or Polanyi’s tacit knowledge (1966) that arises from an understanding of how the examination instruments work and how they could work differently, predicting and avoiding blind spots, making efficient use of the space designated for examination etc. This is precisely the craft-knowledge that is used ‘in reverse’, as it were, by participants who seek to suppress or fabricate evidence in an examination process. And it is, we might add, the knowledge that Dorothy and her friends failed to acquire by only discussing the moral justification of the Wizard’s deceit and not its inner workings.

4 Two Sample Evaluations

4.1 The Scribble Pen

Imagine a pen that could capture the colors that surround you and allow you to draw in those colors. This device would have three components: (i) a light sensor capable of capturing colors and turning them into a digital signal, (ii) an ink cartridge capable of interpreting those signals and mix basic colors (blue, red, green) to reproduce the captured color, (iii) a drawing tip with which to use the resulting mixture. If you think this is part of a science-fiction movie, you are wrong. That is in any case the standpoint of the company that designed the Scribble Pen, the pen that “puts all the colors of the world in your hands.”Footnote 5 How does it work? “The built-in color sensor lets you easily capture any color – simply point it at an object or surface and press the button”. Then, the Scribble’s ink cartridge “connects to a mixer and dispenser that exactly recreates the color you have scanned”. The Scribble pen was advertised as “the last pen you’ll ever need to buy”. In support of their claims, the Scribble company created various promotional videos in which we are shown how the pen works. In a video that gained a lot of attention on the Internet, a voiceover tells us that Scribble allows you to “borrow the colors around you” and “use the world as your inspiration.” We are shown several frames in which a visibly frustrated artist seems to lack inspiration but then is suddenly relieved when she captures some of the colors that surround her with her Scribble Pen. She places the pen against a red apple; a light band starts flashing on the pen (in what appears to be the same shade of red as the apple); then she uses the color that was just ‘captured’. The three relevant frames are reproduced in Fig. 1 below. The same process is then repeated, first with the green of a leaf and then the bright blue of a coffee mug. “It works,” the narrator tells us, “by using a special color sensor to read the colors you scan.”

Fig. 1
figure 1

How the scribble works

Full size image

The only problem with the Scribble Pen is that it does not in fact exist and, as some have argued, it could not exist either, at least not with the technologies we know today.Footnote 6 The Scribble Pen was an elaborate hoax that managed to raise hundreds of thousands of dollars on crowdfunding campaigns such as Kickstarter and Tilt, as well as on their own website, by claiming to have solved immensely complicated engineering problems. The frames of the aforementioned video were cut so as to suggest the ‘capturing’ of colors when in fact the pen was a regular pen pre-filled with ink of the right shade. This is a classic case of a ‘mechanical Turk’ (named after the fraudulent 18th -century chess machine that appeared to be able to play chess when in fact it was just a human operating all the levers from inside). Just as the mechanical Turk is an instance of fabricated evidence by producing the visual experience of “automated” chess moves (that were in fact played by a regular human), so is the Scribble pen fabricating visual evidence of automated color capturing that is in fact done by humans.

What is relevant here for the evidence-gathering process are the three shades of red: (1) the red of the apple, (2) the red of the light band and (3) the red of the drawing made by the pen. The visual evidence suggests, and the voiceover reinforces this suggestion, that these three are correlated automatically by the intelligent pen. But the three colors were in fact selected by flesh-and-blood video directors beforehand so that they match. Presumably, the light band is only a simple led device that lights up when prompted. Similarly, the pen that was used to draw in the next frame is only a simple ballpen of a color matching the apple and the light band. The frames suggest that the apple, or more precisely the color of the apple, sets in motion a complicated process that is both automatic (“sensor”, “mixer and dispenser” etc.) and precise (“exactly recreate the color”).

The evidentiary fallacy here consists in fabricating visual evidence (the video footage) to support the claim that the Scribble Pen can capture colors from the environment and render them on paper. Whether the viewer is convinced or not, both the choice of visual evidence and the way the frames were cut (see Fig. 1) point towards a feature that was neither real at that time nor realized in the meantime. We should add that the editors had plenty of conventional means for making it explicit that the video is simply an imaginary rendition of how the pen could work in the future. Adding the text “This is a simulated illustration” on the images or a disclaimer at the end are common ways of avoiding misinterpretation. But the fabrication process of course needs to hide itself to work – a magician revealing the sleight of hand can no longer claim that a selected card actually “jumped” through the deck or “changed colors”. Finally, note that the fabricated evidence in the footage is tightly connected with the verbal description of the footage and that they reinforce each other. The images show (“proof of”) what the voiceover describes. But to keep terminology tidy we speak of fabrication only when describing the footage, not the statements. The voiceover statements might be described as false, inexact, vague, erroneous, misleading etc., but it is only the evidence that will be described as having been fabricated. As events unfolded, the Scribble Pen campaign was eventually removed from the crowdfunding website precisely due to the flimsiness of the visual evidence.Footnote 7 The Scribble Pen owners were summoned by the crowdfunding website to produce “a new and more illustrative video within 24 hours”, which they eventually failed to do leading to the cancellation of the campaign. Before the cancellation, however, the owners raised more than €320.000 in donations – all based on a three-minute video containing fabricated evidence.

4.2 The Pasteur-Pouchet Debate

Can life arise from dead matter? According to the doctrine of spontaneous generation that was proposed by various scientists in the 19th century, life can and does form spontaneously, not only in the cosmological sense of life on Earth being formed from a primeval soup, but in the more common situation of life arising out of ‘dead matter’ in the laboratory. When fruits and vegetables spoil, is it because of microorganisms that arise from the fruit itself or because of air contamination? We nowadays know the answer to this question – life does not arise spontaneously from dead matter. But as late as the 20th century the debate was still open whether this was the right answer (Collins and Pinch 1998: 79–87).

The problem of spontaneous generation was usually investigated based on the following type of experiments. A watery mixture such as an infusion of hay is boiled to destroy all existing life after which the container is emptied of air (with the help of steam) and subsequently sealed. In these conditions, life was not generated. But when air was let in, life could appear once more usually in the form of mold or bacteria. However, conceptual ambiguities regarding notions such as “sterile solution” or “pure air” gave rise to intense debates. If mold and bacteria were capable of growing in pure air, then this would prove the spontaneous generation thesis, but it was notoriously difficult to obtain “pure air” and even more so to prove beyond doubt that the air in an environment was pure (Collins and Pinch 1998: 83). In this context, two French scientists were engaged in a debate that lasted several years. One was Louis Pasteur who believed that spontaneous generation was impossible and thus insisted that the purified air inside the experimental vessels must be contaminated if life continues to appear. Pasteur did not have an explanation for how the air was contaminated, yet he insisted on some unknown flaw. The other was the established and much older Felix Pouchet who believed in spontaneous generation and insisted, not unreasonably, that the air could not have been contaminated. The debate unfolded initially through a series of letters, but soon the two men would travel to high altitudes to replicate each other’s experiments – all with inconclusive results. The Academie des Sciences was soon involved but the committees set up to settle the matter were all openly opposed to Pouchet (as was the academic community at this point) and their pronouncements had little impact. When the committee asked Pouchet to replicate the experiments, he declined.

In hindsight, all parties were insisting on the truth of their position for all the wrong reasons. As Collins and Pinch note, “experiments of the type that formed the basis of this debate can be confounded in many ways” (1998: 90), whether it is through the persistence of heat-resistant spores, the acidity of the solution as well as the much-debated air contaminations. In addition, neither of the parties made much of the fact that Pouchet and Pasteur were using different infusions (hay and yeast respectively), which we know nowadays to have different properties. Hay allows some heat-resistant spores that yeast usually does not. If Pasteur ever did experiment with hay, those experiments must have failed since he was not aware that a full sterilization requires heating under pressure to a temperature of 160 oC. Collins and Pinch conclude that under these conditions some evidence must have been suppressed by Pasteur who focused obsessively on the purity of the air allowed back in the flask:

As we now know, there were many ways in which Pasteur’s experiments could, and should, have gone wrong. Our best guess must be that they did, but Pasteur knew what he ought to count as a result and what he ought to count as a mistake (Collins and Pinch 1998: 90, italics added).

There are two instances of working with evidence that must be discussed. First, by allowing heat-resistant spores to persist in the infusion, Pouchet effectively fabricated evidence that supported spontaneous generation. However, the two scientists agreed that infusions are sterilized at around 100 oC and had no reason to believe otherwise. From our perspective, Pouchet was fabricating evidence of spontaneous generation, but one can hardly maintain that the agreement between the two should have been different. Although fallaciousness through fabrication did occur, since the agreed-upon ‘pure air’ was not pure, neither of the parties could be accused of it. On the other hand, by not reporting the results from using the hay infusion, Pasteur suppressed evidence beyond the agreed-upon evidentiary procedure. The example illustrates how multiple forms of mis-execution within the same evidentiary procedure might require separate consideration. It also illustrates that our knowledge of the larger context within which the evidentiary procedure was executed can play a decisive role in deciding whether fallacies were committed and, if so, which parties were responsible for the mis-execution. The consequences of this evidentiary fallacy are more difficult to estimate than in the previous case where the crowdfunding campaign gives some approximation of the ‘damage.’ In practice, the damage was more symbolic than material. Pasteur’s reputation, and perhaps that of scientists more generally, was clearly affected by this episode. If there is an objective scientific method out there, preeminent scientists of the day do not seem to follow it religiously – either that or the method does not protect against fabricated evidence. We agree thus with the conclusion that Collins and Pinch draw in the postscript to their discussion of the case:

Pasteur was a great scientist but what he did bore little resemblance to the ideal set out in modern texts of scientific method. It is hard to see how he would have brought about the changes in our ideas of the nature of germs if he had been constrained by the sterile model of behaviour which counts, for many, as the model of scientific method (Collins and Pinch 1998: 90).

5 Extending Fallacy Theory

In the previous two sections we have provided an affirmative answer to two questions formulated in the beginning of Sect. 3. We claimed that it is both conceptually possible and practically beneficial to extend the notion of ‘fallacy’ to cover mis-executions of evidentiary procedures. In this concluding section, we wish to reflect upon the consequences of such a theoretical extension.

One risk we wish to discuss at this juncture is that of opening a Pandora’s Box of conceptual problems and irresolvable complications. If we expand the notion of fallacy to include evidentiary ones, we might be drawn into a disciplinary renewal whose consequences are not entirely positive. One such consequence is that of overextension. For example, if an advertisement could be construed as part of an evidence-gathering procedure (regarding the advertised product), then one might end up seeing cases of fabrication and suppression in just about every economic transaction. Commercials showing clothes of impeccable white, silky-smooth hair, the superstar thoroughly satisfied after using the product, the joyful group of friends – these are all ‘fabricated’ (in that they are not instances of the actual use of the product) and they are meant to hide the plain truth that the experience of using the product is going to differ. Having opened Pandora’s Box, aren’t we forced to accept all this as evidentiary fallaciousness? Perhaps there is a sense in which the footage of a joyful group of friends is a misleading and an unreasonable suggestion that the soft drink they are holding is somehow associated with their joy. But identify this as an instance of fallaciousness would require further investigation in the context. More specifically, it would require us to establish that the agreed-upon conventions relating to commercials are strict enough to disallow such fabrication and the authors of the commercial are doing it anyway, disregarding the agreed-upon procedure. One must expect companies to associate their products with individuals that are happy, radiant, lively, confident etc. without assuming that they will experience those emotions while using the product. Is the looseness of this convention effectively exploited by the authors of the commercial? Perhaps. But sly moves are not wrong moves if they are allowed by the participants’ agreement. This overextension and other similar ones are made possible, at least in part, by a persistent ambiguity in the essential concept of evidence. Surely there are clear-cut cases of objects or events being presented as evidence, e.g., evidence brought before a jury in a court of law. But we have to admit that the great number of borderline cases will easily overshadow the few indisputable ones. Is the impeccable white of the clothes shown in a commercial really shown as evidence for the quality of the detergent? Is the endorsement of the famous actor shown as evidence for the quality of the coffee? The most practical solution to this problem is to undertake the same strategy we observe in the traditional study of argumentative fallacies: start with clear-cut cases first and work our ways towards increasingly more subtle questions and conceptualizations. In this paper we have sought to provide such a first step in the direction of an extension.

A second risk pertains to the underlying relationship between evidence and arguments. We hope to have provided a first determination of this relationship, but many interesting questions remain. For example, the traditional category of appealing to authority (or ‘expert opinion’) will require, in our view, some reconsideration (see recent overviews in Lewinski, 2022; Wagemans, 2011). The difficulty seems to be that an appeal to authority can be simultaneously considered a type of argumentation, one in which the opinion of the authority is brought to bear on the acceptability of the conclusion, and as a type of evidence, one where the speech event of the authority asserting the expressed opinion is observed by the parties involved. If Mary argues that p is acceptable because authority A asserted p, then she is obviously involved in an argumentative interaction; but if Mary and John decide that the acceptability of p should be based on A’s assertion of p and ask A for his opinion, then this is equally obviously an evidentiary procedure. There are of course pragmatic differences between the two episodes. In the first case, it is Mary’s speech act of argumentation (regarding A) that serves the function of convincing John; in the second case, it is A’s speech act – more precisely observing the performance of A’s speech act – that serves the function of convincing both. But from a dialectical point of view A’s authority is, in both cases, brought to bear on the acceptability of p. Is the hair-splitting worthwhile then? We want to leave this question open for we fear for such borderline cases it very much depends on one’s analytical aims and theoretical perspective. But regarding the latter, we should like to add that if one labors under the stipulation that only discourse can convince, then the distinction is of crucial importance. In some approaches, such as pragma-dialectics, only the first episode described above would could count as resolving the difference of opinion:

The resolution of a difference of opinion is not the same as the settlement of a dispute. A dispute is settled when, by mutual consent, the difference of opinion has in one way or another been ended – for example, by taking a vote or by the intervention of an outside party who acts as a judge or arbitrator. Of course, reaching a settlement does not mean that the difference of opinion has really been resolved. A difference of opinion is only resolved if a joint conclusion is reached on the acceptability of the standpoints at issue on the basis of a regulated and unimpaired exchange of arguments and criticism” (van Eemeren and Grootendorst 2004: 58).

In pragma-dialectics, it would be crucial to determine whether A’s authority is cited in an argument or observed in an evidentiary procedure because this would further determine whether the change of commitment resulting from it is an instance of resolving or settling the difference of opinion. Yet, to uphold this distinction, pragma-dialecticians would have to insist that citing A’s opinion in an argument from authority can lead to convincing while hearing A’s opinion during an evidentiary procedure cannot. This consequence seems rather forced. In any case, the subtle differences between argumentative convincing and evidentiary convincing have not been fully worked out in the present paper and the topic deserves further consideration.

The risks of overextension must, however, be judged in relationship with potential benefits. The normative study of argumentative convincing and the resulting interest in traditional fallacies can be seen as a specialization within a broader interest in how the minds are changed by interacting with others in different institutional contexts. The normative study of evidentiary convincing and evidentiary fallacies can be a separate specialization and fruitful connections can be established between the two. Traditionally, some fallacies have already been understood in a theoretical relation with the notion of evidence, as is most prominently the case with ad ignorantiam - often criticized under the moniker “absence of evidence is not evidence of absence”. Evidentiary fallacies could be related to the Starting Point Rule for critical discussions in the pragma-dialectical terminology – as forms of meddling with the starting points by falsely presenting something as an accepted starting point (van Eemeren and Grootendorst 1992) – but other rules within the mentioned approach might capture various instances of evidentiary misdemeanor. For example, employing a premise that has not been accepted by the opponent, as is the case with fallacies such as petitio principii and the slippery slope, can be seen as the argumentative equivalent of fabricating evidence (although see our discussion of the Scribble Pen on the terminological issues). Similarly, using ambiguity, threats, questions or other means to suppress the difference of acceptability between two apparently equivalent uses of a term, i.e., fallacies such as equivocation, ad baculum, shifting the burden of proof, can also be seen as the equivalent in the argumentative realm of suppressed evidence. These ideas need to be further elaborated, of course, but they show that the two territories (argumentative convincing and evidentiary convincing) can engage in a fruitful cross-fertilization.

6 Conclusion

In this paper we have argued that it is meaningful and beneficial to speak of fallacies being committed when evidence is fabricated or suppressed during an evidentiary procedure. We have illustrated this claim and we have discussed some of its possible implications. By way of concluding, let us go back to our example of the Wizard. Remember that it was Toto who unmasked the Wizard. Dorothy and her friends hardly contributed to the unmasking and, as we noted in the introduction, they did not seem to be phased by their complete lack of ability in this sense. Dorothy proceeded directly to a moral evaluation of the Wizard’s acts. We suspect that a significant training in argumentative fallaciousness would not have helped in that case because, as explained, the convincing effect was achieved through strategic use of lightworks and sound engineering. A more extensive study of evidentiary fallaciousness can give Dorothy and her friends not only a healthy suspicion of what is going on (the kind that Toto was gifted with by nature) but also the tools to seek the fabricated evidence. Was it not strange that no one has ever seen the Wizard in person? What is the actual evidence of the Wizard’s power? Should we not pay attention to what happens behind the curtain? In order to approach these issues Dorothy and her friends need a more comprehensive array of critical skills than the ones focused on argumentative convincing that are currently on offer in argumentation studies, informal logic and other related disciplines.