1 Introduction

Explaining classifier decisions has gained much attention in current research. If explanations are intended for the end-user, their main function is to make the human comprehend how the system reached a decision (Miller 2019). In the last years, a variety of approaches to explainability has been proposed (Adadi and Berrada 2018; Molnar 2019): Explanations can be local—focusing on the current class decision—or global—covering the learned model (Ribeiro et al. 2016; Adadi and Berrada 2018). A major branch of research addresses explanations by visualizations for end-to-end image classification (Samek et al. 2017; Ribeiro et al. 2016). Alternatively, explanations can be in form of symbolic rules (Lakkaraju et al. 2016; Muggleton et al. 2018) or in natural language (Stickel 1991; Ehsan et al. 2018; Siebers and Schmid 2019). A further approach to explanations is to offer prototypical examples to illustrate a model (Bien and Tibshirani 2011; Gurumoorthy et al. 2019). Finally, counterexamples can be used as counterfactuals or contrastive explanations. Counterfactuals typically are minimal changes in feature values which would have resulted in a different decision, such as You were denied a loan because your annual income was £30,000. If your income had been £45,000, you would have been offered a loan. (Wachter et al. 2017). In philosophy, counterfactuals have been characterized by the concept of a ‘closest possible world’, that is, the smallest change required to obtain a different (and more desirable) outcome (Pollock 1976). Contrastive explanations have been proposed mainly for image classification. For instance, the contrastive explanation method CEM (Dhurandhar et al. 2018) highlights what is minimally but critically absent in an image to belong to a given class. The MMD-critic (Kim et al. 2016) can identify nearest prototypes and nearest miss instances in image data such as handwritten digits and Imagenet datasets. Furthermore, an algorithm ProtoDash has been proposed to identify prototypes and criticisms for arbitrary symmetric positive definite kernels which has been applied to both tabular as well as image data.

An approach related to counterexamples has been proposed in early AI research by Winston in the context of learning relational concepts such as arch (Winston 1970). He demonstrated that presenting near miss examples where only a small number of relational aspects is missing to make an object a member of a class results in faster learning. Similarly, in cognitive science research, it has been shown that alignment of structured representations helps humans to understand and explain concepts (Gentner and Markman 1994). Gentner and Markman found that it is easier for humans to find differences between pairs of similar items than between pairs of dissimilar items. For example, it is easier to explain the concept of a light bulb by contrasting it with a candle than with a cat (Gentner and Markman 1994).

Induction of relational concepts has been investigated in Inductive Logic Programming (ILP) (Muggleton and De Raedt 1994), statistical relational learning (Koller et al. 2007), and recently also in the context of deep learning with approaches such as RelNN (Kazemi and Poole 2018) and Differentiable Neural Computers (DNCs) (Graves et al. 2016). DNCs have been demonstrated to be able to learn symbolic relational concepts such as family relations or travel routes in the London underground system. These domains are typical examples for domains where ILP approaches have been demonstrated to be highly successful (Muggleton et al. 2018). For DNCs, questions and answers are represented as Prolog clauses. However, in contrast to ILP, the learned models are black-box. For the family domain as well as for an isomorphic fictitious chemistry domain it has been shown, that rules learned with ILP fulfill Donald Michie’s criterion of ultra-strong machine learning (Muggleton et al. 2018). Ultra-strong machine learning according to Michie requires a machine learning approach to teach the learned model to a human, whose performance is consequently increased to a level beyond that of the human studying the training data alone. This characteristics has been related to the comprehensibility of learned rules or explanations (Muggleton et al. 2018): Comprehensibility has been defined such that a human who is presented with this information is able to classify new instances of the given domain correctly.

For ILP as well as for other relational learners such as DNCs, verbal explanations can be helpful to make a system decision transparent and comprehensible. For example, it can be explained why \({\textit{grandfather(ian,kate)}}\) holds by presenting the relations on the path from ian to kate in the family tree given in Fig. 1: Ian is a grandfather of Kate because Ian is male and Ian is the father of Tom and Tom is the father of Kate. Alternatively, it might be helpful for understanding the concept \({\textit{grandfather}}\) to present a contrastive example in form of a near miss explanation. For instance, Jodie is NOT the grandfather of Kate because she is NOT male or Mat is NOT the grandfather of Ian because he is in a child-of-child relation to Ian and NOT in a parent-of-parent relation. The first near miss corresponds to the concept of a grandmother, emphasizing the importance of the attribute male for \({\textit{grandfather}}\). The second near miss corresponds to the concept of a grandson, emphasizing the importance of the relation parent. To our knowledge, generating such near miss examples to explain learned relational concepts has not been investigated yet—neither in the context of ILP nor for other machine learning approaches.

Fig. 1
figure 1

An example family tree. Rectangles denote male persons, ellipses denote female persons, and solid arrows denote the parent relation. The bold solid arrows indicate a trace for a positive example. Non-solid arrows indicate near miss explanations

Full size image

In the following, we discuss the function of near miss examples. Afterwards, we present an algorithmic approach to generate near miss examples in the context of ILP and demonstrate the approach for a generic family domain, a visual domain and a real world domain dealing with file management (Siebers and Schmid 2019). Finally, we present an empirical evaluation with human participants where we compare human preferences of different types of explanations for the family and the arch domain, namely rule-based global explanations, example-based explanations, as well as near miss and far miss contrastive explanations.

2 The function of near miss examples

Near miss examples have been introduced by Winston as a human-like strategy to machine learning (Winston 1970): A near miss example for a concept is an example which does not belong to the concept but has a strong overlap to positive examples. Such near miss examples are helpful to guide the model construction of a machine learning algorithm (Telle et al. 2019). Winston illustrated learning with near misses in the context of relational visual domains. Concepts are represented as compounds of primitive blocks such as cubes. For instance, positive examples for arches must consist of at least two objects playing the role of supporters (pillars) and another object on top of them (roof). Negative examples for an arch might be a tower of several cubes—a far miss—or two pillars with no space between them covered by a roof—a near miss.

In the context of machine learning of relational concepts such as the Winston arches, molecules (King et al. 1996), or Turing-complete languages (Telle et al. 2019), carefully constructed near misses as negative examples can speed up learning (see Fig. 2a). In this case, the machine learning expert has a similar role to that of a school teacher who identifies helpful examples (Schmid et al. 2003). We propose that what is effective for learning is also effective for explaining a learned model (see Fig. 2b): for some concept that an AI system has learned, it can explain its model by constructing a near miss example. While machine learning typically is unidirectional—the human provides training examples and the system generalizes a model—explanations can support interactive machine learning scenarios based on a bidirectional partnership between human and AI system (Nguyen et al. 2018).

Fig. 2
figure 2

Duality of learning and explaining

Full size image

While there are some considerations about what constitutes helpful examples in educational psychology (Gentner et al. 2003) and the insights given in Winston’s seminar work (Winston 1970), there exist no general principles to construct helpful near miss explanations. We base our algorithm presented in the next section on some general observations which we will illustrate with the family domain example of Fig. 1.

3 Near miss explanations

In the following, we will introduce the GeNME algorithm for generating near miss explanations. Our approach complements the interpretable machine learning approach ILP (Muggleton et al. 2018) with a contrastive explanation component. First, we will introduce notation and basic concepts of ILP. Then, the concept of a near miss explanation is introduced formally and the generation algorithm is presented.

3.1 Notation

ILP algorithms learn models for subsets of the logic programming language Prolog (Sterling and Shapiro 1994). In the following, we consider function-free Horn clausal theories.

A term is either a variable or a constant symbol. A predicate symbol followed by a bracketed n-tuple of terms is called atom, or literal. The function \({\text {sym}}(A)\) returns the predicate symbol of atom A. The number of arguments a predicate symbol takes is its arity. A predicate symbol is called attribute if it has arity one and relation if its arity is larger than one. A literal is called ground when it has only constants as arguments. By convention, constants and predicate symbols are represented by lowercase strings and variables by uppercase strings.

A clause is an implication where the antecedent is a finite set of literals and the consequent is an atom. We write the implication reversed, as \(H \Longleftarrow \{ L_1, \ldots , L_m \}\). The consequent of a clause H is called its head and its antecedent is called the body. We define \({\text {head}}(C) = H\) and \({\text {body}}(C) = \{ L_1, \ldots , L_m \}\). For convenience, braces surrounding the body can be omitted. If the body of clause C is the empty set, \(H {\Longleftarrow}\){}, we call C a fact, omit the antecedents, and simply write H. Clauses with non-empty body can also be called rules. A clause is called ground when all its literals are ground. A finite set of clauses is called a finite clausal theory T, or simply a theory. The set of constant symbols occurring in T is \({\mathscr {C}}_T\). If it is clear from the context, we will omit the index.

The function \({\text {vars}}(L)\) returns the set of variables occurring in literal L. This is extended to sets of literals, \({\text {vars}}(\{ L_1, L_2, \ldots , L_n \}) = \bigcup _{i=1}^n {\text {vars}}(L_i)\), and clauses, \({\text {vars}}(C)={\text {vars}}({\text {head}}(C)) \cup {\text {vars}}({\text {body}}(C))\). A substitution is a mapping from variables to terms. We denote a substitution \(\theta\) by \(\{ x_1 \mapsto t_1, \ldots , x_k \mapsto t_k \}\) where \(x_1, \ldots , x_k\) are variables and \(t_1, \ldots , t_k\) are terms. A substitution is applied to a term by simultaneously replacing all \(x_i\) in the term by the corresponding \(t_i\)’s. A substitution is applied to a literal by applying it to all terms in the literal. A substitution is applied to a clause by applying it to all literals in the clause. We denote the application of the substitution \(\theta\) to a term, literal, or clause X by \(X\theta\). A substitution \(\theta\) is called minimal w.r.t. clause C, if \(C\theta\) is ground and there exists no \(\theta ^\prime\) such that \(C\theta ^\prime\) is ground and \(\vert \sigma \vert < \vert \theta \vert\).

If a literal or a set of literals K is true given a clausal theory T, we say that T models K and write \(T \models K\). Theory T models an atom A if there exists a clause \(C \in T\) and a substitution \(\theta\) such that \(A={\text {head}}(C\theta )\) and \(T \models {\text {body}}(C\theta )\). A theory T models a set of literals \(\{ L_1, \ldots , L_n \}\) if there is a substitution \(\theta\) such that T models \(L_i\theta\) for \(1\le i \le n\). By definition, the empty set \(\{ \}\) is modeled by any theory.

3.2 Basic concepts of ILP

ILP is a sub-field of symbolic machine learning which deals with learning clausal theories from examples (Muggleton and De Raedt 1994). Such clausal theories allow to represent relational concepts where the target is a relation or an attribute defined over relational structures. For instance, clauses for the binary relation grandfather can express family relations between persons or the attribute arch can express whether a construction is an arch. ILP learns such clauses from positive examples, like \(\textit{grandfather(ian,kate)}\), and negative examples, such as \(\textit{grandfather(alan,tom)}\) (see Fig. 1). Positive and negative examples for the target concept are represented as ground atoms. Additionally, a background theory must be provided. In the family domain, the facts \(\textit{parent(tom,kate)}\) and \(\textit{male(ian)}\) can be part of the background theory (Fig. 3). If a background theory consists only of facts, it is often called background knowledge. The learned theory together with the background theory must model all positive examples and no negative example.

Fig. 3
figure 3

Background knowledge for the family domain together with a selection of positive and negative examples for the grandfather concept

Full size image

Assume that the learned theory for \(\textit{grandfather}\) consists of a single clause:

$$\begin{aligned} {\texttt{grandfather}} (A,B) {\Longleftarrow} {\texttt{male}}(A), {\texttt{parent}} (A,C), {\texttt{parent}} (C,B). \end{aligned}$$
(1)

In general, a learned theory can include several clauses characterizing the target concept. For example, the target grandparent can be described by a set of clauses taking into account the genders of the respective parents. It can also be the case that target clauses are not exclusive. That is, a positive example P might follow from multiple clauses. With the learned theory, new instances given as ground atoms can be classified. For example, \(\textit{grandfather(alan,kate)}\) will be classified as positive; \(\textit{grandfather(becky,tom)}\) will be classified as negative.

3.3 Near miss examples and explanations

Positive classified instances are modeled by the learned theory as introduced in Sect. 3.1. As mentioned above, theory T consists of predefined background clauses and clauses learned for the target concept. For example, the grandfather relation holds for ian and kate given the theory in Equation 1 together with background knowledge about parent relations and gender of persons in a given family domain as the one in Fig. 3. An explanation for this fact has to make explicit how this can be derived from the theory. The reason why \(\textit{grandfather(ian,kate)}\) holds is that ian is male and ian is a parent of tom and tom is a parent of kate. In general, we call an explanation for a positive example local explanation:

Definition 1

(Local Explanation) A local explanation for a positive example P is a ground clause \(C\theta\) where \(C \in T\) such that \(P = {\text {head}}(C\theta )\) and \(T \models {\text {body}}(C\theta )\).

To emphasize which information is crucial for making someone a grandfather of someone else, a near miss explanation might be helpful. As introduced in Sect. 2, a near miss example is not a positive instance for the target concept, but illustrates a semantically similar concept. For instance, given the positive example \(\textit{grandfather(ian,kate)}\), possible near miss examples could be the female parent of a parent (that is the grandmother) of kate or a male child of a child (that is a grandson) of ian. Formally, we define near miss explanations and near miss examples as follows:

Definition 2

(Near Miss Explanation) Let \(C\theta\) be a local explanation, \(C^\prime\) a minimally changed clause, and \(\theta ^\prime\) a minimal substitution. We call \(C^\prime \theta ^\prime\) a near miss explanation if \(T \models {\text {body}}(C^\prime \theta ^\prime )\) and \(T \not \models {\text {head}}(C^\prime \theta ^\prime )\). Using the operator \(\varDelta\) to mark near miss examples, \(\varDelta L\) is a near miss example if \(L = {\text {head}}(C^\prime \theta ^\prime )\).

What constitutes a minimally changed clause is domain dependent. In general, the most basic change is replacing one literal in the body by its negation. For example, the attribute \(\textit{male(alan)}\) could be changed to \(\lnot \textit{male(alan)}\); an attribute \(\textit{large(x)}\) to \(\lnot \textit{large(x)}\); a relation \(\textit{above(top,bottom)}\) to \(\lnot \textit{above(top,bottom)}\). However, negations are too unspecific for many domains and are not part of our definition of theories. Therefore, we propose to explicitly define semantically opposing predicate symbols when modeling a particular domain. In natural language semantics, such relational opposites are one of the basic relations between lexical units (Palmer 1981). In the family domain, pairs \({{\textit{male} \leftrightarrow \textit{female}}}\) and \({{\textit{parent} \leftrightarrow \textit{child}}}\) are semantic opponents. To explain \(\textit{grandfather(ian,kate)}\), the near miss example \(\varDelta \textit{grandfather(jodie,kate)}\) (which is actually the grandmother) can be derived by replacing \(\textit{male(A)}\) with \(\textit{female(A)}\) in Equation 1. An alternative near miss can be constructed by inverting the parent relation to child. Because grandfather relies on the transitive sequence of two parent relations, both occurrences should be replaced, resulting in \(\varDelta \textit{grandfather(mat,ian)}\) (which should actually read \(\textit{grandson(mat,ian)}\)).

Depending on the domain, a minimal change of a clause C might therefore consist of either replacing a single literal or multiple literals. Which literals are to be replaced may also depend on the semantic opponents involved. Thus, we introduce domain dependent functions called rewriting filters \({{\mathscr {V}}_{p \mapsto q}}\) to formalize this connection. \({{\mathscr {V}}_{p \mapsto q}}(C)\) selects subsets of the literals in \({\text {body}}(C)\) such that replacing the predicate symbol p with the predicate symbol q in the selected literals in the body of C constitutes a minimal change to C.

Consider the clause D, \(\textit{grandfather(X,Y)} \Longleftarrow \textit{male(X)}, \textit{parent(X,Z)}, \textit{parent(Z,Y)}, \textit{male(Z)}\). This is an overly specific definition of grandfather where not only the person denoted as grandfather (e.g., the constant ian bound to variable X) is constrained to male, but also the intermediate parent (e.g., the constant tom bound to variable Z). Given the semantics of the family domain, rewriting of male by its semantic opponent female is only necessary for male(A). Applying \({{\mathscr {V}}_{\textit{male} \mapsto \textit{female}}}\) to D thus should yield \(\left\{ \{ \textit{male(X)} \} \right\}\). Likewise, for the transitive relation parent it makes sense to constrain replacement by its semantic opponent such that parent is replaced by child if and only if the occurring variables in these literals form a chain. Applying \({{\mathscr {V}}_{\textit{parent} \mapsto \textit{child}}}\) to D thus should yield \(\big \lbrace \lbrace \textit{parent(X,Z)}, \textit{parent(Z,Y)} \rbrace \big \rbrace\).

The specific constraints depend on the given domain. Therefore, we assume that the person modelling the domain provides this information explicitly by the rewriting filters. There may be more than one applicable rewriting filter and each filter may yield more than one subset of body literals to change. Changing the literals from exactly one subset constitutes a minimal change. In general, there may be multiple minimally changed clauses \(C^\prime\) for a given clause C. Applying the substitution \(\theta\) of a local explanation \(C\theta\) to a minimally changed clause \(C^\prime\) does not result in a near miss explanation. Since the head of a clause is not changed, \({\text {head}}(C^\prime \theta ^\prime )={\text {head}}(C\theta )\), the head of the near miss explanation would be a positive instance. The substitution \(\theta ^\prime\) for a near miss explanation may share more or less elements with \(\theta\). For a fixed \(C^\prime\) different substitutions \(\theta ^\prime\) constitute near miss explanations with different degrees of similarity to the local explanation \(C\theta\) for the positive instance P:

Definition 3

(Degree of Near Miss Explanation) Given a near miss explanation \(C^\prime \theta ^\prime\) w.r.t. local explanation \(C\theta\), the degree d of the near miss explanation is the number of changed replacements, \(d = |\theta \setminus \theta ^\prime |\).

3.4 Algorithm

The GeNME algorithm (Algorithm 1) identifies near miss explanations for a positive example. Given a theory T, a finite set of rewriting filters O, and a positive example P, it returns a family of sets, \(({\mathscr {E}}_d)_{d \in \mathbb {N}}\), where each \({\mathscr {E}}_d\) contains near miss explanations of degree d, inducing a partial ordering over contrastive explanations in relation to instance P. GeNME follows a generate-and-test approach. First, it generates the set of all near miss candidates.

Definition 4

(Near Miss Candidate) A near miss candidate for a positive example P is a ground atom N which has the same predicate symbol as P, \({\text {sym}}(N) = {\text {sym}}(P)\), but is not modeled by the theory T, \(T \not \models N\).

figure a

As second step, GeNME checks for all near miss candidates whether there is a fitting near miss explanation, iterating over all local explanations in the process. For each local explanation, GeNME iterates over all rewriting filters (line 4) and all selected subsets of body literals to generate a minimally changed clause \(C^\prime\). For each such minimally changed clause, GeNME iterates over all near miss candidates (line 7) and all possible substitutions \(\theta ^\prime\) with constants in \({\mathscr {C}}\) in increasing degree from \(\theta\) (lines 10–18). If there are substitutions \(\theta ^\prime\) such that \({\text {head}}(C^\prime \theta ^\prime )\) equals the near miss candidate and the theory models \({\text {body}}(C^\prime \theta ^\prime )\) for a given degree (line 15), then all near miss explanations for this degree are added to \({\mathscr {E}}_d\) (lines 15 and 19) and GeNME continues with the next candidate.

3.5 Complexity, termination and correctness

The core building blocks of GeNME are consequence tests, that is, whether some set of literals \({\mathbf {L}}\) is modeled by the theory T. Thus, we will assess the complexity of our algorithm by the number of consequence tests (\(T \models {\mathbf {L}}\)) required. Let the p-theory \(T_p\) be the subset of theory T that contains all clauses of T whose head has the predicate symbol p, \(T_p = \lbrace C \in T \mid {\text {head}}(C) = p \rbrace\). For the runtime complexity of GeNME the following theorem holds:

Theorem 1

(Complexity) Given a theory T, set of rewriting filters O, and positive example \(P= p(t_1,\ldots ,t_a)\), the runtime complexity of GeNME is polynomial in the size of the p-theory \(|T_p |\), the number of rewriting filters \(|O |\), and the number of constants in the theory \(|{\mathscr {C}} |\) and exponential in the arity a of p, the maximal number of variables in any clause in the p-theory \(v_{max}\), and the maximal number of literals in the body of any clause in the p-theory \(l_{max}\).

Proof

In its first step, GeNME creates the set of all near miss candidates \({\mathscr {N}}\). Therefore, it must iterate over all clauses C in the p-theory and all minimal substitutions \(\sigma\) for C. For each combination of C and \(\sigma\), one consequence test must be performed (and fail). Since \(\sigma\) is a minimal substitution, \(|\sigma | = {\text {vars}}(C)\). Thus, creating \({\mathscr {N}}\) has a complexity of \({{\mathscr {O}}}(|{\mathscr {C}} |^{v_{max}}|T_p |)\) where \(v_{max}\) is the maximal number of variables in any clause in \(T_p\).

For the second step, we consider the algorithm loops from inside out. For fixed \(C^\prime\), \(\theta _1\), \(\theta _2\), and N, the inner-most loop (lines 13–16) iterates over all possible altered substitutions \(\theta _2^\prime\) and tests whether \(C^\prime (\theta _1 \cup \theta _2^\prime )\) is a near miss explanation. Since every term in \(\theta _2^\prime\) must be different than the corresponding one from \(\theta _2\), there are \((|{\mathscr {C}} |-1)^d\) possible \(\theta _2^\prime\)’s. Thus, the inner-most loop has a complexity of \({{\mathscr {O}}}(|{\mathscr {C}} |^d)\). This loop is repeated for every possible partitioning such that \(\theta _2\) has cardinality d (line 12) where d increases up to \(|\theta |\) unless some near miss explanation is found (line 10). Thus, the complexity of the complete while loop (lines 8–19) is \(\sum _{d=1}^{|\theta |} \left( {\begin{array}{c}|\theta |\\ d\end{array}}\right) (|{\mathscr {C}} | - 1)^d = {{\mathscr {O}}}(2^{|\theta |}|{\mathscr {C}} |^{|\theta |})\).

The while loop is repeated for every near miss candidate. Since every near miss candidate must have the same predicate symbol as P which has arity a, there may be up to \(|{\mathscr {C}} |^a-1\) near miss candidates. For any given clause C, the current selected rewriting filter \({{\mathscr {V}}_{p \mapsto q}}\) (line 4) may select an arbitrary subset of C’s body literals except the empty set (this would imply \(C = C^\prime\)). Consequently, there are up to \(2^{|{\text {body}}(C) |}-1\) literal sets \({\mathscr {L}}\) (line 5). The complexity of finding near miss explanations for given local explanation \(C\theta\) is \({{\mathscr {O}}}(|O | |{\mathscr {C}} |^{|\theta |+a} 2^{|{\text {body}}(C) |+|\theta |})\).

Similar to near miss candidates, local explanations are constructed from some clause C in the p-theory and a minimal substitution \(\theta\) for C. There are up to \(|T_p | |{\mathscr {C}} |^{v_{max}}\) local explanations. Consequently, the complexity of the second part of the algorithm (lines 3–23) is \({{\mathscr {O}}}(|T_p | |O | |{\mathscr {C}} |^{2 v_{max}+a} 2^{l_{max}+v_{max}})\) where \(l_{max}\) is the maximal number of literals in the body of any clause in \(T_p\). Finally, since the complexity of the first step and the complexity to generate all local explanations is lower than the complexity of the second part, the complexity of the algorithm is identical to the complexity of the second part. \(\square\)

Theorem 2

GeNME will perform a finite number of consequence tests.

Proof

Assume, some run of GeNME will perform exactly f(T, O, P) consequence tests. As shown above, \(f(T,O,P) \in {{\mathscr {O}}}(|T_p | |O | |{\mathscr {C}} |^{2 v_{max}+a} 2^{l_{max}+v_{max}})\). Since T is a finite clausal theory and \(T_p\) is a subset of T, \(T_p\) is finite. Since every clause in T is finite, \({\mathscr {C}}\), \(v_{max}\), and \(l_{max}\) are finite. The set of rewriting filters O is finite. Consequently, f(T, O, P) is finite. \(\square\)

Theorem 3

(Termination) GeNME will terminate in finite time.

Proof

As stated in Sect. 3.1, our definition of clausal theories is identical to function-free Horn clauses. Function-free Horn clausal theories are decidable (Tamaki and Sato 1986). Thus, consequence tests can be evaluated in finite time. The theorem directly follows from theorem 2 and this fact. \(\square\)

Theorem 4

(Correctness) GeNME returns a family of sets of local explanations \(({\mathscr {E}}_d)_{d \in \mathbb {N}}\). Each element of any set \({\mathscr {E}}_d\) is a near miss explanation w.r.t a local explanations as given in Definition 2.

Proof

Each \(E \in {\mathscr {E}}\) with \(E=C^\prime (\theta _1 \cup \theta _2^\prime )\) (line 14) is a near miss explanation if and only if (i) \(C^\prime\) is a minimally changed clause, (ii) \(\theta _1 \cup \theta _2^\prime\) is a minimal substitution, (iii) \(T \models {\text {body}}(E)\), and (iv) \(T \not \models {\text {head}}(E)\).

(i) holds because \(C^\prime\) is constructed from C by applying a single rewriting filter as defined in Sect. 3.3 (line 6). (ii) holds since \(\theta _1, \theta _2\) is a partition of the minimal substitution \(\theta\), \(\theta _2^\prime\) replaces variables only with constants and has the same number of elements as \(\theta _2\) (line 13). (iii) is explicitly tested in line 15. (iv) follows from \({\text {head}}(E) = N\) which is explicitly tested (line 15). As N is a near miss candidate (lines 2 and 7), by Definition 4, \(T \not \models N\) holds. \(\square\)

4 Application to example domains

We realized the GeNME algorithm in Prolog. In the following, we demonstrate the generation of near miss explanations applying GeNME for the family domain, a relational visual domain of blocksworld arches, and a real world domain dealing with file management.

4.1 Family domain

Table 1 Number of found near miss explanations by degree in the family domain
Full size table

In Fig. 3 the family domain has been introduced with the background clauses male, female, and parent for a small number of constants (first names). In addition, child has been introduced as semantic opponent in Sect. 3.3. As an example for a target concept grandfather has been defined in Equation 1. We apply GeNME to this target concept and additionally to the concept of a daughter:

$$\begin{aligned} {\texttt{daughter}}(A,B) \Longleftarrow {\texttt{female}}(A), {\texttt{child}}(A,B). \end{aligned}$$
(2)

For both cases, the same family tree is used (see Fig. 1). The following rewriting filters, as introduced in Sect. 3.3, are provided: \({{\mathscr {V}}_{\textit{male} \mapsto \textit{female}}}\), \({{\mathscr {V}}_{\textit{female} \mapsto \textit{male}}}\), \({{\mathscr {V}}_{\textit{parent} \mapsto \textit{child}}}\), and \({{\mathscr {V}}_{\textit{child} \mapsto \textit{parent}}}\). GeNME is applied to explain the positive examples \(P_1 = \textit{grandfather(ian,kate)}\) and \(P_2 = \textit{daughter(becky,jodie)}\).

For grandfather, there are four positive examples in the given domain (see Fig. 1) and given the 10 persons additionally 96 pairs for which the grandfather relation does not hold. Out of these 96 near miss candidates for \(P_1\), GeNME identifies 8 as near miss examples (8.3 %, see Table 1). The near miss explanation with the lowest degree of 1 is \(\textit{grandfather(jodie,kate)} \Longleftarrow \textit{female(jodie)}, \textit{parent(jodie,tom)}, \textit{parent(tom,kate)}\). Indeed, this is intuitively a very close near miss (the grandmother of kate). For the given family tree there are 8 pairs of persons for which the daughter relation holds. GeNME identifies 10 out of 92 candidates for \(P_2\) as near miss examples (10.9 %). For \(P_2\), there is a single near miss explanation with the lowest degree of one, namely jodie’s son (\(\textit{daughter(tom,jodie)} \Longleftarrow \textit{male(tom)}, \textit{child(tom,jodie)}\)).

Both minimal near miss explanations were constructed by exchanging the unary predicates male resp. female by their semantic opponent. Rewriting parent or child also results in plausible near miss explanations, but with degree 2: For \(P_1\), mat is the grandson of ian and for \(P_2\), jodie is the mother of becky.

4.2 Winston arches domain

Winston introduced the concept of an arch in the context of a blocksworld domain by characteristic relations between blocks: contains (a structure contains a block), supports (a block supports another block), (not_)meets (two blocks do (not) meet horizontally), and is_a (a block has a certain shape; the shape can either be wedge or brick). The target concept of arch is defined as:

$$\begin{aligned} \begin{aligned} {\texttt{arch}}(A)&\Longleftarrow {\texttt{contains}}(A, X), {\texttt{contains}}(A, Y), {\texttt{contains}}(A, Z), \\&\quad {\texttt{is\_a}}(X, T), {\texttt{is\_a}}(Y, {\texttt{brick}}), {\texttt{is\_a}}(Z, {\texttt{brick}}), \\&\quad {\texttt{supports}}(Y, X, A), {\texttt{supports}}(Z, X, A), \\&\quad {\texttt{not\_meets}}(Y, Z, A). \end{aligned} \end{aligned}$$
(3)

Figure 4 shows some positive and negative examples for a restricted concept of an arch where all positive examples have a block of type wedge as the top. In Fig. 5, the background knowledge for these examples is given. In addition, supported_by is defined as the inverse of supports.

Fig. 4
figure 4

Some positive and negative example structures for the Winston arches domain

Full size image
Fig. 5
figure 5

Background knowledge and background theory for the Winston arches domain

Full size image

With the positive example \(\textit{arch(struct1)}\) and the rewriting filters \({{\mathscr {V}}_{\textit{supports} \mapsto \textit{supported\_by}}}\), \({{\mathscr {V}}_{\textit{supported\_by} \mapsto \textit{supports}}}\), \({{\mathscr {V}}_{\textit{meets} \mapsto \textit{not\_meets}}}\), and \({{\mathscr {V}}_{\textit{not\_meets} \mapsto \textit{meets}}}\) GeNME yields the following near miss explanation with degree \(d = 1\):

$$\begin{aligned} {\texttt{arch}}\,({\texttt{struct4}}) \Longleftarrow &{\texttt{contains}}\;({\texttt{struct4}}, {\texttt{a1}}), {\texttt{contains}}({\texttt{struct4}}, {\texttt{b}}),\\&{\texttt{contains}}\;({\texttt{struct4}}, {\texttt{c}}), {\texttt{is\_a}}({\texttt{a1}}, {\texttt{wedge}}), {\texttt{is\_a}}\;({\texttt{b}}, {\texttt{brick}}),\\& {\texttt{is\_a}} ({\texttt{c}}, {\texttt{brick}}), {\texttt{supports}}\;({\texttt{b}}, {\texttt{a1}}, {\texttt{struct4}}),\\& {\texttt{supports}}\;({\texttt{c}}, {\texttt{a1}}, {\texttt{struct4}}), {\texttt{meets}}\;({\texttt{b}}, {\texttt{c}}, {\texttt{struct4}}) \end{aligned}$$

\(\varDelta \textit{arch(struct4)}\) is a plausible near miss example since the only change is that the two supporting pillars meet. Further explanations are found for \(d = 3\):

$$\begin{aligned} \begin{aligned} {\texttt{arch}}\;({\texttt{struct6}}) \Longleftarrow&{\texttt{contains}}\;({\texttt{struct6}}, {\texttt{a2}}), {\texttt{contains}}\;({\texttt{struct6}}, {\texttt{b}}),\\&{\texttt{contains}}\;({\texttt{struct6}}, {\texttt{c}}), {\texttt{is\_a}}({\texttt{a2}}, {\texttt{brick}}), {\texttt{is\_a}}({\texttt{b}}, {\texttt{brick}}),\\&{\texttt{is\_a}}({\texttt{c}}, {\texttt{brick}}), {\texttt{supports}}\;({\texttt{b}}, {\texttt{a2}}, {\texttt{struct6}}),\\&{\texttt{supports}}\;({\texttt{c}}, {\texttt{a2}}, {\texttt{struct6}}), {\texttt{meets}}\;({\texttt{b}}, {\texttt{c}}, {\texttt{struct6}}) \end{aligned} \end{aligned}$$
$$\begin{aligned} \begin{aligned} {\texttt{arch}}\;({\texttt{struct5}}) \Longleftarrow&{\texttt{contains}}\;({\texttt{struct5}}, {\texttt{a2}}), {\texttt{contains}}\;({\texttt{struct5}}, {\texttt{b}}),\\&{\texttt{contains}}\;({\texttt{struct5}}, {\texttt{c}}), {\texttt{is\_a}}({\texttt{a2}}, {\texttt{brick}}), {\texttt{is\_a}}({\texttt{b}}, {\texttt{brick}}),\\&{\texttt{is\_a}}\;({\texttt{c}}, {\texttt{brick}}), {\texttt{supported\_by}}\;({\texttt{b}}, {\texttt{a2}}, {\texttt{struct5}}),\\&{\texttt{supported\_by}}\;({\texttt{c}}, {\texttt{a2}}, {\texttt{struct5}}), {\texttt{not\_meets}}\;({\texttt{b}}, {\texttt{c}}, {\texttt{struct5}}). \end{aligned} \end{aligned}$$

These two explanations have a larger distance to the to be explained struct1 than \(\varDelta \textit{arch(struct4)}\). That is, they are “not that near misses” which we will call far misses when we want to discriminate between misses with lower and higher degrees. Table 2 shows the number of near misses GeNME found for the different rewriting filters.

Table 2 Number of found near miss explanations by degree in the Winston arches domain
Full size table

4.3 File management domain

In the file management domain, we aim at identifying irrelevant files which could be deleted by the user (Siebers and Schmid 2019). The domain is represented by relations such as creation_time, file_size, file_name, and media_type. Figure 6 shows an excerpt of the background knowledge for some file system. Additionally, the theory contains clauses for auxiliary relations, such as older, larger, and in_same_folder. A typical rule for irrelevancy is:

$$\begin{aligned} \begin{aligned} {\texttt{irrelevant}}(F) \Longleftarrow&{\texttt{in\_same\_folder}}(F, G), {\texttt{older}}(F, G) \\&{\texttt{media\_type}}(M, F), {\texttt{media\_type}}(M, G). \end{aligned} \end{aligned}$$
(4)
Fig. 6
figure 6

Excerpt of background knowledge for the file management domain together with a selection of positive and negative examples for the irrelevant concept

Full size image

We apply GeNME to explain two arbitrary positive examples, \(\textit{irrelevant(file10)}\) and \(\textit{irrelevant(file11)}\). We provide the rewriting filters \({{\mathscr {V}}_{\textit{older} \mapsto \textit{newer}}}\) and \({{\mathscr {V}}_{\textit{newer} \mapsto \textit{older}}}\) which both allow changing a single literal. As shown in Table 3, GeNME identifies 68 near miss examples from the set of 80 near miss candidates for both examples (85.0 %). For both examples, only a single near miss explanation of degree 1 is found: a file of the same media type located in the same folder which is newer than the file under consideration.

Table 3 Number of found near miss explanations by degree in the file management domain
Full size table

5 Empirical study of human preferences of explanation types

To investigate whether near miss explanations are considered helpful by humans, we conducted an empirical study on preferences of explanation modalities for the abstract relational family domain and the visual relational arches domain. For both domains, a cover story introducing a need for explanation to a specific recipient has been presented. Participants had to evaluate the helpfulness of explanations for the given setting by selecting their preferences in a pairwise comparison. In addition, an explicit rating of the helpfulness of the different modalities for different explanatory goals has been assessed. Details about the material, the method and results are described in the following subsections.

5.1 Rule-based and example-based explanations

Overall, four different types of explanations have been considered:

  • General rule (R) a global explanation of the concept a specific instance belongs to,

  • Example (E) an example-based explanation in form of a specific instance belonging to the concept,

  • Near Miss (N) a contrastive (negative) example which has a high degree of structural similarity to the specific instance under consideration but does not belong to the class,

  • Far Miss (F) a negative example for the considered concept which has a low degree of structural similarity to the specific instance under consideration.

Explanations were presented in form of natural language sentences—or, in the case of the arches domain, partially by visual illustrations. Natural language explanations can be generated from ILP learned rules in a straight-forward manner (Siebers and Schmid 2019; Schmid 2021).

The four types of explanations address different information needs (Miller 2019): To understand the general concept, a global rule can be assumed to be especially helpful. However, it might be the case that the helpfulness is different for abstract in contrast to visual domains. In the second case, a visual prototype might be more effective (Gurumoorthy et al. 2019). In cognitive psychology, visual prototypes have been shown to be an effective means of concept representation for basic categories (Rosch 1979). For simple domains, an arbitrary instance might convey information similar to a prototype. For instance, in medical textbooks, example images are given to illustrate what a specific skin disease looks like. A near miss example should be especially helpful to highlight what (missing) information would be necessary to make an object belong to a class (Gentner and Markman 1994). This is often helpful if feature values or relations are hard to grasp. For instance, mushroom pickers use images to distinguish an edible mushroom from the visually most similar toadstool. Arbitrary negative examples, especially far misses, can be assumed to be less helpful to understand a concept or why a specific instance belongs to a concept. This type of explanation has been introduced as a baseline. We assume that combining different types of explanations can be more effective than each of these explanations alone. Especially, a combination of a global rule with a near miss might be most efficient to explain relational concepts.

As cover story for the family domain (see Sects. 3.2 and 4.1) the family tree of Kate as given in Fig. 1 but without extra arrows for miss examples has been presented. Participants were asked to imagine a conversation with their friend Kate who is originating from a native American tribe. She is curious about the different definitions of family relations in western culture since she is not familiar with them and the definitions that she grew up with are very different from the participant’s. In particular, Kate wants to understand the grandfather relation between herself (Kate) and Ian.

The four explanations to choose from are:

  • (R)ule: A grandfather is a male parent of one of your parents.

  • (E)xample: One of your parents, Tom, has a male parent called Ian. Ian is your grandfather.

  • (N)ear Miss: Jodie, the female parent of your parent Tom is NOT your grandfather; it is your grandmother.

  • (F)ar Miss: Mat, the male child of Tom, who is the child of Ian is NOT the grandfather of Ian; it is his grandson.

The Winston arches domain has been introduced to the participants as shown in Fig. 4 without the object labels. Participants were given the context of playing with building blocks with their five-year-old son introducing a new type of building called arch given the examples and counterexamples in Fig. 4 with focus on the arch labeled struct1.

The explanations given for the arch domain are:

  • (R)ule: An arch consists of two rectangle blocks that do not touch. They support a triangle block.

  • (E)xample: given by presenting the structure labeled struct1.

  • (N)ear Miss: given by presenting the structure labeled struct4.

  • (F)ar Miss: given by presenting the structure labeled struct6.

5.2 Method

Given the cover story, the helpfulness of the different types of explanations has been assessed with a complete pairwise comparisons (Thurstone 1927). Choosing one alternative over another is considered as less prone to subjective biases than a direct rating of each item.

In a first part of the study, all pairings of the four explanation types have been presented in a randomized sequence and participants had to always choose that explanation of the pair which they found more helpful given the cover story. In a second part of the study, pairs of pairs of explanations have been presented in a random order. In a final part, the helpfulness of the four explanation types for different information needs as been assessed explicitly. Participants rated how helpful an explanation is to understand

  • the general concept,

  • a particular example instance for the concept,

  • what is not in the concept (exclusion).

on a scale from 1 to 5 with labels from not at all to absolutely.

The study was conducted as an online experiment with 73 valid participants (42 females, 31 males) with average age 35.72 (min 18, max 64). From an initial 151, we excluded 78 participants that either took 10 minutes or less for the experiment or that gave an incorrect answer to questions testing the participants’ attention. 43 participants were employed, 27 were students, 2 were self-employed and 1 person was retired. About 50% of the participants received first the family domain followed by the arches domain and the other half of participants started with the arches followed by the family domain.

Although this is an exploratory study, given the considerations above, we can formulate the following hypotheses: (1) Near miss examples should be preferred over far miss examples in the pairwise comparisons; (2) Near miss examples should be rated as the most helpful to understand the boundaries of a given concept; (3) For the visual domain example-based explanations should be preferred over the rule-based explanation, while—in contrast—for the abstract domain, the verbal, rule-based explanation is preferred.

5.3 Results and discussion

For the family domain, preference choices for the six pairings of the four explanations resulted in the following frequency ranking (rounded relative frequencies in brackets): R (0.43) > E (0.37) > N (0.17) > F (0.03). The rule for the relational concept was preferred over all other explanation modalities followed by example, near miss and far miss. Preferences between pairs of explanations were assessed by 15 pairwise comparisons. Preference choices resulted in the frequency ranking RE (0.32) > RN (0.21) > EN (0.19) > EF (0.13) > RF (0.13) > NF (0.02).

For the visual arches domain, single preferences favoured the example as explanation, near miss was preferred over far miss: E (0.45) > R (0.30) > N (0.18) > F (0.08). For the 15 pairwise comparisons, preference choices resulted in the frequency ranking EN (0.27) > RE (0.25) > EF (0.21) > RN (0.14) > RF (0.08) > NF (0.04).

Exact binomial tests comparing preferences for near miss and far miss examples show significant preferences for near miss examples for both the family (empirical mean \(= 0.890\), \(p < 0.001\)) and arches domain (empirical mean \(= 0.795\), \(p < 0.001\)).

The first hypothesis, that near miss explanations are generally preferred over far miss explanations was tested by comparing the frequencies of the single choices as well as by comparing pairs containing near miss explanations with such containing far miss explanations (see Table 4).

Table 4 Frequencies of preferences of near miss over far miss explanations (significance tested with exact binomial test, Holm correction has been used to adapt the p-values for multiple testing, *** denotes a \(p < 0.001\))
Full size table

The helpfulness ratings of the different types of explanations for different explanatory goals are summarized in Table 5. As expected for the abstract relational family domain, if the goal is to understand the general concept of a grandparent, the rule-based explanation is preferred over example-based explanations. For the goal to understand why a particular instance belongs to a concept, the example is the preferred explanation. For the purpose of understanding the boundaries of the concept, the near miss explanation is rated as most suitable and is significantly higher than the rating for the next preferred far miss explanation (\(t-\)test for dependent samples, df \(= 72\), \(t = -4.6382\), \(p < 0.001\)). Likewise, for the visual relational arches domain the near miss example was rated to be most helpful when the goal is to highlight what makes an instance to be outside the concept, but the difference is rather small and not significant (\(t-\)test for dependent samples, df \(= 72\), \(t = -1.0691\), \(p-\)value \(= 0.2886\)). Interestingly, for the goals of understanding the general concept and why some instance is an example for a concept, the ratings are interchanged: The rule was rated as better suited to understand why a particular example belongs to the concept and the example was rated most suited to explain the general concept.

The second hypothesis—that near misses are rated most helpful for the goal to understand the boundaries of a concept—has been confirmed although the difference to the next-best choice is only significant for the family domain.

Table 5 Results of the questions on which explanations fulfilled which purpose in (a) the family domain and (b) the arches domain
Full size table

The results for the helpfulness ratings in Table 5 already show an interesting difference between the family and the arches domain with respect to rule-based versus example-based explanations. The frequencies of preferences for rule versus example in the single pairwise comparisons is summarized in Table 6. There is a significant interaction of the preferred explanation type and the domain: 38 choices are for the rule in the family domain and the example in the arches domain. In the family-domain, in general the rule has been preferred over the example (51 vs. 22) while in the arches domain, the examples has been preferred over the rule (58 vs. 15).

Table 6 Frequencies of preferences of rule and of example for the Family and the Arches domain (combined values for the single choice for the pair rule—example and for the pairwise choices with rule in one pair and example in the other; McNemar’s \(\chi ^2 = 30.625\), df \(= 1\), \(p < 0.001\))
Full size table

The empirical results confirm that near misses are considered helpful by humans to understand the boundaries of a concept and thereby to avoid false positives. Our findings furthermore indicate that (1) different types of explanations are rated as most helpful for different explanatory goals, and (2) that what type of explanation is helpful also depends on the domain. Instead of the use of a cover story, a more realistic setting should be investigated as a next step. Similar to an experiment in the context of explaining the choice of moves in a strategy game (Ai et al. 2021), an explanation interface can be added to the learned models. Then, it can be assessed whether participants getting the explanation considered most helpful by the system show better performance than participants getting another explanation. In addition to performance, the effect of explanations on trust in the machine learned model can give interesting insights for the design of helpful explanations (Thaler and Schmid 2021).

6 Conclusions and further work

We introduced near miss explanations for relational concepts learned with ILP. As cognitive science research suggests (Gentner and Markman 1994), near miss explanations can play an important role to highlight what aspects are necessary for an instance to belong to a given class. The GeNME algorithm has been presented which generates near miss explanations with different degrees of nearness to a specific positive instance for a given concept.

The current version of GeNME is realized in Prolog and relies on a generate-and-test strategy, first generating and then checking all near miss candidates. Checking the candidates also follows a generate-and-test strategy, first minimally changing a clause and constructing a minimal substitution, and then checking whether these constitute a near miss explanation. There are different possibilities to improve this approach. For instance, near miss candidates might be used to restrict the choice of minimal substitutions. As the complexity of GeNME is exponential in the size of the substitution, substantial improvement may be gained. Alternatively, the set of near miss candidates might not be generated explicitly beforehand but could be constructed step by step. This strategy might reduce the number of consequence tests. In the worst case, the current algorithm as well as the proposed improvements have the same complexity. To determine which algorithmic strategy is more promising regarding average complexity, extensive empirical tests with different domains will be required.

In an empirical study, we investigated the helpfulness of near miss explanations in contrast to other example-based explanations and rule-based explanations. The abstract relational family domain and the perceptual relational arches domain were presented. Results showed that humans rated near miss explanations as helpful. Interestingly, for the abstract relational domain, rule-based explanations were favored over example-based explanations while for the perceptual relational domain example-based explanations were preferred. In general, there was a significant preference for near miss explanations over far misses. Our empirical findings together with the proposed GeNME algorithm introduce near miss explanations as a new type of explanations in relational domains.