1 Introduction

Topic modelling as evidenced by [1,2,3,4] is a well-established probabilistic generative model popular within computer science, focusing mainly on text mining, document classification, information retrieval, summarization, and many others. Some diverse applications involving topic modelling are presented in these articles [1, 5,6,7,8,9,10,11]. Topic modelling is an unsupervised learning technique that finds the latent topics of a document corpus using the word co-occurrence patterns present in the documents. Developing a complete generative model, such as the Latent Dirichlet Allocation (LDA), marked the beginning of topic modelling [5, 12,13,14,15]. Even though topic modelling using LDA is a popular and effective technique, LDA is considered relatively complex, tends to be topic model specific, and may not provide an accurate model in practice.

Recent development in deep learning has given rise to much efficient topic modelling techniques, such as neural topic models, recurrent neural networks, variational autoencoders, and transformer-based models. The research community considers the neural network-based models a black box function due to their complicated structure and nonlinearity. The exceptional precision of the models rarely justifies their reliability and complex user interactions. This obscure aspect might lead to issues if the model delivers incorrect results or malfunctions, particularly in fields like agriculture, forestry, health, and climate that affect human lives [16]. Explaining the neural network-based models can justify their decision-making process and outputs. With the advent of Explainable Artificial Intelligence (XAI), explaining the inner workings/predictions of obscure and sophisticated machine learning models was possible [17,18,19,20,21]. It helped researchers, developers, subject matter experts, and users better comprehend the model while utilizing its high performance and accuracy.

GNTM [22, 23] is a neural topic modelling technique [24, 25] that combines a graph neural network with a variational autoencoder. GNTM models the document corpus’s underlying graph structure and generates the topic distribution for each document. The document relation graph of the corpus serves as the input of the graph neural network that extracts the relationship between documents. The documents and words in the corpus become a node in the document relation graph. The nodes are connected based on how often a document and a word appear together. Finally, the variational autoencoder finds the latent representations of documents and topics. The output of the GNTM consists of documents-topics and topics-words probability distribution matrices.

The practical ramifications of GNTM are significant. This technology demonstrates exceptional performance when documents display complex linkages since it effectively identifies topics that include content-based relevance and relational importance. The GNTM model represents a notable progression in topic modeling, effectively addressing the challenge of integrating content and context while analyzing large collections of documents.

However, the applicability of GNTM is often questioned due to its inherent black-box nature, not only within the context of topic modeling but also in various other application domains. This lack of transparency present in GNTM decision-making processes can impede its utility and explainability, limiting its applicability. In the context of topic modelling, researchers and domain experts require a clear understanding of how topics are generated and how specific words and documents are assigned to a particular topic to impart confidence and be able to use the model’s results. Recognizing these challenges and limitations, the paper investigates the feasibility of explaining individual predictions of GNTM applied in a topic modelling context.

The main contribution of the paper is to develop a local explanation model equivalent to the GNTM. Decision trees, Naïve Bayes, and random forests are some probable candidates for the explanation model. This paper selects the Naïve Bayes classifier as the explanation model for its suitability to the specific NLP problem being addressed and as its predictions are more straightforward probabilistic calculations, making it relatively easy to understand how it arrives at its classifications or predictions. Experimental analyses demonstrate the similarities between the GNTM and the proposed explanation technique and how explanations are obtained.

The remainder of the paper is structured as follows: Sect. 2 describes relevant research works in the topic modelling context. The background theory of GNTM, Naive Bayes, the proposed methodology, and pseudocode are mentioned in Sect. 3. Section 4 discusses the details of NLP datasets, the experimental results showing the resemblance between the GNTM and trained local explainable model based on individual prediction, comparison with baseline topic modelling techniques, theoretical and practical implications, and limitations. The Section also demonstrates how explanation is achieved. Finally, Sect. 5 forms the conclusions and possible future directions.

2 Related Works

The inability of artificial intelligence and machine learning techniques to promote trust and acceptability due to their functional opaqueness has led to the concept of explainability. The international community has developed various techniques and methodologies to bring explainability to the present framework. This Section discusses relevant research on topic modelling, including graph neural network and explainable topic modelling techniques. The Section also identifies the research gap present in the context of topic modelling using GNTM.

A generative probabilistic model of a corpus known as the LDA is one of the most popular topic modelling techniques [12,13,14,15, 26, 27]. Modelling documents as a random mixture over latent topics, with each topic represented by a distribution over words, is the basic idea behind LDA. The need for an appropriate choice of distribution for each latent variable or the tolerance of laborious and case-by-case customized theoretical formulation restricts the flexibility and scalability of the model design. Traditional topic modelling techniques, including Markov Chain Monte Carlo [28, 29] or probabilistic variational inference [30], also suffer from this problem of flexibility and scalability.

The success of inferring topic models with a Variational autoencoder [31] has attracted more people to use deep learning techniques in topic modelling contexts. A variational autoencoder is a parameterized neural network that uses variational distributions to estimate the posterior of latent variables. A generalization of LDA known as the Neural Topic Model (NTM) is proposed in the article [32]. Neural topic models are rapidly growing, resulting from combining topic modelling and deep neural networks. Neural understanding problems such as text generation, summarization, and language models have witnessed many neural topic models achieving excellent performance. A summary of the research progress and a discussion of outstanding issues and potential future approaches is presented in the article [33]. A NTM particularly suited for conversational scenarios known as the Conversational Neural Topic Model (ConvNTM) is proposed in [34], in which the topics are discovered by formulating the multi-turn structure in dialogues. Various variants of neural topic models for topic modelling [35] has been developed recently. Graph neural network is one such neural topic model. The literature [36] gives a comprehensive overview of neural graph networks categorized into convolution, recurrent, spatial-temporal graph neural networks, and graph autoencoder. The article also addresses the possibility of applying graph neural networks in various applications, including natural language processing.

Instead of viewing documents in a corpus as a bag of words or sequences, it is possible to represent them as a graph. This representation enables graph neural networks to extract latent topics. The work proposed in the literature [22] introduces a topic modelling technique using a graph neural network. The proposed Graph Neural Topic Model (GNTM) transforms each document in the corpus into directed graphs with edges representing word dependency between word nodes. The model learns using a neural variational inference approach to encode document graphs. Instead of using the word co-occurrence, the work proposed in [23] uses commonsense relationships to explicitly imply semantic relevance. A relational graph neural network is used to capture the relational information present inside the graph. Furthermore, manifold regularization imposes constraints on the documents’ topic distributions. Another neural topic model, Graph Topic Model (GTM) [37], also represents the corpus as a graph representing the relationships between documents. In this graph, both documents and words in the corpus are indicated as nodes and linked to one another depending on the co-occurrence of words inside documents. The topical representation of a document node in GTM is derived by aggregating information from its multi-hop neighborhood, which consists of both document and word nodes. This aggregation process utilizes the Graph Convolutional Network (GCN) algorithm.

Several approaches, such as GNNExplainer [38], XGNN [39], SubgraphX [40], and Probabilistic Graphical Model (PGM)-Explainer [41], have been developed recently to explain the predictions obtained from graph neural networks. These methods are capable of providing instance and model-level explanations. Examples of instance-level explanation techniques include gradient/feature, perturbation, surrogate, and decomposition-based methods. An extension of the Local Interpretable Model Agnostic Explanation (LIME) [42] approach known as GraphLime [43], RelEx, and PGM-Explainer are surrogate methods. On the other hand, model-level approaches seek to provide general insights and high-level expertise to explain deep graph models. In particular, they look at what kinds of input graph patterns might cause a graph neural network to act in a given way, like maximizing a target prediction. Due to graph models’ discontinuous graph topology information, a model-level explanation of the graph neural network will be complicated. XGNN belongs to the model-level explanation technique using graph generation. The article [44] explains the challenges, different instance and model-level methods of graph neural networks, and comparative analysis using other evaluation metrics. Graph Neural Networks (GNNs) often struggle to explain the discovered latent relations in a manner that ensures their reasonableness and independence. Additionally, GNNs often need textual content of edges, which is often not present in real-world datasets. Topic-Disentangled Graph Neural Network (TDG) proposed in the literature [45] aims to overcome these limitations. The proposed approach involves using a topic module to efficiently manage node attributes for the purpose of constructing distinct and explainable semantic subspaces. Subsequently, a neighborhood routing mechanism assigns appropriate relation topics to each graph connection based on their association with these subspaces.

A multi-modal causability technique in medical analysis using GNNs is developed by introducing multi-modal embeddings and interactive explainability in the research work [46]. Another work [47] presents a novel approach called the Semantic Reinforcement Neural Topic Model (SR-NSTM) that addresses the challenges of sparse and explainable text representation. To enhance the quality of text representations, the SR-NSTM model takes a closer look at the generation process of sparse topic models and integrates contextual information using Bi-LSTM. The literature introduces a Topic-Disentangled Graph Neural Network (TDG) to address the challenge posed by GNNs, which often struggle to explain the extracted latent relations and ensure their plausibility and independence. Additionally, this technique aims to overcome the need for detailed textual information about these relationships, which is frequently absent in real-world datasets.

It’s apparent from examining the existing research papers that numerous resources are dedicated to explaining GNNs and NTM. However, there’s a noticeable lack of similar work explaining the outputs of the GNTM. This gap in providing explanations for GNTM results is identified as the research challenge that is being tackled in this paper.

3 Local Explainability Technique for Graph Neural Topic Models

This Section delineates the methodology of the proposed explainability technique for GNTM. The Section begins by providing a concise overview of the GNTM, local explainable model-Naive Bayes. The Section also gives the pseudo-code and a graphical representation of the proposed explainability technique.

3.1 Graph Neural Topic Model (GNTM)

GNTM [22] is a machine learning technique that combines graph neural networks and topic models to analyze and model large amounts of graph-structured text data. Graph neural network embeds the graph-structured text data formed from the datasets into a low-dimensional vector space. Furthermore, each vector in the low-dimensional vector space represents a document from the dataset. Furthermore, the documents are represented as directed semantic graphs, including word dependency as edges connecting the word nodes. The extraction of topics depends not only on the information inside individual documents but also on the evaluation of the impact of adjacent documents that are related by edges in the graph. The semantic structure of the texts is captured by propagating information along the network in accordance with the co-occurrence relationships between words in the documents. The GNTM tries to learn the low-dimensional vector representation (embedding) of each node in the graph. It then uses these embeddings to model the topics and relationships between the documents in the graph. Using both the graph structure and the text content of the documents, the GNTM can capture complex dependencies between the documents and make more accurate topic models than traditional topic models. The crux of the GNTM framework is in this interplay between the graph structure and topic modeling.

During the training process, GNTM acquires knowledge in two fundamental aspects, namely, document-topic distributions and topic-word distributions. The former examines the distribution of topics within individual documents, considering both the content and relational context. The latter pertains to characterizing the probability of each word’s occurrence inside each topic. Using a dual learning process guided by the graph structure results in enhanced precision and context sensitivity in topic representations.

Graph neural topic model achieves better performance than other traditional topic modelling techniques at the cost of being highly complex. This complexity makes the GNTM opaque, making it difficult for the user to understand why the model arrived at a particular output. A proper explainability technique can make GNTM even more fruitful in human-centric applications.

3.2 Local Explainable Model: Naive Bayes

In this work, Naive Bayes classifier [48, 49] is selected to function as the local explainable model. The Naive Bayes Classifier is a probabilistic classification technique based on the concepts of the Bayesian Theorem, which Thomas Bayes first introduced. Its core objective in classification tasks is establishing an optimal mapping between a new data instance and a predefined set of classifications within a specific domain. Mathematical operations are used to transform joint probabilities into the product of prior and conditional probabilities to facilitate probabilistic computation for this mapping. The term “naive” in Naive Bayes refers to the assumption of conditional independence among the features included in the model. This implementation is a versatile toolkit applicable across a wide spectrum of classification domains.

Consider a dataset x having n instances, \(x_{i}, i=1,2,\ldots ,n\) with ‘m’ attributes i.e., \(x_{i}=(x_{i1},x_{i2},x_{i3},\ldots ,x_{im})\). Assume that each instance belongs to one and only class, \(y \; \epsilon \; (y_{i1},y_{i2},y_{i3},\ldots ,y_{in})\). Naive Bayes learning pertains to the development of a Bayesian probabilistic model that gives a posterior class probability to a particular data instance \(P(Y=y_{i}\vert X=x_{i})\). The basic Naive Bayes classifier utilizes these probabilities to assign an instance to a class. The basic formulation of Naive Bayes applying Bayes theorem [50] is given in Eq. 1

$$\begin{aligned} P(y_{i}\vert x_{i})=\frac{P(x_{i} \vert y_{i}) \times P(y_{i})}{P(x_{i})} \end{aligned}$$
(1)

The numerator in Eq. 1 represents the joint probability between \(x_{i}\) and \(y_{i}\). The numerator can be written as given in Eq. 2

$$\begin{aligned}&P(x \vert y_{i}) \times P(y_{i})=P(x_{1}\vert x_{2},x_{3},\ldots ,x_{m},y_{i})\cdot P(x_{2}\vert x_{3},x_{4},\ldots ,x_{m},y_{i}) \nonumber \\&\quad \ldots ,\cdot P(x_{m} \vert y_{i}) \times P(y_{i}) \end{aligned}$$
(2)

Assuming the data instances \(x_{i}\) are independent to each other, \(P(x_{1}\vert x_{2},x_{3},\ldots ,x_{m},y_{i}) = P(x_{1}\vert y_{i})\). Thus, the numerator in Eq. 1 reduces to Eq. 3.

$$\begin{aligned} P(x \vert y_{i}) \times P(y_{i})&=P(x_{1} \vert y_{i})\cdot P(x_{2} \vert y_{i})\ldots \cdot P(x_{m} \vert y_{i})\cdot P(y_{i}) \nonumber \\&= \prod _{k=1}^{m} P(x_{k} \vert y_{i}) \cdot P(y_{i}) \end{aligned}$$
(3)

Substituting Eq. 3 in Eq. 1, the basic formulation of Naive Bayes becomes as given in Eq. 4.

$$\begin{aligned} P(y_{i}\vert x)=\frac{\prod _{k=1}^{m} P(x_{k} \vert y_{i}) \cdot P(y_{i})}{P(x)} \end{aligned}$$
(4)

The term P(x) in the denominator of Eq. 4 is independent of any output class \(y_{i}\). The term serves as a scaling factor and can be excluded from calculation due to the assumption that each instance belongs to one and only one class. Thus, Eq. 4 the mathematical formulation of Naive Bayes becomes,

$$\begin{aligned} P(y_{i}\vert x)=\prod _{k=1}^{m} P(x_{k} \vert y_{i}) \cdot P(y_{i}). \end{aligned}$$
(5)

3.3 Proposed Explainability Technique for Graph Neural Topic Models

The GNTM in a topic modelling context generate two outputs, namely,

  1. 1.

    Documents topics distribution

  2. 2.

    Topics words distribution

The proposed approach tries to provide an explanation of the documents topics distribution output of the GNTM. Explainability is achieved by training a local explainable model. Suppose the trained model produces a documents topics distribution similar to the GNTM. In that case, it is possible to say that, a black box model such as the GNTM can be explained by introducing an inherently explainable local model. The overall flow of the proposed approach is as follows.

  1. 1.

    The corpus is first preprocessed.

  2. 2.

    Using the preprocessed corpus, topics words distributions are generated using a GNTM.

  3. 3.

    A vectorized labelled training dataset is generated with the topics words distribution.

  4. 4.

    A local explainable model is built using the labelled training dataset.

This approach builds a local explainable model equivalent to a GNTM. The schematic representation of the proposed explainability technique, highlighting the various steps involved, is depicted in Fig. 1. The proposed technique comprises of two stages: topic modelling using a GNTM and building a local explainable model. In this paper, the function of the local explainable model is carried out using a Naive Bayes classifier.

Fig. 1
figure 1

Schematic representation of proposed local explainability technique for graph neural topic model

Full size image
Fig. 2
figure 2

Training of proposed local explainability technique for graph neural topic models

Full size image
Fig. 3
figure 3

Testing of proposed local explainability technique for graph neural topic models

Full size image

Stage 1: Topic Modelling using Graph Neural Topic Model: The pre-processed training dataset is converted to a dense vector representation of words using Global Vectors for Word Representation (GloVe). The GNTM uses the graph-structured data obtained by converting the vector representations to form the topic models. The topic models contain a list of words associated with different topics. The formation of the topic model, including various pre-processing techniques, is adapted from the literature [22].

Stage 2: Building a Local Explainable Model: A labelled training dataset with words as features and topics as class labels are formed using the topic model generated by the GNTM. The labelled training dataset is vectorized using Bag of Words (BoW). The local explainable model is built by training the model using the labelled vectorized training dataset. The trained local model classifies the new, unseen documents into any one of the predefined topics.

Figure 2 is a pictorial illustration of the training of the proposed explainability technique using the Naive Bayes classifier. From Fig. 2, it can be noted that the vectorized documents is used to train the GNTM. The GNTM provides two outputs in matrices. The first matrix provides the probability distribution of words over topics. The second matrix, a probability distribution of topics over documents, is the individual prediction of GNTM, which the proposed technique explains with the help of a local explainable model. The Naïve Bayes classifier is trained using the probability distribution of words over topics (first matrix). During topic modelling of a new unknown document, the class conditional probability table of the trained Naïve Bayes classifier is used.

Figure 3 illustrates the testing of the proposed topic modelling explainability technique. The test documents are vectorized (BoW) using the trained GNTM. Naïve Bayes classifier acts on these vectorized documents generating the individual prediction by forming the documents- topics probabilities.

figure a

Algorithm 1 Pseudo code of the Local Explainability Technique For Graph Neural Topic Model

Table 1 Notations and definitions of attributes used in Algorithm 1
Full size table

Algorithm 1 gives a pseudo-code of the proposed explainability technique, and the definition of notations used in Algorithm 1 is given in Table 1. Pseudocode implementation of the proposed local explainability technique begins with two inputs, namely training and testing datasets tr, ts, respectively. The training dataset tr is first preprocessed before performing vectorization and forming the graph-structured data. Vectorization is done using GloVe. The graph-structured data is then be applied to the GNTM function \(gnn\_fn\) to form the topic model tm associated with the dataset. Using this topic model, a labelled topic model tmlab is generated, and each record in the labelled topic model contains a specified number of words associated with each topic. The labeled topic model, vectorized using BoW, forms the vectorized labeled topic model denoted as \(vct\_tmlab\). This vectorized representation is used to train the local explainable model, which, in this case, is Naive Bayes. This completes the training of the proposed technique. Testing is done by first performing preprocessing and vectorization using BoW. It should be noted here that the BoW used for vectorization of the testing dataset should align with that in the training phase for consistent performance. The vectorized testing dataset is given to the trained model to form the topics-document distribution. A process flow diagram of Algorithm 1 is illustrated in Fig. 4.

Fig. 4
figure 4

Process flow diagram of Algorithm 1

Full size image

4 Experimental Results and Discussion

The experimental analyses are carried out to establish the similarity between GNTM and the proposed local explainability technique for GNTM. The similarity is evaluated at the corpus level and document level. At the corpus level, similarity is assessed in terms of the Euclidean distance among the normalized average topic mix and topic-wise word cloud. Regarding document level similarity evaluation, the document level percentage of topics matched is calculated. Once the similarity is established, the details of how the explanations are achieved are demonstrated. The similarity is evaluated using three NLP datasets of varying dimensions. Also, the performance of the proposed explainability technique for GNTM is compared with a few baseline topic modelling techniques. The datasets include Reuters-21578 https://www.kaggle.com/datasets/nltkdata/reuters, 20 News Groups (20NG) https://scikit-learn.org/stable/modules/generated/sklearn.datasets.fetch_20newsgroups.html, and Tag My News (TMN) https://www.kaggle.com/datasets/rmisra/news-category-dataset. The dimensional details of the datasets and the train-test split-up are given in Table 2.

Table 2 Specifics of benchmark datasets
Full size table

The experimental analyses are carried out using a laptop with an 11th-generation Intel(R) Core(TM) i5-1135G7 CPU running at 2.40 GHz and 16 GB of RAM. The scripts of the techniques are implemented in Python 3.9.7, and various natural language processing functions are used from the Natural Langauge Toolkit (NLTK) (NLTK 3.8.1) available at https://www.nltk.org/.

4.1 Experimental Results

This Section focuses on the various experiments to evaluate the similarities between GNTM and the proposed local explainability model. Evaluating similarity is a key part of determining if the proposed local explainability model can successfully mimic GNTM. The similarity is evaluated at the corpus and document levels. In corpus-level similarity evaluation, the similarity between the GNTM and local explainability technique for the GNTM is calculated using Euclidean distance among the normalized average topic mix and topic-wise word cloud. At the document level, a similar evaluation is done by assessing the percentage of matched topics and with respect to fidelity between GNTM and the proposed technique. The latter part of the experimentation also expresses an evaluation of GNTM and the local explainable model using a specific test sample from different datasets. This evaluation of similarities will facilitate the development of an explanation for the GNTM, as demonstrated in Sect. 4.2. A comparison with a few baseline topic modelling techniques is also carried out to better position the performance of the proposed technique.

4.1.1 Corpus Level Similarity Evaluation in Terms of Euclidean Distance Among Normalized Average Topic Mix

This Section evaluates similarity by calculating the Euclidean distance between the normalized average topic mix obtained using the GNTM and the proposed local explainability technique.

The document-topics probability distribution for each document is evaluated first and averaged over the entire corpus to obtain the average topic mix. The average topic mix is then normalized using min–max normalization. The similarity is evaluated in terms of the Euclidean distance [51, 52]. Equation (6) gives the mathematical formula to find the Euclidean distance between two points in Euclidean space.

$$\begin{aligned} E_{d}=\sqrt{\left( x_{2}-x_{1} \right) ^{2}+\left( y_{2}-y_{1} \right) ^{2}} \end{aligned}$$
(6)

where \(E_{d}\) represents the Euclidean distance, (\(x_{1}\), \(y_{1}\)) and (\(x_{2}\), \(y_{2}\)) represents the coordinates of two points.The normalized average topic mix obtained using the GNTM and local explainability technique for the GNTM for five model topics represented in two-dimensional Euclidean space using the Tag My News dataset is shown in Fig. 5.

Fig. 5
figure 5

Normalized average topic mix obtained using graph neural topic model and proposed local explainability technique for graph neural topic model of five topics (Tag My News dataset) in two-dimensional Euclidean space

Full size image

In Fig. 5, the x and y-coordinates denote the topic index and normalized average topic probability distribution, respectively. It can be observed that the x-coordinates are the same. Hence, the Euclidean distance given in Eq. (6) is the difference between the y-coordinates, i.e., the normalized average topic probability distribution. Thus, Eq. (6) becomes,

$$\begin{aligned} E_{d}(t) = |y_{nb}(t) - y_{gntm}(t) |\end{aligned}$$
(7)

where t denotes topic index, \(E_{d}(t)\) is the \(t{\textrm{th}}\) topic Euclidean distance, \(y_{nb}(t)\) and \(y_{gntm}(t)\) represents the normalized average topic probability of the \(t{\textrm{th}}\) topic obtained using local explainability technique for GNTM and GNTM respectively.

The Euclidean distance evaluated for five topics of three datasets using Eq. (7) is given in Table 3. From the Table, the Euclidean distance obtained is very close to zero, indicating that the local explainability technique for the GNTM can produce outputs close to that of the GNTM. In fact, topics 3 and 4 in Tag My News, topic 3 in 20 News Groups, and topic 2 in Reuters-21578 are zero (indicated in bold), indicating that both techniques are identical in modelling the respective topics.

Table 3 Euclidean distance between normalized average topic mix obtained using local explainability technique for graph neural topic model and graph neural topic model for different datasets [Best results bolded]
Full size table

4.1.2 Corpus Level Similarity Evaluation in Terms of Topic-Wise Word Cloud

In this Section, the similarity between the topic models obtained using GNTM and the proposed local explainable model is showcased by visualizing it using topic-wise word clouds. Analyzing these topic-wise word clouds makes it possible to identify the prominent words present in each topic obtained using both models. Furthermore, the frequency of words within a particular topic can also be visualized by the size of words in the word cloud. The prominent three words in the topic-wise word cloud prepared over the entire corpus are given the same color format for a straightforward interpretation of similarity. Figure 6 illustrates the topic-wise word cloud for five model topics evaluated using three datasets obtained with the local explainable model and GNTM. Figure 6a, c, e represent the topic-wise word cloud over corpus obtained using GNTM with Tag My News, 20 News Groups, and Reuters datasets, respectively. Similarly, the topic-wise word cloud obtained using the local explainability technique for the GNTM is illustrated in Fig. 6b, d, f.

Considering Fig. 6a, it can be observed that the top three prominent words that constitute topic 0 of the topic model obtained using GNTM are ’world,’ ’games,’ and ’books.’ On comparing these with the prominent words of topic 0 obtained using the proposed local explainability method, as shown in Fig. 6b, it can be noted that they are the same. This indicates that the proposed local explainability method can produce outputs similar to GNTM. This behaviour is also notable in the case of other topics and with all datasets. This indicates that the local explainability technique for GNTM can produce prediction outputs similar to that obtained using GNTM.

Fig. 6
figure 6

Topic-wise word cloud of graph neural topic model and proposed local explainability technique for graph neural topic model for different datasets with five model topics, a Tag My News, b 20 News Groups, c Reuters-21578 (Red, Green, and Blue indicates first, second and third top words)

Full size image

4.1.3 Document Level Similarity Evaluation in Terms of Percentage of Topics Matched

The average document-wise percentage of topics matched is calculated by comparing how the topics are distributed with respect to their probabilities in the documents as obtained using both techniques.

The average document-wise percentage of topics matched is evaluated in two scenarios. The first scenario evaluates the matching by considering whether the top n percent of model topics are present in both outputs. The second scenario considers the presence and the order in which these model topics are present in the results.

The percentage of topics matched is determined for the top 80, 60, and 40 percent of model topics. Initially, the total number of topics in the output of topic modelling techniques is fixed at five. Later, the number of topics increases in steps up to 15, and its effect on matching percentage is studied. The results obtained are shown in Figs. 7 and 8. The red, green, and blue lines indicate the percentage matching obtained using 20NG, Reuters-21578, and TMN datasets.

4.1.3.1 Percentage of Topics Matched Considering only The Presence of The Top n Percent of Model Topics

Here, the top n percent of model topics from the GNTM outputs and the proposed technique are compared without considering their order. Figure 7 depicts the average document-wise percentage of topics matched for different top n percent of model topics. The total number of model topics is varied in the case of different datasets, and its effect on the matching percentage is also illustrated in the figure. Figure 7a–c give the percentage of topics matched for the top 80, 60, and 40 percent model topics, respectively. From the Figure, the following observations can be made. For the top 80 percent of model topics shown in Fig. 7a, it can be noted that the overall percentage of topics matched is above 80 percent. A similar result is also observed for the top 60 percent of model topics. In the case of the top 40 percent of model topics, the average document-wise percentage of topics matched is around 80 percent, except for 12 topics. As a concluding note, in most cases, the portion of topic matched stayed above 80 percent.

Fig. 7
figure 7

Percentage of topics matched (without order) for different top n percent model topics in the case of different benchmark datasets, a top 80 percent model topics, b top 60 percent model topics, c top 40 percent model topics

Full size image
Fig. 8
figure 8

Percentage of topics matched (with order) for different top n percent model topics in the case of different benchmark datasets, a top 80 percent model topics, b top 60 percent model topics, c top 40 percent model topics

Full size image
4.1.3.2 Percentage of Topics Matched Considering The Presence as well as The Order of Top n Percent of Model Topics

In addition to the presence of the top n percent of model topics in outputs obtained using GNTM and the proposed technique, the order in which these topics are present in the output documents is considered. The average document-wise percentage of topics matched applied in different datasets is depicted in Fig. 8. Figure 8a–c illustrate the results obtained for various top n percent model topics matched. When the order of model topics is also considered, the overall percentage of topics matched for five topics is above 80 percent in the case of the top 80, 60, and 40 model topics. As the number of model topics increased, a similarity of around 60 percent is observed.

4.1.4 Document Level Similarity Evaluation in Terms of Fidelity

In this Section, the overall performance of the local explainability technique is evaluated in terms of fidelity using the entire test set of the three datasets. Fidelity, as described in [53, 54], makes sure that the proposed local explainability technique captures how GNTM makes decisions so that the explanations that can be used to understand GNTM are accurate.

The fidelity of the local explainability technique obtained using three datasets is given in Fig. 9. From the Figure, it can be noted that the fidelity of the proposed local explainability technique for GNTM is 88.39\(\%\), 78.9\(\%\) and 83.4\(\%\) with TMN, 20NG and Reuters datasets. This shows that the proposed technique can capture the decision process of GNTM with respectable accuracy.

Fig. 9
figure 9

Fidelity of Proposed Local Explainability Technique for Graph Neural Topic Model using Tag My News, 20 News Group and Reuters-2157 Datasets

Full size image

The following conclusions can be made from all the experimental analyses conducted in this Section.

  • The Euclidean distance between the normalized average topic mix over the corpus obtained using the GNTM and local explainability technique for the GNTM was found close to zero. This indicated that both techniques produced similar document topic probability distributions.

  • The topic-wise word cloud over the entire corpus illustrated the resemblance between the top frequently occurring words in the output obtained using both techniques.

  • Document-wise percentage of topics matched calculated by comparing the topic distribution obtained using both methods indicated a matching percentage of nearly 80 percent when the presence of the top n percent of model topics was considered. Similarly, the proposed technique produced more than 60 percent similarity when the presence and order of top n percent model topics were considered.

  • Document-wise similarity evaluation in terms of fidelity showed that the proposed explainability technique has a fidelity measure of 88.39\(\%\), 78.9\(\%\) and 83.4\(\%\) using TMN, 20NG and Reuters datasets respectively.

The above conclusions indicate that the proposed local explainability technique for the GNTM could mimic the GNTM at respectable levels.

4.1.5 Baseline Comparison of Proposed Local Explainability Technique

In order to better position the performance of the proposed local explainability technique for GNTM, a comparison with baseline topic modelling techniques is carried out. Baseline algorithms include Latent Dirichlet Allocation (LDA) [55, 56], Latent Semantic Analysis (LSA) [57], BERTopic [58], and GNTM. Being an unsupervised technique, the performance of topic modelling techniques is assessed in terms of topic diversity and topic coherence.

Table 4 gives the performance comparison for the proposed explainability technique across five topics using 20NG dataset. The Table shows that the proposed technique’s topic diversity and coherence align with the baseline topic modelling techniques.

An illustration demonstrating the performance of the proposed explainability technique for GNTM across five and fifteen topics using 20NG, TMN and Reuters datasets are also given in Fig. 10. The Figure shows that, as seen in Table 4, the topic diversity and topic coherence of the proposed technique for five and fifteen topics also align with the baseline topic modelling techniques.

Thus, from Table 4 and Fig. 10, it can be observed that,

  1. 1.

    The performance of the proposed explainability technique for GNTM is comparable with the performance of the baseline modelling techniques.

  2. 2.

    Explainability (Demonstrated in Sect. 4.2) is achieved without compromising the performance.

Table 4 Baseline comparison of proposed local explainability technique for GNTM in terms of topic diversity and topic coherence across five topics with 20NG
Full size table
Fig. 10
figure 10

Baseline comparison of the proposed GNTM explainability technique across various topic numbers using different benchmark datasets, a Topic Diversity [20NG], b Topic Coherence [20NG], c Topic Diversity [TMN], d Topic Coherence [TMN], e Topic Diversity [Reuters-2157], f Topic Coherence [Reuters-2157]

Full size image

4.1.6 Evaluation of Graph Neural Topic Model and Proposed Local Explainability Technique Using Diverse Test Samples

In this Section, three test samples from 20NG,TMN, and Reuters-2157 are selected and evaluated using GNTM and local explainability technique for the GNTM.

Fig. 11
figure 11

Sample Test Document from Tag My News, 20 News Group and Reuters-2157 Datasets

Full size image

Topic modelling is carried out with 5 topics, each comprising 15 words. The topic modelling obtained using GNTM trained using the three datasets are given in Table 5.

Table 5 Topic modelling obtained using graph neural topic model for five topic words with Tag My News, 20 News Group, and Reuters-2157 Datasets
Full size table

The three sample test documents selected are presented in Fig. 11a–c. Table 6 presents the probability distribution of topics across the individual selected test samples from three datasets, using the GNTM and the local explainability technique with the GNTM. Based on the data presented in the Table, it is apparent that, according to the probability distribution of topics derived from applying GNTM to the 20NG dataset, Topic 2 is the most prominent, followed by Topic 4. Conversely, Topics 0,1 and 3 demonstrate the least prominence. Upon performing an in-depth analysis, it is evident that the probability distribution of topics derived from the proposed local explainability technique exhibits similar behavior, with Topic 2 emerging as the most prevalent topic and the rest of the topics also following the same order as seen with GNTM. This phenomenon is also seen in the TMN and Reuters-2157 datasets.

Table 6 Probability distribution of topics over test documents of Tag My News, 20 News Group, and Reuters-2157 Datasets using Graph Neural Topic Model and Proposed Local Explainability Technique of Graph Neural Topic Model
Full size table

4.2 Explainability of Graph Neural Topic Model: A Demonstration

The explainability of the proposed technique is demonstrated using a sample test document taken from Tag My News dataset, as shown in Fig. 11a. A GNTM configured with five model topics is trained using the Tag My News dataset. The topic models generated using the GNTM for the Tag My News dataset with a maximum of 5 topic words are shown in Table 5.

A labelled vectorized form of topic models obtained using a GNTM is used to train the Naïve Bayes classifier. The trained Naive Bayes classifier generates a class conditional probability table, and the classifier uses this Table to make predictions on any unknown documents.

The class condition probability table obtained is given in Table 7, with a value of 0.036 if a particular word is present in a topic and a probability value of 0.018 if the word is absent. The class conditional probability table is limited to the first five words (for illustration purposes), as shown in Table 7. It is possible to deduce from Tables 5 and 7 that the word “actor” appears only in Topic 1 and Topic 4. As a result, the probability value for these two topics for the word is 0.036, and the probability value for other topics is 0.018. The formation of the class conditional probability table of the Naïve Bayes classifier concludes the training.

Table 7 Class conditional probability table of first five words in the topic model of the proposed local explainability technique using Tag My News dataset
Full size table

After the formation of topic model, testing of the sample test document is carried out. Let the test document after pre-processing be “tweeters school amazingly dumb Donald Trump over special council typo politics the special council is a unit of the space force Lee Moran”. The vectorized test document after pre-processing is given in Table 8.

Table 8 Vectorized test document
Full size table

Table 8 indicates the presence/absence of words in the test document with respect to the words present in the topic model given in Table 5. If a particular word is present, the value is 1; otherwise, it is treated as 0.

The individual prediction, i.e., the probability distribution of topics over the test document obtained using the trained Naïve Bayes classifier for Tag My News dataset [Extracted from Table 6 for ease of interpretation] is given in Table 9.

Table 9 Probability distribution of topics over test document from Tag My News dataset using local explainability technique for graph neural topic model
Full size table

The mathematical formulation of the Naïve Bayes classifier given in Eq. (5) can be rewritten to obtain the prediction probability for the sample test document in the topic modelling context as given in Eq. (8).

$$\begin{aligned} P(T_{k}\vert W)=\prod _{i=1}^{N} P(W_{i}\vert T_{k})\times P(T_{k}) \end{aligned}$$
(8)

where \(T_k\) represents the \(k{\textrm{th}}\) topic, W represents the set of words from the test document which is present in the topic model, i is the index of the words in W, N is the total number of words in W, \(W_i\) gives the \(i{\textrm{th}}\) word in W, \(P(T_k\vert W)\) represents the probability of the \(k{\textrm{th}}\) topic given all the words in W, \(P(W_i\vert T_k)\) represents the probability of \(i{\textrm{th}}\) word in W given \(k{\textrm{th}}\) topic, \(P(T_k)\) is the probability of \(k{\textrm{th}}\) topic among all topics.

As Naïve Bayes is an interpretable model, it is possible to calculate the probability distribution of topics over the test document as given in Table 9 using the basic mathematical formulation of Naïve Bayes given in Eq. (8). \(P(W_i \vert T_k)\) which is the probability of \(i{\textrm{th}}\) word in the pre-processing test document given \(k{\textrm{th}}\) topic is obtained with the help of the class conditional probability table given in Table 7 and the vectorized test document represented in Table 8.

Suppose the probability of zeroth topic in the document topic probability matrix is calculated then, \(\hbox {k}=0\);

Therefore, Eq. (8) implies

$$\begin{aligned} P(T_{0}\vert W)=\prod _{i=1}^{N} P(W_{i}\vert T_{0})\times P(T_{0})=0.25 \end{aligned}$$
(9)

The value obtained in Eq. (9) is the same as obtained by the Naïve Bayes classifier, as shown in Table 9. The details of the calculation of \(P(T_0\vert W)\) are given in Fig. 12. Similarly, all other probabilities for topics 1, 2, 3, and 4 can be calculated. On observing Table 9, it can be noted that the test document is confined to topics 0, 2, and 4. The test document probability distribution obtained using the GNTM is given in Table 10 (Extracted from Table 6 for ease of interpretation.), which also has the same relationship among topics as in Table 9.

Fig. 12
figure 12

Calculation of posterior probability of topic 0

Full size image
Table 10 Probability distribution of topics over test document from Tag My News dataset using graph neural topic model
Full size table

The demonstration presented in this Section shows how explanations of individual predictions of GNTM can be made using the proposed technique.

4.3 Discussion

4.3.1 Theoretical Implications

Development of the proposed local explainability technique as a means of explanation for GNTM has significant theoretical ramifications within the field of XAI. Theoretical significance comes from its potential to convert the opaque character of GNTM into a more easily understood framework. The ability of the proposed technique to comprehend and clarify the relationships between topics and documents within the GNTM framework presents novel opportunities for theoretical investigation in graph-based topic modeling. These findings can potentially be applied across diverse domains, including natural language processing, social network analysis, and recommendation systems.

Another theoretical implication of the proposed technique is that all the theories built on explainable models can be applied to black-box models, and their validity can be evaluated. This opens up a wide array of opportunities for understanding and harnessing the potential of black box models in various applications while maintaining a certain level of interpretability and transparency.

4.3.2 Practical Implications

The proposed explainability technique allows researchers to understand the factors that influence predictions. The achieved level of transparency provides the ability to make well-informed choices and effectively address any issues related to model behavior. Consequently, this generates more dependable and practical insights in various applications.

In the context of health care applications such as disease prediction, the proposed explainability technique enables the model to reveal the essential traits and circumstances present in a patient’s medical history that contributed to the prediction of a given illness or medical condition. The information systematically emphasizes relevant patient data, including symptoms, laboratory test results, medical records, and underlying risk factors. The degree of clarity at hand provides healthcare providers with diverse skills. First and foremost, this enables healthcare specialists to verify the model’s suggestions by comparing them with their clinical experience and the patient’s real health condition. Furthermore, equipped with a comprehensive understanding of the factors that influence the predictions made by the model, healthcare practitioners can develop customized treatment strategies to meet the particular requirements of individual patients, including distinct risk factors and medical backgrounds taken into account. Finally, within the field of medical research, this practice not only facilitates the discovery of novel ideas but also enhances the refinement of research procedures, thus propelling the progress of the healthcare and medical science domains.

When it comes to e-commerce applications such as loan dispersal or credit card approval also, explainability of the model is crucial. This ensures that applicants are provided with transparent and understandable justifications for the results of their applications, whether they are approved or denied. Proper justifications enable financial institutions to ensure that their decisions are based on objective criteria, reducing the potential for bias or discriminatory practices. This will also enable the applicant to understand the factors that influence their application status, such as credit score, income, or outstanding debts, and also promote the applicant to maintain good financial discipline.

4.3.3 Limitations

When it comes to providing explanations in a more human-friendly symbolic manner, the proposed technique faces significant constraints mainly due to its probabilistic nature. The proposed method just tries to mimic and cannot contribute towards interpretability.

5 Conclusions and Future Works

Graph neural topic model is an effective neural topic modelling technique capable of capturing topics’ hierarchical structure and modeling the complex relationships between words in a document. However, the complex black-box nature of the GNTM makes it less applicable to some applications. This paper presented a local explainability technique for explaining the documents topics probability distributions output of the GNTM. The explanation was provided by learning a local interpretable model such as a Naive Bayes classifier. Extensive experimental analyses using three datasets clearly indicated the close resemblance between the individual predictions obtained using the GNTM and the local explainable model. The performance of the proposed technique was also compared with a few baseline topic modelling techniques. The paper also presented a demonstration of how explanations are formed.

In future endeavours, the attainment of explainability can be tried using other explainable models such as random forests, decision trees, etc.