1 Introduction

Online learning has gradually become a very important way of learning, especially as the coronavirus disease (COVID-19) pandemic has forced higher-education activities in many countries around the world to utilize online learning or a combination of face-to-face learning and online learning (García-Morales et al., 2021). This sudden change means online learning is becoming a regular education model (Carolan et al., 2020). Learners’ conceptual knowledge and academic achievement in online learning are often used as primary learning outcomes (Wei et al., 2021). These learning outcomes are assessed most often by tests, exams, etc. (Deng et al., 2019). Researchers have pointed out that understanding students’ learning only by scores in most cases can only reveal the retelling of knowledge, which cannot deeply reveal the learners’ learning status (Cheng et al., 2013). However, having a real comprehensive understanding of learners’ learning status is crucial for educators to implement effective teaching design (Chen et al., 2021). Knowledge structures represent the organization and integration of concepts in personal memory and can reflect the form of knowledge organization of learners (Jonassen et al., 1993). Knowledge structure measures can assess learners’ conceptual understanding (Weinerth et al., 2014), identify knowledge loss and misunderstanding (Kim et al., 2019), predict classroom learning and achievement (Lopez et al., 2014), distinguish learners’ learning style (Hay, 2007), and describe cognitive status changes during problem solving (Hung & Lin, 2015). Recent studies have also used knowledge structures to evaluate learners’ higher-order thinking abilities, such as systematic thinking (Khajeloo & Siegel, 2022). Therefore, understanding knowledge structures is considered a more comprehensive way of understanding learner cognition (Kim et al., 2019). Knowledge structures can be elicited and analyzed through a variety of visualization methods (Clariana et al., 2014; Eseryel et al., 2013; Jonassen et al., 1993; Tang & Clariana, 2017). Concept maps place more emphasis on the extraction of concepts and the establishment of knowledge relations, which is considered a suitable representation approach of knowledge structures in the actual teaching process (Jonassen & Marra, 1994). Computer-supported concept maps provide a convenient means to construct maps while recording the resulting concept maps through standardized data (Farrokhnia et al., 2019). This visualization method of knowledge structures provides the basis for the automated analysis and evaluation of knowledge structures (Weinerth et al., 2014). Current learning analysis techniques are widely used in online learning to improve teaching, learning, and educational decision-making (Brown et al., 2022). Researchers use the data mining approach to analyze learning behaviors (Tlili et al., 2021) and interactive discourse (Liu et al., 2022) to find education and learning rules. However, the data mining approach is rarely used to investigate online learners’ knowledge structures.

To fill these research gaps, we used a self-developed online learning platform that supported concept mapping to collect online learners’ knowledge structures and identified knowledge structure patterns through the data mining approach. Our study explored an automatic and fast method to analyze knowledge structures and find identifiable patterns of knowledge structure in online learners. The findings provided support for educators to better understand the connotation of learners’ knowledge structures, and help them conduct online instructional design and evaluation.

2 Literature review

2.1 Knowledge structures as a way to understand learning

Knowledge structures are a collection of interrelated facts about specific topics within a given domain, also known as structural knowledge or cognitive structure, which provide connections between procedural and declarative knowledge and represent learners’ ability to apply what they know (Clariana et al., 2013; Jonassen et al., 1993). Meaningful learning theory defines learning as occurring when individuals absorb new information in their preexisting knowledge structures and when learners choose to link new information to prior knowledge (Ausubel, 1963). Moreover, meaningful learning also leads to a more complex knowledge structure (Novak, 2010). Knowledge structures have been found to measure students’ understanding or conceptual level in teaching research based on many subjects, such as science education (Schaal et al., 2010), biology (Wadouh et al., 2014), mathematics (Gogus, 2013), physics (Koponen & Pehkonen, 2010), and medicine (Hung & Lin, 2015). Previous studies have found that there are significant differences between the knowledge structures of experts and those of novices (Chi et al., 1981). For example, Gogus (2013) found differences in the conceptualization of complex mathematical problems between novices and experts by comparing their knowledge structures. Koponen and Pehkonen (2008) found that the knowledge structures of experts in physics were consistent and hierarchical, while knowledge structures of novices were fractured. The knowledge structures of experts were more closely integrated than those of novices (Ifenthaler et al., 2011). Therefore, when defining learners’ ability, the integrity and good connection of knowledge structures are important considerations (Wadouh et al., 2014). As a more reliable method to determine learning results, knowledge structures can not only simply reflect the recognition and recall of concepts but also provide higher-order learning evidence to reflect learners’ understanding of complex fields (Spector, 2006). Since mining and analyzing learners’ knowledge structures is considered to be an important understanding of learners’ learning quality (Lopez et al., 2014), it is of great significance to study the knowledge structures of online learners to help online learning participants (including teachers and students) deeply understand the level of online learning.

2.2 Using concept maps to investigate knowledge structures

Knowledge structures can be elicited and analyzed by card classification, word association, text analysis, concept maps, complex semantic network tools, and other methods (Clariana et al., 2014; Eseryel et al., 2013; Tang & Clariana, 2017). Although there have been different attempts made to elicit knowledge structures, the process of self-constructing concept maps is considered to be a more straightforward approach (Jonassen & Marra, 1994). This approach can not only visualize learners’ knowledge structures but also effectively promote learners’ ability improvement and learning performance, including reading ability (Clariana et al., 2014) and writing ability (Liu, 2011). Concept maps were developed by Novak and Gowin (1984) based on Ausubel’s relevance theory of meaningful learning, and the key elements of concept maps include nodes, relationships, and propositions that are semantic understandings formed by nodes and relationships. The concept map can visually represent the knowledge structures of learners through its key elements (Jonassen et al., 1993). Constructivism theory mentions that the process in which learners try to solve their conflicts in how to represent and mark knowledge is called internal negotiation, while in the self-constructing concept map, learners achieve internal negotiation by creating nodes, describing node relations, and constantly reviewing node relations (Jonassen & Marra, 1994). The process of self-constructing concept maps allows teachers the opportunity to observe the extensive and integrated conceptual knowledge of students, which is difficult to observe through traditional assessment forms (such as multiple-choice questions) (Walker & King, 2003). Therefore, the process of self-constructing concept maps is a suitable method for eliciting learners’ knowledge structures in online learning, realizing the representation of learners’ knowledge structure, and including the learning results of learners’ deep learning characteristics (Jonassen et al., 1993).

In the research of using concept maps to analyze knowledge structure, researchers have focused on how to quantitatively analyze learners’ concept maps. The traditional concept map evaluation method is mainly evaluated by raters based on the level and quality of propositions in the concept map (Watson et al., 2016). With the development of the automatic evaluation of concept maps, researchers can calculate the structural indicators of maps based on graph theory and network theory, including the number of nodes, the number of layers (Novak & Gowin, 1984), overall width (Hao et al., 2010), maximum layer width (Jonassen et al., 1993), etc. In addition, Clariana et al. (2013) used the centrality index of concept maps to measure the knowledge structure of learners in their study of computer-supported collaborative problem solving. Kapuza et al. (2020) proposed three measures based on the basic structure index of the concept map to analyze the concept map generated during the statistical data analysis course to understand the knowledge structure of learners.

However, Kinchin et al. (2000) warned that quantitative evaluation of concept maps should be carefully used because quantitative analysis ignores the structure of wrong concepts, and these mistakes convey the learning development trend of learners in the future (Kinchin, 2014). Therefore, the authors proposed a qualitative analysis method of map morphology to describe the concept map as three modes, namely, spoke, chain, and network (Kinchin et al., 2000), thereby further associating the morphological evolution of the concept map with the learning mode (Hay et al., 2008). Some researchers have further conducted a qualitative analysis of knowledge structure based on Kinchin’s analysis method. Koponen and Pehkonen (2008) reconstructed concept maps constructed by learners into two topologies, namely, network and tree, for analysis and showed that the structure and topological features of concept maps have the ability to distinguish between novices and experts. Hung and Lin (2015) found three types of concept maps, namely, isolated mapping, departmental mapping, and integrated mapping; they also found that more integrated mapping occurs among learners who used intervention strategies in problem-solving teaching by the empirical study. The morphological analysis of the map provides a unique perspective to compensate for the lost learning information of the quantitative evaluation map and provides a new idea for the analysis of learners’ knowledge structures in online learning. However, the qualitative analysis of map forms requires great energy from teachers and it is difficult to promote in online learning. Based on previous quantitative research on concept maps, our study combined the quantitative analysis method with the morphological analysis method of map structure to explore a data mining method to analyze online learners’ knowledge structures. The study identified online learners’ knowledge structure patterns and further investigated the relationship between knowledge structure patterns and online learning achievement.

The following three questions are mainly investigated:

  1. (1)

    Do learners’ knowledge structures have identifiable patterns in online learning?

  2. (2)

    How can different types of online learners be identified from the distribution of their knowledge structure patterns?

  3. (3)

    What are the differences in the online learning achievement of the different types of online learners?

3 Methods

This study aims to identify meaningful patterns of knowledge structure in online learning by analyzing the features of learners’ knowledge structures, and to further discover the learner types with different patterns of knowledge structures. For this purpose, we adopted clustering to automatically identify similar learners. Clustering is an unsupervised data mining algorithm that divides similar samples into the same cluster. In other words, the algorithm can uncover unique learner types with similar knowledge structure patterns. Besides, we used a statistical analysis method to compare the learning achievements of different types of online learners. By doing so, it was expected to provide insights about the relationship between knowledge structure and learning achievements in online learning environments.

3.1 Learning context

In this study, a flipped classroom was mainly adopted as the learning context. The flipped classroom is a common blended learning model, in which learners should complete online learning tasks before entering the classroom (Devi et al., 2021). Specifically, due to the impact of the COVID-19 epidemic, the learning model was implemented in a computer programming course at a university in China. More specifically, the blended learning model was supported by an online platform and an online conference system.

The course contains 14 lessons. For each lesson, the teacher provided learning materials, which covered basic knowledge through the online platform. The students were asked to complete these materials before the face-to-face sessions. Subsequently, the teacher explained the focus of the course in the face-to-face sessions. Due to the impact of COVID-19, the face-to-face sessions were organized through the Tencent Conference system in this course. Each lecture was given in two consecutive sessions, totaling 90 min. The students participated in teaching the course once a week.

3.2 Participants

Thirty-six students participated in this course, including 15 males and 21 females. They are freshmen majoring in data science and big data technology. The average age of these students was 18 years old. They had not taken other programming-related courses before participating in this course and thus were considered novice learners in the field. Prior to attending this course, all the participants had no experience in concept mapping, so they were considered unbiased in the learning instruments.

3.3 Procedure

All of the participants completed one lesson of learning per week from week 1 to week 14. The specific learning process weekly was introduced in session 3.1. The students also received training on the use of the online learning system and concept mapping in week 1. The training about concept mapping used the method proposed by Roessger et al. (2018) and mainly focused on how to extract relationships between concepts. In addition, all students participated in the exam through an additional online examination system during the middle and the end of the course. The midterm exam was held in week 9, while the final exam was delayed until three months after the course due to COVID-19.

3.4 Concept map task

The knowledge structure is hidden in the human mind and cannot be observed directly, but it can be represented through the concept map (Jonassen & Marra, 1994). Studies have suggested that low-orientation concept map tasks can better express the knowledge structures of learners (Cañas et al., 2012). A low-orientation concept map task means that someone should construct a concept map without any concept, structure, or relationship provided in advance (Ruiz-Primo, 2000). The study used a low-orientation concept map task to extract learners’ knowledge structures. All the participants were asked to construct a concept map to summarize the knowledge in the unit after completing online learning.

3.5 Online learning platform

In this study, an online learning platform called “Xiaoya” was developed by our research team to support online learning. The system provides the basic functions of a learning management system. More specifically, the main functions of the platform include learning materials, quizzes, discussions, and notes. Through the notes function, students are able to generate document notes and concept maps. The concept maps constructed by students can include concepts, relationships, and cross-links, as shown in Fig. 1. All the concept maps constructed by the learning platform are recorded in a standard format.

Fig. 1
figure 1

The interface of constructing concept maps in Xiaoya

Full size image

In our study, the teacher constructed a virtual curriculum space and provided learners with various forms of learning materials, including videos and slides. Besides, the teacher assigned concept map tasks as the questions in the quiz function. The students who took the course had to complete the materials, then construct concept maps through notes and upload them in the quiz function (Fig. 2).

Fig. 2
figure 2

Course space in the Xiaoya online learning platform

Full size image

3.6 Data collection

The researchers collected the concept map artifacts (N = 359) that were constructed by the 36 students from week 2 to week 12. Because the first lesson was only an introduction and lesson 13 and 14 were optional, the concept map artifacts were excluded for these three weeks. These concept map artifacts are recorded in JavaScript object symbol format through the Xiaoya online learning platform. We used Python to parse the concept map artifact in the JSON format derived from the Xiaoya platform, and each concept map artifact was reconstructed into a directed network topology (Fig. 3). The directed network topology retained all structural features in the original concept map artifact, which was used as the learner knowledge structure representation. In addition, we also collected midterm and final exam scores as learning achievements for online learners.

Fig. 3
figure 3

Constructing the directed topological network from the concept map as online learners’ knowledge structures. Note. (i) Original learners’ concept map. (ii) Reconstructed map (the content information of the concept map is removed, and only the structural features are retained)

Full size image

3.7 Data analysis

Based on the three research questions posed in our study, the data analysis process (Fig. 4) was performed as follows.

Fig. 4
figure 4

Data analysis process. Note: KS patterns represent knowledge structure patterns

Full size image

To answer research question 1, the knowledge structure patterns of online learners were identified by a K-means clustering algorithm. The algorithm is an iterative solution clustering analysis algorithm that is widely used in educational data mining research (Dutt et al., 2015). Graph theory has been used in many studies on concept map analysis (Ifenthaler et al., 2011). This research summarizes the structural indicators proposed in previous concept map research and identifies 19 structural indicators (Table 1) as feature vectors for cluster analysis based on graph theory. Before calculating the indicators, the teacher checked the artifacts, and none were off-task. Besides, the topic of the concept map task may affect the scale of the map, while the maps collected in this study were from 11 different topics. Therefore, we standardized the structural metrics of the maps with standard concept maps provided by the teacher to eliminate this effect. In addition, the sum of square error (SSE) was used to determine the optimal K-clustering value (Nainggolan et al., 2019), and the Calinski-Harabaz (CH) index was used to determine the best clustering results (Rajabi et al., 2019). Both the indicators’ calculation of concept maps and cluster analysis were executed automatically in Python programming language. In addition, the frequency and percentage of each knowledge structure pattern were calculated for the study semester, and the distribution of different kinds of knowledge structure patterns over time throughout the semester (12 weeks) was visualized.

Table 1 Structural indicators of concept maps
Full size table

To answer research question 2, we counted the frequency of different types of knowledge structure patterns identified by research question 1 during the semester (12 weeks) as clustering features. The K-means clustering analysis method was used to discover different types of online learners. Furthermore, the distribution of patterns of different types of online learners and their changes from week 2 to week 12 were also visualized.

To answer research question 3, we compared the learning achievements among the different types of online learners discovered in question 2. Due to the small number of samples in this study, a non-parametric test with the Mann-Whitney U test was taken.

4 Results

4.1 RQ1: Do learners’ knowledge structures have recognizable patterns in online learning?

The concept map artifacts constructed by online learners from week 2 to week 12 (N = 359) were analyzed by the K-means algorithm. The optimal clustering results showed that there were three clusters in the directed network topology of the concept map, namely, Cluster_0, Cluster_1, and Cluster_2. The descriptive statistics of each category are shown in Table 2. Among them, Cluster_0 is the largest with 225 copies, accounting for 62.7% of the total number of submitted maps; Cluster_1 is the second largest with 106 copies, accounting for 29.5% of the total number of submitted maps; and Cluster_2 is the smallest, with 28 copies, accounting for 7.8% of the total number of submitted maps. In terms of structural indicators from V1 to V18, the three categories of Cluster_0, Cluster_1, and Cluster_2 increase in sequence. For the average value of V10, Cluster_0, Cluster_1, and Cluster_2 decrease in order because V10 and V9 have a reciprocal relationship. In terms of indicator V19, there is little difference between Cluster_1 and Cluster_2, which are both larger than Cluster_0. Overall, Cluster_0, Cluster_1, and Cluster_2 increase successively in structural complexity.

Table 2 The statistical description of structure indicators of three categories of concept maps
Full size table

The K-means algorithm uses the centroid of each class to describe the cluster, but the centroid is not necessarily the location of the sample. Therefore, this research calculated the sample closest to the centroid as the representative of the cluster, reconstructed the corresponding concept map according to the reconstruction rules, and then visualized it for morphological analysis. The topology of the directed network reconstructed by Cluster_0, Cluster_1, and Cluster_2 is shown in Fig. 5(i)-(iii).

Fig. 5
figure 5

Directed network topology of representative concept maps of three clusters. Note. (i) Cluster_0: spoke pattern; (ii) Cluster_1: small-network pattern; (iii) Cluster_2: large-network pattern

Full size image

Morphological analysis of the three directed topology network visualization legends found the following. First, from the scale of the directed network topology, Cluster_2, Cluster_1, and Cluster_0 decreased in sequence. Second, from the perspective of the type of directed network topology, there were cross-links (red link) in both Cluster_1 and Cluster_2. Because of the existence of cross-links, both groups belong to the network structure. However, there was no cross-link in Cluster_0, which was essentially a tree-like structure showing a radial shape. Third, from the perspective of the aggregation of branches, nodes in the network structure gather into communities through connections, and cross-links connect small communities into larger communities. Since there were more cross-links in Cluster_2 than in Cluster_1, small communities formed larger communities due to cross-links, and the community sizes of nodes in Cluster_0, Cluster_1, and Cluster_2 increased sequentially.

Based on the descriptive statistical analysis and morphological analysis of the three categories, Cluster_0, Cluster_1, and Cluster_2 were three knowledge structure patterns that gradually increased in scale. Cluster_0 represented the tree-like structure, while Cluster_1 and Cluster_2 represented the networked structure. Therefore, this research named the three categories of knowledge structure patterns, i.e., Cluster_0, Cluster_1, and Cluster_2, the spoke pattern, the small-network pattern, and the large-network pattern, respectively.

4.2 RQ2: How can different types of online learners be identified from the distribution of their knowledge structure patterns?

The knowledge structures of learners reflect the organizational form of domain knowledge in their minds, which is not immutable (Ifenthaler et al., 2011). Thus, it is of great significance to analyze and study the changes in learners’ knowledge structure patterns over a long period of time to reveal their learning (Jonassen et al., 1993). Research question 2 further used the frequency of learners’ occurrence of different structural patterns over the course of the study semester as a description of learners’ characteristics and again discovered different types of online learners through cluster analysis. Since not all the learners who participated in the experiment completed the 11 concept map tasks, we added a nonpattern to describe a situation in which a concept map was not submitted. Finally, we constructed a feature vector containing four dimensions (i.e., statistical frequency in non, spoke, small-network, and large-network) and used the K-means algorithm to conduct clustering analysis on 36 learners.

The optimal clustering results showed that there were two clusters, namely, Cluster_a and Cluster_b. The descriptive statistics of each cluster are shown in Table 3. In Cluster_a, the mean frequency of learners’ occurrence of the four dimensions was non (n = 1.75), spoke (n = 7.69), small-network (n = 1.50), and large-network (n = 0.06). In Cluster_b, the mean frequency of learners’ occurrence of the four dimensions was non (n = 0.45), spoke (n = 4.90), small-network (n = 4.30), and large-network (n = 1.35). We visualized the distribution of online learners in Cluster_a and Cluster_b in 4 dimensions (Fig. 6). There were more large-network patterns and small-network patterns in Cluster_b than in Cluster_a, while there were more spoke patterns and non patterns in Cluster_a than in Cluster_b. We defined Cluster_a as simple knowledge structure online learners and Cluster_b as complex knowledge structure online learners, thereby combining the characteristics of the two clusters in the statistical analysis results and visual analysis results.

Table 3 The statistical description of four patterns distributions of two learners’ clusters
Full size table
Fig. 6
figure 6

Patterns’ distribution histogram of two learner clusters

Full size image

To further analyze the characteristics of the knowledge structures of the two types of online learners changing over time, we visualized the distribution changes of the knowledge structures of the two types of learners from Week 2 to Week 12 (Fig. 7). For simple knowledge structure online learners, the small-network pattern appeared from Week 4, and the large-network pattern appeared from Week 10. Moreover, the total number of small-network and large-network patterns was relatively high in Weeks 5, 6, 10, and 11. In addition, the nonpattern appeared in simple knowledge structure online learners every week except Week 6, and nonpattern counts increased in Week 5, Week 8, Week 10, and Week 12. For complex knowledge structure online learners, the small-network pattern appeared from Week 2, and the large network pattern appeared from Week 5. Moreover, the total number of small network and large network patterns showed two continuous and regular evolves from Week 2 to Week 6 and from Week 8 to Week 11. Similar to simple knowledge structure online learners, complex knowledge structure online learners showed the peak of small network and large network patterns at Weeks 5, 6, 10, and 11.

Fig. 7
figure 7

Patterns’ distribution histogram of two learner clusters by weeks

Full size image

4.3 RQ3: What are the differences in the online learning achievement of the different types of online learners?

The Mann-Whitney U test results showed in Table 4. In terms of midterm scores, the complex type learners scored significantly higher grades than the simple type learners (P = 0.03). Meanwhile, in terms of final scores, the complex group still had higher scores than the simple group, but the difference was not significant (P = 0.46).

Table 4 Mann-Whitney U test results of online learning achievement between two types of online learners
Full size table

5 Findings and discussion

Based on graph theory, this study explored a new method of analyzing the knowledge structure of online learners through data mining by using structural indicators of maps as characteristics. This research identified knowledge structure patterns of online learners through a K-means clustering algorithm and revealed the morphological characteristics of online learners’ knowledge structure. One of the main findings of the research was the identification of three knowledge structure patterns of online learners, namely, spoke patterns, small network patterns, and large network patterns. Statistical analysis and morphological analysis were used to further explore the characteristics of the three structural patterns. The results showed that, first, the scale of the three patterns increased in turn. Second, the spoke pattern did not have cross-links and represented the tree structure, while the small network pattern and the large network pattern both contained cross-links and represented the network structure. Third, the difference in cross-links affected the degree of node aggregation in the three patterns. As the number of cross-links among spoke patterns, small network patterns, and large network patterns increased, the scale of node aggregation in knowledge structure patterns also increased. The structural patterns automatically identified by the data mining method in this research were basically consistent with the concept map structural patterns found by the previous qualitative analysis. Kinchin et al. (2000) first proposed three forms of spoke, chain, and net using the qualitative analysis of concept maps. In the study of Kinchin et al. (2000), a spoke was a tree structure with a single level in which many concepts were directly connected to the central concept, and a chain was a linear sequence of understanding in which each concept was only connected to the concept above or below, and the net was a highly integrated and hierarchical network. Hung and Lin (2015) further divided map morphology into isolated mapping, departmental mapping, and integrated mapping when applying map morphology to the study of knowledge structure in problem-solving learning. In the isolated mapping, several single concepts were linked to the core concepts. In the departmental mapping, several single concepts were combined into conceptual units, but no network was formed. In the integrated mapping, the whole map formed a network, including not only units for concept formation but also horizontal connections between units. In the qualitative analysis of map morphology, the connectivity of map structure is considered to be an important indicator (Kinchin et al., 2005), which has been verified in the identification of knowledge structure patterns using data mining methods.

This study had an additional finding based on the identification of three online learners’ knowledge structure patterns. In the preclass learning in flipped classrooms without additional teaching design, most learners showed the knowledge structure of spoke patterns, while only a few learners showed large-network patterns. In fact, this distribution pattern was consistent with the pattern found in the previous qualitative evaluations of map morphology. The measurement of knowledge structure usually includes the number of nodes in the structure and the relationship between nodes (Clariana et al., 2013). The relationship between nodes promotes the formation of coherent networks with isolated knowledge (Koponen & Pehkonen, 2010). Novices present spoke patterns of knowledge structures so that new information can be added to existing knowledge structures in a timely manner (Kinchin et al., 2000). Domain experts tend to be more aggregated than novices in terms of knowledge structures (Ifenthaler et al., 2011). The knowledge structure of network attributes often reflects learners’ good understanding of knowledge (Joseph et al., 2017). The knowledge structure collected in this research came from the preclass learning of learners in a flipped classroom. Before participating in the experiment, the learners had not received any knowledge learning experience related to this course and were thus considered novices in this knowledge field. The distribution characteristics of knowledge structure patterns found in the research were in line with the novice status of learners and reflected the real situation that learners were not well prepared in the preclass learning of flipped classrooms without especially teaching intervention. In research on teaching strategies based on problem solving, Hung et al. found that teaching intervention strategies promoted the evolution of learners’ knowledge structures from isolated mapping to integrated mapping, and the evolution of knowledge structure reflected the differentiation and integration of learners’ concepts (Hung & Lin, 2015). Therefore, teachers should attach importance to the preclass teaching design of flipped classrooms because the potential benefits of the flipped classroom may be weakened due to learners’ insufficient preclass preparation (Chuang et al., 2018).

Another main finding of the study was to identify two types of online learners with different distributions among the three identified online knowledge structure patterns. We found some common characteristics and differences worthy of further discussion by comparing and analyzing the performance of different types of knowledge structure pattern distributions between the two types of learners. Compared with the simple knowledge structure online learners, the complex knowledge structure online learners appeared with large network patterns and small network patterns earlier, and there were two-periodic evolution increases in knowledge structure complexity from Week 2 to Week 6 and from Week 8 to Week 12. The patterns transformed from spoke to net include the development of the learner’s learning, and the net patterns represent a high level of understanding of the topic (Kinchin et al., 2000). The development of knowledge structure reflects changes in individual knowledge and understanding and can be used as evidence of meaningful learning (Jonassen et al., 1993), which can distinguish between deep learning and shallow learning in the learning process of learners (Hay, 2007). For the complex knowledge structure, online learners with gradual improvement regarding the connectivity of knowledge structures with the progress of learning evidence the development of knowledge structure patterns from spoke patterns to large network patterns. The increased connectivity in knowledge structure during learning reflects the occurrence of meaningful learning by learners (Joseph et al., 2017). In addition, the two types of online learners simultaneously showed relatively more complex knowledge structures in Weeks 5, 6, 10, and 11. After reviewing the teaching content, it was found that the above four teaching units belong to the teaching focus of this course and contain more knowledge points, which may lead to the main reason for the improvement of the complexity of the knowledge structure patterns of the two types of online learners.

A comparison of the learning achievement of the two types of learners found that the complex type learners had outperformed the counterpart in both the midterm and final exams. However, this difference was not significant in the final exam scores. One possible explanation was that the final exam did not fully reflect the learner’s knowledge at the end of the course due to the delay in the exam. The performance of some students might be interfered by other factors. Even so, in general, the results of the learning achievement analysis showed that complex type learners achieved better grades than simple type learners. Through meaningful learning, the learner completes the integration of new knowledge with prior knowledge (Novak, 2010), forming a richer and more aggregated knowledge structure (Ifenthaler et al., 2011). The differences in learning achievement between the two types of learners were consistent with the differences in the distribution of knowledge structure patterns. Consistency findings also support the validity of automatic analysis of knowledge structures through this data mining approach.

6 Conclusion and limitations

Concept maps constructed by learners represent their knowledge structures and reflect learners’ deep understanding of knowledge (Jonassen & Marra, 1994). Analyzing these knowledge structures to better understand learners’ learning can promote teachers’ teaching transformation from content to understanding (Kinchin et al., 2000). However, online learners’ knowledge structures have not been widely analyzed; there is especially a lack of analysis using the data mining method. This study explored a data mining method to analyze the knowledge structures of learners and realized the automatic analysis of the morphological characteristics of knowledge structures through a clustering algorithm. First, three online learners’ knowledge structure patterns were identified, and the distribution characteristics of these three knowledge structure patterns were found in the pre-class of the flipped classroom without any special teaching interventions. Second, based on the identified knowledge structure patterns of online learners, two types of learners, namely, complex knowledge structures and simple knowledge structures, were further discovered based on the long-term changes in the knowledge structure patterns of learners. Finally, through statistical analysis, it was verified that complex knowledge structure online learners had better learning achievement than simple knowledge structure online learners.

Different learning strategies may lead to different knowledge structures in a regular format (Kim et al., 2019), and observing the morphological differences in knowledge structures provides us with a unique perspective with which to observe the changes in learners’ online learning to measure whether learning strategies truly produce effects. Generally, our study proposed a new method for analyzing the knowledge structure patterns of online learners by data mining and initially explores the correlation between knowledge structure patterns and learning achievements. This innovative analysis approach provided a convenient way for educators in online teaching to understand the knowledge structure of students. Furthermore, the findings in this study on the distribution of knowledge structure patterns in flipped classroom learning context provided empirical evidence for the need of pre-class instructional design.

Although this research preliminarily explored a new means of using data analysis to analyze the knowledge structure of online learners, it still had some limitations. First, this research did not distinguish semantic correctness when analyzing the structural features of concept maps. This is mainly because our study aims to study the online knowledge structure from the perspective of structural features. Previous studies emphasized that invalid concepts in concept maps had also revealed learners’ thinking processes (Kinchin et al., 2000), and concept map assessment studies focused on accuracy had been criticized for ignoring important details (Kinchin, 2014). Moreover, before cluster analysis, the teacher checked all the concept maps to ensure none were off-task. For these reasons, our study can still provide valid and reliable findings from the perspective of morphological features of knowledge structures. In future studies, we can further evaluate the accuracy of concept maps through natural language processing, and compare the impact of accuracy on understanding the knowledge structure. Second, the online learning context studied in this paper is preclass autonomous learning or mixed learning based on flipped classrooms. The research finds that the characteristics of the knowledge structure mode of learners’ online learning are consistent with this teaching context, and further research is needed to generalize this research conclusion to other online learning contexts.