1 Introduction

Performing every cognitive task involves the employment of some cognitive resources and operations. Depending on the nature of the task, various cognitive resources might be employed by the performer to achieve a certain objective. For example, reading comprehension as one of the widely-used linguistic tasks in second language teaching is reliant on a variety of cognitive resources and cognitive operations such as long-term memory, working memory [11], higher-level control functions [5, 9], decoding [44], syntactic knowledge [10, 32], and vocabulary knowledge [44]. In many cases, we observe that an individual is relatively more successful in performing a certain group of cognitive activities than others. Such cases raise this critical question as to whether this group of tasks involve similar cognitive operations, in other words, ‘Are the same mechanisms employed by the individual while performing these tasks?’ If the answer is ‘yes’, a high degree of correlation between every pair of tasks would be expected. In such cases, we might conclude that some people are in a stronger position to perform a certain group of cognitive tasks. Here, a critical point is to identify the cognitive tasks that can be included in one category. Perhaps, one way to identify such categories is to give various tasks to a group of people. If we find a high degree of positive correlation between the performance of individuals in a pair of tasks, we might assume that both tasks involve similar cognitive resources or cognitive operations. For example, results of some past studies have shown that working memory is one of the key predictors of reading comprehension ability (e.g., [6, 11, 31]). If a strong correlation is found between performance in reading comprehension test and performance in cloze test, it may be hypothesized that working memory is an influential factor in cloze test. The objective of this study was to examine this question with regards to causal pattern test and cloze test.

In L2 learning, various cognitive tasks can be given to L2 learners in order to investigate possible correlations among them. This study set out to examine the degree of correlation between the performance of Iranian L2 learners in cloze passage tests and causal pattern tests. A cloze test is a test in which the participants must find missing words in a text with blank spaces. For each blank space, participants are given several options, one of which can best fill in the blank. In causal pattern tests, the participants have to find the patterns of cause-effect within a text. Causality connections play a key role in the understanding of texts [8]. Various types of cause-effect relationship can exist in a text. Rottman et al. [36] have discussed five types of causal patterns: common effect, common cause, causal chain, positive feedback, and negative feedback. Definition and examples of these causal patterns have been given in Table 1.

Table 1 Definitions of causal patterns [36]
Full size table

In this study, the first three types of causal patterns (common cause, common effect, causal chain) were included in the test to examine the ability of Iranian L2 learners in finding causal patterns within both English and Persian texts. The results of the two causal pattern tests (one in Persian and one in English) were compared with each other and the results of a cloze passage test. Building on the past studies that have investigated the processes involved in reading comprehension, this study intended to investigate the possibility of any significant relationship between the cognitive processes that are involved in these two special types of reading comprehension tests. To achieve this objective, correlations between participants’ scores were calculated for each pair among the three tests. Therefore, this study intended to answer the following questions:

  • Q1. Is there any significant correlation between the performance of test takers in cloze passage test and causal patterns test?

  • Q2. Are cognitive resources and cognitive operations involved in cloze passage test similar to cognitive resources and cognitive operations involved in causal patterns test?

It was hypothesized that the ability of the test takers to simultaneously activate various semantic properties (as measured by the cloze passage test) might be correlated with their ability in identifying causal patterns (common cause, common effect, causal chain) among various elements of a text (as measured by the causal pattern test). If this is the case, it could be hypothesized that the cognitive resources and cognitive operations involved in cloze test are likely to be similar to cognitive resources and cognitive operations involved in causal pattern test. This hypothesis was based on the assumptions of distributed models of conceptual representation. Based on these models, it was hypothesized that those individuals who are more skillful in activating some specific semantic features of a given term (e.g. based on its neighboring terms) would be more successful in both cloze passage tests and causal pattern tests.

2 Review of the literature

The ability to spontaneously notice key relational patterns in the flow of experience is a type of expertise that has not been extensively discussed in the literature of cognitive psychology [18]. However, causal patterns and the mechanisms through which they are discovered by people have been the subjects of several studies [1, 8, 19, 22, 23, 33]. Sensitivity to causal patterns and the ability to discover them during daily experiences is an important aspect of human cognition [2, 24, 39]. It has been demonstrated that employing reading strategies is one of the key predictors of success in reading comprehension among bilinguals [15]. Therefore, sensitivity to causality connections in the text, which can be seen as a reading strategy, is an important factor in reading success among bilinguals. According to Falkenhainer et al. [14] and Gentner and Markman [16], analogical comparison leads to the discovery of common relations through a process of structural alignment. Based on the findings of three studies conducted in educational contexts [17, 21, 34], it has been suggested that analogical comparison promotes transfer from a specific learning context to another context. However, Goldwater and Gentner [18] emphasize we cannot be sure that analogical comparison can improve transfer via abstracting the common structure.

One type of tests which may have some kind of similarity with causal pattern test is cloze passage. In cloze passage tests, the participant must read an incomplete text and identify the missing words. Several proposals have been suggested about the cognitive mechanisms that are involved in cloze tests. Anderson [3] suggests that cloze tests measure a sentence-level understanding and thus only assesses lower-order skills. On the other hand, Sasaki [38] proposes that cloze tests measure higher-level abilities that are beyond the understanding of single sentences in isolation. After conducting a large number of research projects in recent decades, it is now widely believed that both lower-level skills (skills involving the processing of clauses and sentences) and higher level skills (skills involving the processing of inter-sentential relations and making inferences) are involved in cloze tests [48]. Brown [4] argues that cloze tests involve a variety of skills, depending on the linguistic ability of the individual. He adds that for low-level language learners, it taps skills at the level of vocabulary and sentence grammar, for higher-level language learners, it taps skills that involve inter-sentential relationships and inferential abilities as well as vocabulary and grammatical abilities.

The degree of possible correlation between the causal pattern test and cloze passage test is a subject that has not been studied in the literature of this field. Various models might be used to describe the processes involved in causal pattern tests and cloze passage tests. One type of such models is distributed models of conceptual representation, according to which meanings of concepts are essentially componential; that is, the meaning of any concept is represented by small units of meaning, which are called semantic features [40, 43]. Distributed models of conceptual representations hold that every semantic feature or every feature node of a concept is represented in a connectionist network, and the understanding of that concept involves the co-activation of its semantic features or feature nodes [12, 13, 28,29,30, 42, 45]. The key assumption of distributed models is that the whole meaning consists of smaller units of meaning. That is, meaning of every concept can be analyzed into a set of independent semantic features. From the perspective of these models, when a concept is processed in the mind of a comprehender, its semantic features are activated in a complex connectionist network (e.g., [40, 42, 43]). Therefore, during the processing of a sentence (or a text) in the mind of the comprehender, semantic features of several concepts that are involved in that sentence are simultaneously activated in a complex connectionist network. This could happen in a more complex way when the comprehender tries to identify causal patterns in a text or identify the missing words in a cloze text, as the relationships and interactions between semantic features are critically important to identify causal patterns or missing words in such texts.

Causal pattern tests involve the extraction of cause–effect relationships among many elements within a set. In other words, the test taker must identify how certain elements of a set are related to each other and how they interact with each other. What is critical here is the nature of interaction among elements of a set. On the other hand, cloze tests involve finding the words that can best fill in the blanks in an incomplete text. Again, what is critical is the way that certain elements, embedded within a wider set of elements, interact with each other. These elements may be syntactic, semantic, cause–effect, etc. Therefore, it may be predicted that the cognitive resources involved in these two tests and the cognitive processes triggered by them are likely to be similar. One point that needs to be indicated is the role of L1 and L2. If both causal pattern tests and cloze tests are in L2, the clues that can be employed by the individual might be similar. On the other hand, if the causal pattern test is in L1, the individual might have access to some resources that are not available in a causal pattern test in L2.

The aim of the study was to find the degree of possible correlation between participants’ scores in each pair among the three tests (two causal pattern tests and one cloze passage test) taken. A possible high degree of correlation between every two tests could suggest that both involve similar cognitive resources or both trigger similar cognitive processes in the mind of an L2 learner.

3 Method

3.1 Participants

Participants of this study were selected from fifth semester students of English Translation in the Department of English in Chabahar Maritime University (Iran). This group consisted of 30 students, 17 were females and 13 were males (Mdn = 20; Qn = 0 [i.e., a negligible variability]; range = 20–24). All participants were Persian native speakers. To make sure that participants were at the same level of proficiency in English, a sample of the TOEFL test was administered (Mdn = 65 [i.e., competent users of English or a score of 6 in the IELTS]; Qn = 3.28; range = 57–70). All participants either received course creditor or participated voluntarily and gave their written informed consent. The study was carried out according to the declaration of Helsinki [47] and was approved by the local ethics committee.

3.2 Materials

Three tests were used in this study. In the first test which consisted of 12 texts written in English, the participants were expected to find the causal pattern in each text (see Appendix 1 in the supplementary files in https://figshare.com/projects/Causal_Pattern_Test_and_Cloze_Test/113841). Three types of causal patterns were included in the texts: common effect, common cause, and causal chain. Each text represented one type of casual pattern (thus four texts per causal pattern). Since other types of causal patterns may overlap with these three types, they were not used in this test, as they could cause confusion for participants of the study. Each correct answer received 3.5 scores and incorrect answers received 0. Therefore, the maximum and minimum possible scores in this test were 0 and 42. In the second test, participants were expected to find causal patterns in 12 texts written in Persian. The same three types of causal patterns used in the first test were also included in the Persian test. Like the English test, each text represented one type of causal pattern and had a score of 3.5. The third test consisted of two Cambridge Michigan ECP cloze passages, each one including 20 questions (see Appendix 2 in the supplementary files in https://figshare.com/projects/Causal_Pattern_Test_and_Cloze_Test/113841). A correct answer to each question was counted as one score. Therefore, the maximum and minimum scores in this test were 0 and 40, respectively.

3.3 Procedure

Before administering the tests, participants were provided with clear oral instructions to make sure they knew how to answer the questions. The first test (English causal patterns) was administered within 45 min. At the beginning of the test administration, three samples were given to the participants to make them familiar with test items. These samples included three different types of causal patterns (common cause, common effect, casual chain). The papers were collected, and the second test (Persian causal patterns) was immediately administered. The allotted time for the second test was 25 min. After the second test, participants had a 15-min break. Then, the third test (cloze passage) was administered in 40 min.

3.4 Data analysis

Robust methods were used for the descriptive and inferential statistics (see Mair and Wilcox [25] for details). The median (Mdn) and the Qn (see Rousseeuw and Croux [37]) were used for the estimation of location and scale (i.e. dispersion or spread) respectively. Planned correlations were assessed via percentage bend correlations (rpb; see [46]). Patterns of association were represented via robust locally weighted regression lines (LOWESS). Planned pairwise comparisons of dependent groups were carried out by comparing the Harrell-Davis quantiles of the marginal distributions associated with the two dependent groups (see “Robust tests for two dependent groups” section in [25]). All tests were assessed at α = 0.05 and descriptive and inferential statistical results are shown within the figures and their captions.

4 Results

The results from the three tests are represented in Fig. 1. The results indicate that there was a gain in scores from ‘test one’ (English causal patterns) to ‘test two’ (Persian causal patterns) and that the scores decreased from ‘test two’ to ‘test three’ (English cloze passages). Also, the scores in ‘test three’ were lower than the scores in the other two tests and all pairwise comparisons of median scores were statistically significant.

Fig. 1
figure 1

Boxplots representing the data’s distributions. The red circles represent individual scores and the notches around the medians represent approximately 95% confidence intervals (actual values shown in square brackets). Pairwise median comparisons suggested statistically significant differences in all cases: ‘test one’ and ‘test two’ (diff = − 5.04; 95% CI = [− 7.94, − 3.26]; p = 0), ‘test one’ and ‘test three’ (diff = 5.34; 95% CI = [1.72, 7.55]; p = 0.004), and ‘test two’ and ‘test three’ (diff = 10.39; 95% CI = [6.70, 13.55]; p = 0)

Full size image

The results of the correlation tests are shown in Fig. 2. These results suggest significant correlations between participants’ scores in every pair of these three tests. Note that in all three planned correlations, the associations were positive (see LOWESS fitting lines) and their associated p-values were well below the chosen alpha benchmark.

Fig. 2
figure 2

Results of the percentage bend correlation test between the three tests. The patterns of the associations are represented via LOWESS fitting lines (red dash lines). Actual pairwise observations are shown as black dots. The statistic of the test and its associated p-value are shown within each plot

Full size image

The data set and R code are available at https://figshare.com/projects/Causal_Pattern_Test_and_Cloze_Test/113841.

5 Discussion

As mentioned, the goal was to find out whether there was any similarity between cognitive resources and cognitive processes involved in causal pattern test and cloze passage test. The results indicated that there was a significant correlation between participants’ scores in tests of causal patterns (English and Persian) and their scores in cloze passage test. Also, correlation between participants’ scores in the English causal pattern test and the Persian causal pattern test was significant. Cloze passage tests involve finding the words or phrases that can best replace the missing parts of a text. Various types of clues provided in the text can be employed by the participant to correctly fill in the blank parts. The grammatical structure of each sentence is perhaps the first provider of clues needed to perform the task. A given blank space cannot be randomly filled by any category of words. The participant must first discover the abstract frame of the sentence and the relevant word order and only then will be able to identify the category of words that can fill the missing part of the sentence. However, since in most cloze passage tests, the provided options belong to the same category, this discovery cannot directly lead to choosing the right option.

The next type of clue is semantic in nature. Since some parts of the text are missing, the overall meaning is incomplete. The test taker constructs a general understanding of the text in her/his mind on the basis of the information that has been directly included in the text. However, the missing information must be derived by inference-based processes. In other words, the missing information must be deduced from or constructed out of existing information. The background knowledge of the test taker or his/her previous knowledge of the subject matter can be extremely useful in discovering the missing information (see Marmolejo-Ramos et al. [27]). Constructing the main idea or the whole meaning of a text is a dynamic process that can be facilitated by an interaction between background knowledge and inference (e.g., [7, 26, 35]). The cooperation between background knowledge and inference mechanism can be conceived as an additive process, that is, when a piece of information is deduced, the derivation of other pieces of information can be facilitated. In fact, when the text is made semantically more complete or coherent, the guessing of missing words or information would be easier.

Another source of very useful clues can best be described using distributed models of conceptual representation [40, 43]. As was mentioned, according to these models, meanings of words are essentially componential. Hence, when the meaning of a given term is processed in the mind, a set of semantic features and relationships are activated. This would indicate that throughout the processing of a cloze passage text, each word activates its own set of semantic features and relationships. The dynamic interaction among these sets of semantic features creates a partial meaning in the mind of the test taker. Here, we must distinguish between two types of interaction: firstly, the interaction among semantic features of each single concept; secondly, the interaction among semantic features of different concepts that appear in the text sequentially. The sum of these two types of interaction leads to the construction of the general idea of the text. Looking at this dynamic interaction from the perspective of distributed models of conceptual representation, one can assume that the general idea of the text is derived by a huge number of neural activities within a very large network of nodes. One part of these neural activities represents the semantic components or semantic properties of every single concept. Within each set of neural activities that represent a single concept, some tend to occur almost simultaneously. Such neural activities represent the semantic properties that are shared by many terms. On the other hand, there are some patterns of neural activity that tend not to occur simultaneously. These patterns represent distinguishing semantic features; that is, those semantic features that distinguish one concept from another related or unrelated concept. In this large neural network, the activation of some nodes facilitates the activation of others. This might be the reason for simultaneous occurrence of some neural activities [40, 43]. The simultaneous activities may represent those semantic features that are shared by many concepts. On the other hand, the neural activities that tend not to co-occur could represent those semantic features that distinguish a given concept from other concepts. These distinguishing neural activities represent the defining features of a concept. In fact, these are the point of departure, from which a concept might be distinguished from other concepts.

If we look at the processes involved in finding the missing words in a cloze passage test, it can be said that grasping the meaning of each word in the text corresponds to the activation of a set of neural connections. The activation of some of these neural connections can facilitate the activation of the concepts whose properties are represented by the activation of the same neural connections. In this way, the understanding of some words prepares the ground for the activation of the semantic features of other words. A large body of research on priming (for example, [20, 41]) has confirmed that words are more accessible when they are primed by other semantically-related words. This is particularly the case for the concepts that share many semantic properties. Therefore, if the patterns of neural activity that represent a missing word are similar to patterns of neural activity that represent the existing words, it can be predicted that the process of discovering the missing words is likely to be facilitated. In fact, the more features the two concepts share, the easier it is to find the missing words. As was mentioned, some semantic features or properties have a high degree of correlation [40, 43]; that is, they tend to co-occur in various concepts. Strong correlation among semantic features plays a key role in discovering the missing words of a cloze passage text, because the processing of each word could lead to the activation of semantic features of its neighboring words. In this way, the discovery of missing words is facilitated. In fact, it might be said that each word can function as a facilitating prime for the activation of its neighboring words.

Now we are in the position to explain the results obtained in this study based on distributed models of conceptual representation. A high level of success in a cloze passage test can be the result of successful co-activation of semantic features of words that are missing in the passage. In fact, the understanding of existing words leads to the activation of their semantic features. This can serve as a stimulating prime for the activation of missing words. People who have a higher ability in activating more semantic features are expected to perform better in finding the missing words of a cloze passage test. Here, the strongly-intercorrelated features could play a key role, because they are processed faster than weakly-intercorrelated features [29]. In this way, in a large network of interacting semantic properties, various semantic features are combined to create a concept that is represented by the missing word of the cloze passage test. Therefore, the key to success might be the co-activation of semantic features and their effective combination.

The results obtained in this study indicated that people who are more successful in cloze passage tests perform better in causal pattern tests, too. ‘Do both tasks involve similar cognitive resources and similar cognitive mechanisms?’ Although a significant correlation between the performance of individuals in two tasks cannot always be seen as a proof for similarity between cognitive processes that are involved in those tasks, it may be said that causal pattern test and cloze passage test share some similarities in the processes and mechanisms that are employed. Distributed models of conceptual representation are effective models that could be employed to describe the underlying similarities between the cognitive processes of these two tests. In the previous paragraphs, the possible processes involved in cloze passage tests were described based on distributed models of conceptual representation. As for causal pattern tests, it can be said that the test taker must construct the overall structure of the text in his/her mind to discover the causal patterns within a text. This overall structure can be likened to the map of an area which shows how various places are connected to each other. If a test taker is going to discover the causal pattern among the elements of a text, s/he must have a clear idea of how semantic features or properties of each element within the text are related to properties of other elements. The first step in the discovery process is to identify which semantic properties are associated with a ‘cause’ and which semantic properties are associated with an ‘effect’. For instance, one of the semantic features of the term ‘virus’ is ‘being the cause of diseases’. On the other hand, ‘being caused by a virus’ is one of the semantic features of those diseases which are the result of a virus entering the body. In fact, the cause-effect relationship between virus and disease is part of the meaning of these two terms. The rapid activation of these semantic features in the mind of the test taker is likely to facilitate the process of finding causal patterns embedded within a text. If the cause-effect semantic property of these terms has a high degree of co-occurrence with other semantic properties, its processing will be easier [29]. In other words, the strength of intercorrelation among various semantic features of terms is critical for discovering the causal patterns within a text.

6 Summary

This article investigated the cognitive processes involved in cloze passage tests and causal pattern tests (English and Persian). The results obtained in this study indicated that levels of success in these two tests are significantly correlated. The participants who were successful in one of these tests were also successful in the other test. Based on the distributed models of conceptual representation, it was suggested that a possible reason for this is the strong co-activation of semantic features of the words which play a key role in the text. In the case of the cloze passage test, this co-activation enables the individual to identify the semantic features of missing words. In the case of causal pattern tests, it enables the test taker to identify the semantic features that are specifically related to cause-effect relationships. Therefore, it is suggested that cognitive mechanisms employed while taking these tests are likely to be similar. Although a significant correlation between the performance of individuals in two types of task cannot always be the result of similar cognitive processes involved in those tasks, this article suggested that causal pattern test and cloze passage test share some basic similarities in the cognitive mechanisms that are employed by test takers. These similarities were described on the basis of distributed models of conceptual representation. It must be noted that this study investigated the performance of Persian native speakers. If speakers of several other languages had participated in this study, more reliable results could have been achieved. This was the major limitation of this study. Finally, this article looked at the cognitive mechanisms involved in causal patterns test and cloze passage test only from the perspective of distributed models of conceptual representation. There might be variety of cognitive resources and cognitive mechanisms involved in these two tests that cannot be captured through the perspective of these models. Identifying and describing these resources and mechanisms is a question that can be addressed in future research.