Abstract
This experimental investigation seeks to corroborate a knowledge structure sorting task approach as a measure to more fully account for prior knowledge when reading. A latent semantic analysis (LSA) network derived from thousands of texts typically read by first year college students was used to create a prototypical referent network model of the global collective knowledge structure of the key terms in the text. Bilingual Chinese-English participants (n = 205) were randomly assigned to four treatments to sort terms in both languages, then to read an English expository text of an unfamiliar topic, then sort in both languages again, and lastly complete a comprehension posttest. All pre- and post- sorting tasks data were converted to Pathfinder networks as measures of knowledge structure. Multiword clusters in the LSA network were present in the initial pre-reading group-average sorting networks of both languages, but especially in Chinese (their L1), and these clusters tended to persist after reading. Reading had only a small influence on the post-reading group-average networks. Sorting in Chinese had a stronger influence downstream than did sorting in English (L1 > L2 influence). For researchers, these innovative approaches to establish local and global collective knowledge networks show promise as complementary measures to explain learning in terms of knowledge structure alignment and transitions, and pragmatically, sorting tasks are relatively easy to implement and interpret in real classrooms as formative diagnostic measures of conceptual understanding.
Similar content being viewed by others
References
Anderson, R. C. (1984). Some reflections on the acquisition of knowledge. Educational Researcher, 13(10), 5–10.
Arnon, I. (2009). Starting big—The role of mulit-word phrases in language learning and use. [Doctoral Dissertation], Stanford University. https://tinyurl.com/Arnon-2009
Arnon, I., & Christiansen, M. H. (2017). The role of multiword building blocks in explaining L1–L2 differences. Topics in Cognitive Science, 9, 621–636. https://doi.org/10.1111/tops.12271
Asino, T., Clariana, R.B., Dong, Y., Groff, B., Ntshalintshali, G., Taricani, E., Techatassanasoontorn, C., & Yu, W. (2012). The effect of independent and interdependent group collaboration on knowledge extent, knowledge form, and knowledge convergence. In Proceedings of selected research and development papers presented at the national convention of the association for educational communications and technology (Vol. 35, pp. 20–29) (Louisville, KY, November 2012). https://members.aect.org/pdf/Proceedings/proceedings12/2012/12_02.pdf
Balloo, K., Pauli, R., & Worrell, M. (2016). Individual differences in psychology undergraduates’ development of research methods knowledge and skills. Procedia-Social and Behavioral Sciences, 217, 790–800.
Barron, B. (2003). When smart groups fail. Journal of the Learning Sciences, 12(3), 307–359.
Bonilla, J. L., & Johnson, M. K. (1995). Semantic space in Alzheimer’s disease patients. Neuropsychology, 9(3), 345–353.
Bransford, J. D., Brown, A. L., & Cocking, R. R. (Eds.). (2000). How people learn: Brain, mind, experience, and school. National Academy Press.
Britton, B. K., & Gülöz, S. (1991). Using Kintsch’s computational model to improve instructional text: Effects of repairing inference calls on recall and cognitive structures. Journal of Educational Psychology, 83, 329–345.
Brysbaert, M., Mandera, P., McCormick, S. F., & Keuleers, E. (2019). Word prevalence norms for 62,000 English lemmas. Behavior Research Methods, 51, 467–479. https://doi.org/10.3758/s13428-018-1077-9
Chan, S. S., Butters, N., & Salmon, D. P. (1997). The deterioration of semantic networks in patients with Alzheimer’s disease: A cross-sectional study. Neuropsychologia, 35(3), 241–248.
Chase, W. G., & Simon, H. A. (1973). Perception in chess. Cognitive Psychology, 4, 55–81.
Chen, B., Scardamalia, M., Resendes, M., Chuy, M., & Bereiter, C. (2012). Students’ intuitive Understanding of promising-ness and promising-ness judgments to facilitate knowledge advancement. In Proceedings of the 10th international conference of the learning sciences: The future of learning, ICLS 2012 - Sydney, NSW, Australia.
Chen, W. (2017). Knowledge convergence among pre-service mathematics teachers through online reciprocal peer feedback. Knowledge Management & E-Learning, 9(1), 1–18.
Chen, X., Dong, Y., & Yu, X. (2018). On the predictive validity of various corpus-based frequency norms in L2 English lexical processing. Behavior Research Methods, 50, 1–25. https://doi.org/10.3758/s13428-017-1001-8
Chi, M. T., Feltovich, P. J., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5, 121–152.
Chi, M. T., Glaser, R., & Rees, E. (1982). Expertise in problem solving. In R. Sternberg (Ed.), Advances in the psychology of human intelligence (pp. 7–75). Erlbaum.
Chollet, S., Valentin, D., & Abdi, H. (2014). Free sorting task. In P. Varela & G. Ares (Eds.), Novel techniques in sensory characterization and consumer profiling. CRC Press, Taylor and Francis. https://doi.org/10.1201/b16853
Christiansen, M. H., & Arnon, I. (2017). More than words: The role of multiword sequences in language learning and use. Topics in Cognitive Science, 9, 542–551. https://doi.org/10.1111/tops.12274
Clariana, R. B. (2010a). Deriving group knowledge structure from semantic maps and from essays. In D. Ifenthaler, P. Pirnay-Dummer, & N. M. Seel (Eds.), Computer-based diagnostics and systematic analysis of knowledge (pp. 117–130). Springer.
Clariana, R. B. (2010b). Multi-decision approaches for eliciting knowledge structure. In D. Ifenthaler, P. Pirnay-Dummer, & N. M. Seel (Eds.), Computer-based diagnostics and systematic analysis of knowledge (Chapter 4) (pp. 41–59). Springer.
Clariana, R. B., Follmer, D. J., & Li, P. (2019). Sentence versus paragraph processing: Linear and relational knowledge structure measures. Presented at the 7th International Workshop on Advanced Learning Sciences (IWALS 2019), June 17–19, 2019, University of Jyväskylä, Finland. https://www.slideshare.net/rbc4/sentence-versus-paragraph-processing-linear-and-relational-knowledge-structure-measures
Clariana, R. B., & Wallace, P. E. (2009). A comparison of pair-wise, list-wise, and clustering approaches for eliciting structural knowledge in information systems courses. International Journal of Instructional Media, 36(3), 287–302.
Clariana, R. B., Wallace, P. E., & Godshalk, V. M. (2009). Deriving and measuring group knowledge structure from essays: The effects of anaphoric reference. Educational Technology Research and Development, 57, 725–737.
Crowley, K., & Jacobs, M. (2002). Building islands of expertise in everyday family activity. In G. Leinhardt, K. Crowley, & K. Knutson (Eds.), Learning conversations in museums (pp. 333–256). Lawrence Erlbaum Associates.
Davis, M. H., & Guthrie, J. T. (2015). Measuring reading comprehension of content area texts using an assessment of knowledge organization. The Journal of Educational Research, 108(2), 148–164.
Draper, D. C. (2010). The instructional effects of knowledge-based community of practice learning environments on student achievement and knowledge convergence. [Doctoral Dissertation], The Pennsylvania State University. https://etda.libraries.psu.edu/files/final_submissions/133
Elman, J. L. (2004). An alternative view of the mental lexicon. Trends in Cognitive Science, 8(7), 301–306.
Elman, J. L. (2009). On the meaning of words and dinosaur bones: Lexical knowledge without a lexicon. Cognitive Science, 33, 547–582.
Feng, S., & Law, N. (2021). Mapping artificial intelligence in education research: A network-based keyword analysis. International Journal of Artificial Intelligence in Education, 31, 277–303. https://doi.org/10.1007/s40593-021-00244-4
Fischer, F., & Mandl, H. (2005). Knowledge convergence in computer-supported collaborative learning—The role of external representation tools. Journal of the Learning Sciences, 14, 405–441.
Furtner, M. R., Rauthmann, J. F., & Sachse, P. (2009). Nomen est omen: Investigating the dominance of nouns in word comprehension with eye movement analyses. Advances in Cognitive Psychology, 5, 91–104.
Gentner, D. (1983). Structure-Mapping: A theoretical framework for analogy. Cognitive Science, 7, 155–170.
Gentner, D., & Hoyos, C. (2017). Analogy and abstraction. Topics in Cognitive Science, 9, 672–693. https://doi.org/10.1111/tops.12278
Gentner, D., & Markman, A. B. (1997). Structure mapping in analogy and similarity. American Psychologist, 52, 45–56.
Georgakopoulos, T., & Polis, S. (2018). The semantic map model: State of the art and future avenues for linguistic research. Language and Linguistics Compass, 12, 1–33. https://doi.org/10.1111/lnc3.12270
Glaser, B. G., & Strauss, A. L. (1967). Discovery of grounded theory: Strategies for qualitative research. Aldine.
Grabowski, B. (2003). Generative learning contributions to the design of instruction and learning. In D. H. Jonassen (Ed.), Handbook of research on educational communications and technology (2nd ed., pp. 719–743). Lawrence Erlbaum Associates.
Graesser, A., Karnavat, A., Pomeroy, V. & Wiemer-Hastings, K. (2000). Latent Semantic Analysis captures casual, goal-oriented, and taxonomic structures. In Proceedings of the annual meeting of the cognitive science society, 22. https://escholarship.org/uc/item/2mw8430f
Günther, F., Dudschig, C., & Kaup, B. (2016). Latent semantic analysis cosines as a cognitive similarity measure: Evidence from priming studies. The Quarterly Journal of Experimental Psychology, 69(4), 626–653. https://doi.org/10.1080/17470218.2015.1038280
Hecker, A. (2012). Knowledge beyond the individual? Making sense of a notion of collective knowledge in organization theory. Organization Studies, 33(3), 423–445.
Hesse, M. B. (1966). Models and analogies in science. University of Notre Dame Press.
Hesse, M. B. (2008). Models and analogies. In W. H. Newton-Smith (Ed.), A companion to the Philosophy of Science (pp. 299–307). Blackwell Publishers Ltd.
Hulin, W. S., & Katz, D. (1935). The Frois-Wittmann pictures of facial expression. Journal of Experimental Psychology, 18(4), 482–498. https://doi.org/10.1037/h0056770
Ifenthaler, D. (2010). Relational, structural, and semantic analysis of graphical representations and concept maps. Educational Technology Research and Development, 58, 81–97.
Ifenthaler, D. (2011). Identifying cross-domain distinguishing features of cognitive structure. Educational Technology Research and Development, 59, 817–840.
Jeong, H., & Chi, M. (2007). Knowledge convergence and collaborative learning. Instructional Science, 35, 287–316.
Jonassen, D. H., Beissner, K., & Yacci, M. (1993). Structural knowledge: Techniques for representing, conveying, and acquiring structural knowledge. Lawrence Erlbaum Associates.
Kauhanen, I. (2006). Norms and sociolinguistic description. In A man of measure: festschrift in honour of Fred Karlsson on his 60th Birthday (pp. 34–46). (SKY Journal of Linguistics; Vol. 19, No. Special supplement). The Finnish Linguistics Association. http://www.linguistics.fi/julkaisut/SKY2006_1/1FK60.1.4.KAUHANEN.pdf
Kemp, C., & Tenenbaum, J. B. (2008). The discovery of structural form. Proceedings of the National Academy of Sciences, 105(31), 10687–10692. https://doi.org/10.1073/pnas.0802631105
Kim, K., & Clariana, R. B. (2015). Knowledge structure measures of reader’s situation models across languages: Translation engenders richer structure. Technology, Knowledge and Learning, 20, 249–268.
Kim, K., & Clariana, R. B. (2018). Text signals influence second language expository text comprehension: Knowledge structure analysis. Educational Technology Research and Development, 65, 909–830. https://doi.org/10.1007/s11423-016-9494-x
Kim, K., & Clariana, R. B. (2019). Applications of Pathfinder Network scaling for identifying the optimal use of a first language to support second language text comprehension. Educational Technology Research and Development, 67, 85–103. https://doi.org/10.1007/s11423-018-9607-9
Kim, K., Clariana, R. B., & Kim, Y. (2019). Automatic representation of knowledge structure: Enhancing learning through knowledge structure reflection in an online course. Educational Technology Research and Development, 67, 105–122. https://doi.org/10.1007/s11423-018-9626-6
Kintsch, W., & Mangalath, P. (2011). The construction of meaning. Topics in Cognitive Science, 3, 346–370. https://doi.org/10.1111/j.1756-8765.2010.01107.x
Koda, K. (2007). Reading and language learning: Cross linguistic constraints on second language reading development. Language Learning, 57(1), 1–44.
Krabbe, H. (2014). Digital concept mapping for formative assessment. In D. Ifenthaler & R. Hanewald (Eds.), Digital knowledge maps in education: Technology-enhance support for teachers and learners (pp. 275–297). Springer. https://doi.org/10.1007/978-1-4614-3178-7_15
Krethlow, G., Fargier, R., & Laganaro, M. (2020). Age-specific effects of lexical–semantic networks on word production. Cognitive Science, 44(11), e12915. https://doi.org/10.1111/cogs.12915
Kurtz, K. J., & Honke, G. (2020). Sorting out the problem of inert knowledge: Category construction to promote spontaneous transfer. Journal of Experimental Psychology: Learning, Memory, and Cognition, 46(5), 803–821. https://doi.org/10.1037/xlm0000750
Landauer, T. K., & Dumais, S. T. (1997). A solution to Plato’s problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychological Review, 104(2), 211–240. https://doi.org/10.1037/0033-295X.104.2.211
Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge University Press.
Leshchenko, Y., Dotsenko, T., & Ostapenko, T. (2015). Cross-linguistic interactions in bilingual mental lexicon and professional linguistic competence formation: An experimental research with native speakers of the Komi-Permyak and Russian languages. Social and Behavioral Sciences, 214, 1039–1047.
Levy, P. (1999). Collective intelligence (R. Bononno, trans.). Perseus Books.
Li, P., & Clariana, R. B. (2019). Reading comprehension in L1 and L2: An integrative approach. Journal of Neurolinguistics, 50, 94–105.
Louwerse, M. M. (2011). Symbol interdependency in symbolic and embodied cognition. Topics in Cognitive Science, 3, 273–302.
Lu, L., Yuan, Y. C., & McLeod, P. L. (2012). Twenty-five years of hidden profiles in group decision making: A meta-analysis. Personality and Social Psychology Review, 16, 54–75.
Mak, M. H. C., & Twitchell, H. (2020). Evidence for preferential attachment: Words that are more well connected in semantic networks are better at acquiring new links in paired-associate learning. Psychonomic Bulletin & Review, 27, 1059–1069. https://doi.org/10.3758/s13423-020-01773-0
McCauley, S. M., & Christiansen, M. H. (2017). Computational investigations of multiword chunks in language learning. Topics in Cognitive Science, 9, 637–652. https://doi.org/10.1111/tops.12258
McComb, S. A. (2007). Mental model convergence: The shift from being an individual to being a team member. In F. Dansereau & F. J. Yammarino (Eds.) Multi-level issues in organizations and time (Research in multi-level issues) (Vol. 6, pp. 95–147). Emerald Group Publishing Limited. https://doi.org/10.1016/S1475-9144(07)06005-5
McNamara, D. S., & Magliano, J. (2009). Toward a comprehensive model of comprehension. In B. H. Ross (Ed.), Psychology of learning and motivation (Vol. 51, pp. 297–384). Academic Press.
Mun, Y. (2015). The effect of sorting and writing tasks on knowledge structure measure in bilinguals’ reading comprehension (Master’s thesis). https://scholarsphere.psu.edu/concern/generic_works/x059c7329.
NCES. (2019). Indicator 8: English language learners in public schools. Institute of Education Science: National Center for Education Statistics. Retrieved April, 2022, from https://nces.ed.gov/programs/raceindicators/indicator_rbc.asp
Ntshalintshali, G. N., & Clariana, R. B. (2020). Paraphrasing refutation text improved higher knowledge forms and hindered lower knowledge forms: Examples from repairing relational database design misconceptions. Educational Technology Research and Development, 68, 2165–2183. https://doi.org/10.1007/s11423-020-09758-5
Oden, D. L., Thompson, R. K. R., & Premack, D. (1990). Infant chimpanzees spontaneously perceive both concrete and abstract same/different relations. Child Development, 61, 621–631.
Raudszus, H., Segers, E., & Verhoeven, L. (2017). Quality of situation model building predicts first and second language reading comprehension. Presented at the 27th Annual Meeting of the Society for Text and Discourse, Philadelphia, PA. http://www.societyfortextanddiscourse.org/wp-content/uploads/2017/09/STD.Program.2017.pdf
Raudszus, H., Segers, E., & Verhoeven, L. (2019). Situation model building ability uniquely predicts first and second language reading comprehension. Journal of Neurolinguistics, 50, 106–119.
Reed, S. K. (2012). Learning by mapping across situations. Journal of the Learning Sciences, 21(3), 353–398. https://doi.org/10.1080/10508406.2011.607007
Rosch, E. H. (1973). Natural categories. Cognitive Psychology, 4, 328–350.
Roschelle, J. (1992). Learning by collaborating: Convergent conceptual change. Journal of the Learning Sciences, 2(3), 235–276.
Schuelke, M. (2012). jRateDrag version 2.0 [Computer software]. https://drive.google.com/drive/folders/0B62ahmj_ECTCUjBDMGswRXg3d0U?resourcekey=0-pzzOXhwN1aKWN7pd1KnR7Q
Schvaneveldt, R. W. (2020). JPathfinder software. https://research-collective.com/PFWeb/
Schvaneveldt, R. W., Durso, F. T., & Dearholt, D. W. (1989). Network structures in proximity data. In G. Bower (Ed.), The psychology of learning and motivation: Advances in research & theory (pp. 249–284). Academic Press.
Spinozzi, G. (1993). Development of spontaneous classificatory behavior in chimpanzees (Pan troglodytes). Journal of Comparative Psychology, 107, 193–200. https://doi.org/10.1037/0735-7036.107.2.193
Surowiecki, J. (2004). The wisdom of crowds: Why the many are smarter than the few and how collective wisdom shapes business, economies, societies, and nations. Doubleday.
Tang, H., & Clariana, R. (2017). Leveraging a sorting task as a measure of knowledge structure in bilingual settings. Technology, Knowledge and Learning, 22(1), 23–35.
Taricani, E. M., & Clariana, R. B. (2006). A technique for automatically scoring open-ended concept maps. Educational Technology Research and Development, 54(1), 65–82.
Tenser, A. (2016). Semantic map borrowing—Case representation in northeastern Romani dialects. Journal of Language Contact, 9(2), 211–245. https://doi.org/10.1163/19552629-00902001
Teplovs, C., & Scardamalia, M. (2007). Visualizations for knowledge building assessment. [Conference presentation]. AgileViz workshop, CSCL 2007 Convention, New Brunswick, NJ, United States. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.590.1779
Trumpower, D. L., & Goldsmith, T. E. (2004). Structural enhancement of learning. Contemporary Educational Psychology, 29, 426–446.
Trumpower, D. L., & Sarwar, G. S. (2010). Effectiveness of structural feedback provided by Pathfinder networks. Journal of Educational Computing Research, 43, 7–24.
Tseng, Y.-H., Chang, C.-Y., Rundgren, S.-N.C., & Rundgren, C.-J. (2010). Mining concept maps from news stories for measuring civic scientific literacy in media. Computers and Education, 55, 165–177.
Ursachi, G., Horodnic, I. A., & Zait, A. (2015). How reliable are measurement scales? External factors with indirect influence on reliability estimators. Procedia Economics and Finance, 20, 679–686.
Vincent-Lamarre, P., Massé, A. B., Lopes, M., Lord, M., Marcotte, O., & Harnada, S. (2016). The latent structure of dictionaries. Topics in Cognitive Science, 8, 625–659.
Winkielman, P., Halberstadt, J., Fazendeiro, T., & Catty, S. (2006). Prototypes are attractive because they are easy on the mind. Psychological Science, 17(9), 799–806.
Zareva, A., & Wolter, B. (2012). The ‘promise’ of three methods of word association analysis to L2 lexical research. Second Language Research, 28(1), 41–67. https://doi.org/10.1177/0267658311423452
Zemčík, T. (2020). Failure of chatbot Tay was evil, ugliness and uselessness in its nature or do we judge it through cognitive shortcuts and biases? AI & Society, 36, 361–367. https://doi.org/10.1007/s00146-020-01053-4
Zhang, J., Scardamalia, M., Lamon, M., Messina, R., & Reeve, R. (2007). Socio-cognitive dynamics of knowledge building in 9- and 10-year-olds. Educational Technology Research and Development, 55, 117–145.
Zhao, X., & Li, P. (2013). Simulating cross-language priming with a dynamic computational model of the lexicon. Bilingualism: Language and Cognition, 16, 288–303.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Clariana, R.B., Tang, H. & Chen, X. Corroborating a sorting task measure of individual and of local collective knowledge structure. Education Tech Research Dev 70, 1195–1219 (2022). https://doi.org/10.1007/s11423-022-10123-x
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11423-022-10123-x