Skip to main content
SpringerLink
Log in

A Technique for Automatically Scoring Open-Ended Concept Maps

  • Published:
Educational Technology Research and Development Aims and scope Submit manuscript

Abstract

In this descriptive investigation, we seek to confirm and extend a technique for automatically scoring concept maps. Sixty unscored concept maps from a published dissertation were scored using a computer-based technique adapted from Schvaneveldt (1990) and colleague's Pathfinder network approach. The scores were based on link lines drawn between terms and on the geometric distances between terms. These concept map scores were compared to terminology and comprehension posttest scores. Concept map scores derived from link data were more related to terminology whereas concept map scores derived from distance data were more related to comprehension. A step-by-step description of the scoring technique is presented and the next steps in the development process are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Ausubel, D. P. (1968). Educational psychology: a cognitive view. New York, NY: Holt, Rinehart, and Winston.

    Google Scholar 

  • Buzydlowski, J. W. (2002). A comparison of self-organizing maps and Pathfinder networks for the mapping of co-cited authors. Doctoral dissertation completed at Drexel University. Retrieved October 3, 2004, from http://faculty.cis.drexel.edu/jbuzydlo/papers/the-sis.pdf.

  • Cañas, A. J., Coffey, J. W., Carnot, M. J., Feltovich, P., Hoffman, R. R., Feltovich, J., & Novak, J. D. (2003). A summary of literature pertaining to the use of concept mapping techniques and technologies for education and performance support. A report for The Chief of Naval Education and Training at Pensacola, Florida. Retrieved October 3, 2004, from http://www.ihmc.us/users/aCañas/Publications/ConceptMapLitReview/IHMC /LiteratureReviewonConceptMapping.pdf.

  • Clariana, R. B. (2002). ALA-Mapper, version 1.0. Retrieved October 3, 2002, from http://www.personal.psu.edu/rbc4/ala.htm.

  • Clariana, R. B. (2003). Latent semantic analysis writing prompt, used with the permission of Knowledge Analysis Technologies, LLC. Retrieved October 30, 2003 from http://www.personal.psu.edu/rbc4/frame.htm.

  • Clariana, R. B., Koul, R., & Salehi, R. (in press). The criterion related validity of a com-puter-based approach for scoring concept maps. International Journal of Instructional Media, 33(3), in press.

  • Clariana, R. B., & Poindexter, M. T. (2004). The influence of relational and proposition-spe-cific processing on structural knowledge. Presented at the annual meeting of the American Educational Research Association, April 12–16, San Diego, CA. Retrieved April 15, 2004 from http://www.personal.psu.edu/rbc4/aera_2004.doc.

  • Crocker, L. A., & Algina, J. (1986). Introduction to classical and modern test theory. New York, NY: Holt, Rinehart, and Winston.

    Google Scholar 

  • Dearholt, D. W. & Schvaneveldt, R. W. (1990). Properties of Pathfinder Networks. In Schvaneveldt (Ed.), Pathfinder associative networks: studies in knowledge organization (pp. 1–30). Norwood, NJ: Ablex Publishing Corporation.

  • Deese, J. (1965). The structure of associations in language and thought. Baltimore, MD: The Johns Hopkins Press.

    Google Scholar 

  • Durso, F. T., & Coggins, K. A. (1990). Graphs in the social and psychological sciences: empirical contributions of Pathfinder. In Schvaneveldt (Ed.), Pathfinder associative networks: Studies in knowledge organization (pp. 31–51). Norwood, NJ: Ablex Publishing Corporation.

  • Dwyer, F. M. (1972). A guide for improving visualized instruction. State College, PA: Learning Services.

    Google Scholar 

  • Elman, J. L. (in press). An alternative view of the mental lexicon. Trends in Cognitive Science, in press. Retrieved December 10, 2004 from http://www.crl.ucsd.edu/-elman/Papers/elman_tics_opinion_2004.pdf.

  • Frase, L. T. (1969). Structural analysis of the knowledge that results from thinking about text. Journal of Educational Psychology, 60(6, Part 2), 1–16.

    Google Scholar 

  • Goldsmith, T. E., & Davenport, D. M. (1990). Assessing structural similarity of graphs. In Schvaneveldt (Ed.), Pathfinder associative networks: studies in knowledge organization (pp. 75–87). Norwood, NJ: Ablex Publishing Corporation.

  • Harper, M. E., Hoeft, R. M., Evans, A. W., & Jentsch, F. G. (2004). Scoring concepts maps: Can a practical method of scoring concept maps be used to assess trainee's knowledge structures? Paper presented at the Human Factors and Ergonomics Society 48th Annual Meeting.

  • Herl, H. E., O'Neil, H. F. Jr., Chunga, G. K. W. K., & Schacter, J. (1999). Reliability and validity of a computer-based knowledge mapping system to measure content understanding. Computers in Human Behavior, 15, 315–333.

    Article  Google Scholar 

  • Humphreys, M. S., Bain, J. D., & Pike, R. (1989). Different ways to cue a coherent memory system: A theory for episodic, semantic and procedural tasks. Psychological Review, 96, 208–233.

    Article  Google Scholar 

  • Jonassen, D. H. (1987). Assessing cognitive structure: verifying a method using pattern notes. Journal of Research and Development in Education, 20(3), 1–13.

    Google Scholar 

  • Jonassen, D. H., Beissner, K., & Yacci, M. (1993). Structural knowledge: techniques for representing, conveying, and acquiring structural knowledge. Hillsdale, NJ: Lawrence Erlbaum Associates.

  • Kinchin, I. M. (2000). Using concept maps to reveal understanding: A two-tier analysis. School Science Review, 81, 41–46.

    Google Scholar 

  • KNOT. (1998). Knowledge Network and Orientation Tool for the Personal Computer, version 4.3. Retrieved October 3, 2003 from http://interlinkinc.net/.

  • Lambiotte, J., & Dansereau, D. (1992). Effects of knowledge maps and prior knowledge on recall of science lecture content. Journal of Experimental Education, 60, 189–201.

    Google Scholar 

  • Lee, Y., & Nelson, D. (2005). Viewing or visualizing'which concept map strategy works best on problem solving performance? British Journal of Educational Technology, 36(2), 193–203.

    Google Scholar 

  • Luckie, D. (2001). C-TOOLS: Concept-Connector Tools for online learning in science. A National Science Foundation grant. Retrieved October 3, 2003 from http://www.msu.edu/luckie/CTOOLS.pdf.

  • McClure, J., Sonak, B., & Suen, H. (1999). Concept map assessment of classroom learning: reliability, validity, and logistical practicality. Journal of Research in Science Teaching, 36, 475–492.

    Article  Google Scholar 

  • McClelland, J. L., McNaughton, B. L., & O'Reilly, R. C. (1995). Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models. Psychological Review, 102, 419–457.

    Article  Google Scholar 

  • Novak, J. D., & Gowin, D. B. (1984). Learning how to learn. Cambridge, UK: Cambridge University Press.

    Google Scholar 

  • Pike, R. (1984). A comparison of convolution and matrix distributed memory systems. Psychological Review, 91, 281–294.

    Google Scholar 

  • Robinson, D. H., Corliss, S. B., Bush, A. M., Bera, S. J., & Tomberlin, T. (2003). Optimal presentation of graphic organizers and text: A case for large bites? Educational Technology Research and Development, 51(4), 25–41.

    Google Scholar 

  • Ruiz-Primo, M. A., & Shavelson, R. J. (1996). Problems and issues in the use of concept maps in science assessment. Journal of Research in Science Teaching, 33, 569–600.

    Google Scholar 

  • Ruiz-Primo, M. A., Schultz, S., Li, M., & Shavelson, R. J. (1999). On the cognitive validity of interpretations of scores from alternative concept mapping techniques. (CSE Tech. Rep. No. 503). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).

  • Rye, J. A., & Rubba, P. A. (2002). Scoring concept maps: an expert map-based scheme weighted for relationships. School Science and Mathematics, 102, 33–44.

    Google Scholar 

  • Safayeni, F., Derbentseva, N., & Cañas, A. J. (2003). Concept maps: a theoretical note on concepts and the need for cyclic concept maps. Retrieved October 3, 2004 from http://cmap.ihmc.us/Publications/ResearchPapers/Cyclic%20Concept%20Maps. pdf.

  • Schau, C., Mattern, N., Zeilik, M., & Teague, K. W. (1999). Select-and-fill-in concept map scores as a measure of students' connected understanding of introductory astronomy. Paper presented at the Annual Meeting of the American Educational Research Association (Montreal, Canada, April). Available from ERIC ED434909.

  • Shavelson, R. J., Lang, H., & Lewin, B. (1994). On concept maps as potential authentic assessment is science. (CSE Tech. Rep. No. 388). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).

  • Schvaneveldt, R. W. (1990). Pathfinder associative networks: Studies in knowledge organization. Norwood, NJ: Ablex Publishing Corporation.

    Google Scholar 

  • Schvaneveldt, R. W., Dearholt, D. W., & Durso, F. T. (1988). Graph theoretic foundations of Pathfinder networks. Computers and Mathematics with Applications, 15, 337– 345.

    Article  Google Scholar 

  • Tan, S. C. (2000). The effects of incorporating concept mapping into computer assisted instruction. Journal of Educational Computing Research, 23, 113–131.

    Article  Google Scholar 

  • Taricani, E. M. (2002). Effects of the level of generativity in concept mapping with knowledge of correct response feedback on learning. Dissertation Abstracts International, ISBN # 0–493–67101–3.

  • Yin, Y., Vanides, J., Ruiz-Primo, M. A., Ayala, C. C., & Shavelson, R. J. (2004). A comparison of two construct-a-concept-map science assessments: created linking phrases and selected linking phrases. (CSE Tech. Rep. No. 624). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).

Download references

Author information

Authors and Affiliations

  1. College of Education, The Pennsylvania State University, Pennsylvania

    Ellen M. Taricani & Roy B. Clariana

Authors
  1. Ellen M. Taricani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Roy B. Clariana
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Roy B. Clariana.

Additional information

Ellen M. Taricani and Roy B. Clariana [RClariana@psu.edu] are with the College of Education at The Pennsylvania State University.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Taricani, E.M., Clariana, R.B. A Technique for Automatically Scoring Open-Ended Concept Maps. EDUCATION TECH RESEARCH DEV 54, 65–82 (2006). https://doi.org/10.1007/s11423-006-6497-z

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11423-006-6497-z

Keywords

Search

Navigation