Skip to main content

An Evaluation Methodology for Concept Maps Mined from Lecture Notes: An Educational Perspective

  • Conference paper
  • First Online:
Computer Supported Education (CSEDU 2014)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 510))

Included in the following conference series:

  • 814 Accesses


Concept maps are effective tools that assist learners in organising and representing knowledge. Recent efforts in the area of concept mapping work toward semi- or fully automated approaches to extract concept maps from various text sources such as text books. The motivation for this research is twofold: novice learners require substantial assistance from experts in constructing their own maps, introducing additional hurdles, and alternatively, the workload required by academics in manually constructing expert maps is substantial and repetitive. A key limitation of an automated concept map generation is the lack of an evaluation framework to measure the quality of concept maps. The most common evaluation mechanism is measuring the overlap between machine-generated elements (e.g. concepts) with expert maps using relevancy measures such as precision and recall. However, in the educational context, the majority of knowledge presented is relevant to the learner, resulting in a large amount of information being retrieved for knowledge organisation. Therefore, this paper introduces a machine-based approach to evaluate the relative importance of knowledge by comparing with human judgment. We introduce three ranking models and conclude that the structural features are positively correlated with human experts (rs ~ 1) for courses with rich content and good structure (well-fitted).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others


  1. 1.

  2. 2.

  3. 3.


  • Atapattu, T., Falkner, K., Falkner, N.: Automated extraction of semantic concepts from semi-structured data: supporting computer-based education through the analysis of lecture notes. In: Liddle, S.W., Schewe, K.-D., Tjoa, A.M., Zhou, X. (eds.) DEXA 2012, Part I. LNCS, vol. 7446, pp. 161–175. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  • Atapattu, T., Falkner, K., Falkner, N.: Acquisition of triples of knowledge from lecture notes: a natural language processing approach. In: Proceedings of the 7th International conference on Educational Data Mining, London, United Kingdom (2014)

    Google Scholar 

  • Alves, A., Pereira, F., Cardoso, F.: Automatic reading and learning from text. In: International Symposium on Artificial Intelligence (2002)

    Google Scholar 

  • Ausubel, D., Novak, J., Hanesian, H.: Educational Psychology: A Cognitive View. Holt, Rinehart and Winston, New York (1978)

    Google Scholar 

  • Chen, N., Kinshuk, Wei, C.: Mining e-learning domain concept map from academic articles. Comput. Educ. 50, 1009–1021 (2008)

    Google Scholar 

  • Coffey, J., Carnot, M., Feltovich, P., Feltovich, J., Hoffman, R., Canas, A., Novak, J.: A summary of literature pertaining to the use of concept mapping techniques and technologies for education and performance support. The Chief of Naval Education and Training (2003)

    Google Scholar 

  • Dali, L., Rusu, D., Fortuna, B., Mladenic, D., Grobelnik, M.: Question answering based on semantic graphs. In: Language and Technology Conference, Poznan, Poland (2009)

    Google Scholar 

  • Gouli, E., Gogoulou, A., Papanikolaou, K., Grigoriadou, M.: COMPASS: an adaptive web-based concept map assessment tool. In: Proceedings of the First International Conference on Concept Mapping (2004)

    Google Scholar 

  • Kinchin, I.: Developing PowerPoint handouts to support meaningful learning. Br. J. Educ. Technol. 37(4), 647–650 (2006)

    Article  Google Scholar 

  • Klein, D., Manning, C.: Accurate unlexicalized parsing. In: Proceedings of the 41st Meeting of the Association for Computational Linguistics, pp. 423–430 (2003)

    Google Scholar 

  • Leake, D., Maguitman, A., Reichherzer, T.: Understanding knowledge models: modelling assessment of concept importance in concept maps. In: Proceedings of CogSc (2004)

    Google Scholar 

  • Manning, C., Raghavan, P., Schutze, H.: Introduction to Information Retrieval. Cambridge University Press, Cambridge (2008)

    Book  MATH  Google Scholar 

  • Novak, J., Gowin, D.: Learning How to Learn. Cambridge University Press, New York/Cambridge (1984)

    Book  Google Scholar 

  • Olney, A.M., Graesser, A., Person, N.: Question generation from Concept maps. In: Special issue on Question generation, Dialogue and Discourse (2012)

    Google Scholar 

  • Salton, G., McGill, M.: Introduction to Modern Information Retrieval. McGraw-Hill Inc., New York (1986)

    MATH  Google Scholar 

  • Sleator, D., Temperly, D.: Parsing English with a links grammar. In: Third International Workshop on Parsing Technologies (1993)

    Google Scholar 

  • Villalon, J., Calvo, R.: Concept map mining: a definition and a framework for its evaluation. In: International Conference on Web Intelligence and Intelligent Agent Technology (2008)

    Google Scholar 

  • Zouaq, A., Nkabou, R.: Evaluating the generation of domain ontologies in the knowledge puzzle project. IEEE Trans. Knowl. Data Eng. 21, 1559–1572 (2009)

    Article  Google Scholar 

  • Zouaq, A., Gasevic, D., Hatala, M.: Voting theory for concept detection. In: Simperl, E., Cimiano, P., Polleres, A., Corcho, O., Presutti, V. (eds.) ESWC 2012. LNCS, vol. 7295, pp. 315–329. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to Thushari Atapattu .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Atapattu, T., Falkner, K., Falkner, N. (2015). An Evaluation Methodology for Concept Maps Mined from Lecture Notes: An Educational Perspective. In: Zvacek, S., Restivo, M., Uhomoibhi, J., Helfert, M. (eds) Computer Supported Education. CSEDU 2014. Communications in Computer and Information Science, vol 510. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-25767-9

  • Online ISBN: 978-3-319-25768-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics