Skip to main content
SpringerLink
Log in

Eliciting, Representing, and Evaluating Adult Knowledge: A Model for Organizational Use of Concept Mapping and Concept Maps

  • Conference paper
  • First Online:
Innovating with Concept Mapping (CMC 2016)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 635))

Included in the following conference series:

Abstract

After nearly two decades of knowledge preservation activity, relatively little work has explored the organizational use of Concept Maps elicited from experts. This paper describes an attempt to get back to the roots of Concept Mapping as a means of both representing and evaluating knowledge, in the context of professional work. It describes a pilot project in which the authors used Concept Mapping to elicit and represent knowledge from domain experts, then demonstrated the use of Concept Maps for assessing the mental models of other professionals. The authors introduce Sero! – a prototype Concept Map-based learning assessment platform, and a general model for the organizational use of Concept Mapping and Concept Maps.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Novak, J.D., Cañas, A.J.: The Theory Underlying Concept Maps and How to Construct and Use Them. Institute for Human and Machine Cognition, Pensacola (2008)

    Google Scholar 

  2. Moon, B., Hoffman, R.R., Novak, J.D., Cañas, A.J. (eds.): Applied Concept Mapping: Capturing, Analyzing, and Organizing Knowledge. CRC Press, New York (2011)

    Google Scholar 

  3. Moon, B., Hoffman, R.R., Ziebell, D.: How did you do that. Electr. Perspect. 34(1), 20–29 (2009)

    Google Scholar 

  4. Ziebell, D., Hoffman, R.R., Feltovich, P., Klein, G., Fiore, S., Moon, B.: Accelerating the Achievement of Mission-Critical Expertise: A Research Roadmap. EPRI, Palo Alto (2008)

    Google Scholar 

  5. Schafermeyer, R.G., Hoffman, R.R.: Using knowledge libraries to transfer expert knowledge. IEEE Intell. Syst. 31(2), 89–93 (2016)

    Article  Google Scholar 

  6. Moon, B.M., Baxter, H.C., Klein, G.: Expertise management: challenges for adopting naturalistic decision making as a knowledge management paradigm. McLean (2015)

    Google Scholar 

  7. Hayutin, A., Beals, M., Borges, E.: The Aging us Workforce: A Chartbook of Demographic Shifts, vol. 7. Stanford Center on Longevity, Stanford (2013). http://library.constantcontact.com/download/get/file/1102783429573-323/The+Aging+US+Workforce

    Google Scholar 

  8. DeLong, D.W.: Lost Knowledge: Confronting the Threat of an Aging Workforce. Oxford University Press, New York (2004)

    Book  Google Scholar 

  9. Crandall, B., Klein, G.A., Hoffman, R.R.: Working Minds: A Practitioner’s Guide to Cognitive Task Analysis. MIT Press, Cambridge (2006)

    Google Scholar 

  10. Electric Power Research Institute: Guidelines for Capturing Valuable Undocumented Knowledge from Energy Industry Personnel, Palo Alto (2002)

    Google Scholar 

  11. Kelley, M., Sass, M., Moon, B.: Maturity of a nuclear-related knowledge management solution. In: Annual Meeting of the American Nuclear Society (2013)

    Google Scholar 

  12. Moon, B., Kelley, M.: Lessons learned in knowledge elicitation with nuclear experts. In: 7th International Topical Meeting on Nuclear Plant Instrumentation, Control and Human Machine Interface Technologies, Las Vegas (2010)

    Google Scholar 

  13. Hoffman, R., Moon, B.: Knowledge capture for the utilities. In: 7th International Topical Meeting on Nuclear Plant Instrumentation, Control and Human Machine Interface Technologies (2010)

    Google Scholar 

  14. Vaughan, K.: Workplace Learning: A Literature Review. Competenz, Auckland (2008)

    Google Scholar 

  15. Vaughan, K., Cameron, M.: Assessment of Learning in the Workplace: A Background Paper. Industry Training Federation, Wellington (2009)

    Google Scholar 

  16. Weber, L.: Today’s personality tests raise the bar for job seekers. Wall Str. J. (2015)

    Google Scholar 

  17. Bound, H., Lin, M.: Singapore Workforce Skills Qualifications (WSQ), Workplace Learning and Assessment (Stage I), p. 26. Institute of Adult Learning, Singapore (2010)

    Google Scholar 

  18. Hoffman, R.R., Ward, P., Feltovich, P.J., DiBello, L., Fiore, S.M., Andrews, D.H.: Accelerated Expertise: Training for High Proficiency in a Complex World. Psychology Press, New York (2013)

    Google Scholar 

  19. Bound, H., Lin, M.: Singapore Workforce Skills Qualifications (WSQ), Workplace Learning and Assessment (Stage I), p. 25. Institute of Adult Learning, Singapore (2010)

    Google Scholar 

  20. Vaughan, K., Cameron, M.: Assessment of Learning in the Workplace: A Background Paper, p. 5. Industry Training Federation, Wellington (2009)

    Google Scholar 

  21. Vaughan, K., Cameron, M.: Assessment of Learning in the Workplace: A Background Paper, p. 9. Industry Training Federation, New Zealand, Wellington (2009)

    Google Scholar 

  22. Moon, B., Johnston, C., Tuxbury, B., Hoffman, R.R., Guarino, S., Jarvis, R., Young, D., Romero, V.: Sero!: evaluating a concept mapping game for its potential to improve cognitive capabilities. In: Proceedings of the Sixth International Conference on Concept Mapping, Sao Paulo (2014)

    Google Scholar 

  23. Ruiz-Primo, M.A., Shavelson, R.R.: Problems and issues in the use of concept maps in science assessment. J. Res. Sci. Teach. 33(6), 569–600 (1996). Wiley, Hoboken

    Article  Google Scholar 

  24. Shavelson, R.J., Ruiz-Primo, M.A.: On the Psychometrics of Assessing Science Understanding. Assessing Science Understanding, pp. 303–341. Academic Press, Cambridge (2000)

    Google Scholar 

  25. Cooke, N.J., Schvaneveldt, R.W.: Effects of computer programming experience on network representations of abstract programming concepts. Int. J. Man-Mach. Stud. 29(4), 407–427 (1988). Elsevier Ltd, Houston

    Article  Google Scholar 

  26. WestEd, Council of Chief State Officers: Science Framework for the 2009/2011, p. 110. NAEP (2010)

    Google Scholar 

  27. WestEd, CCSO: Science Assessment and Item Specifications for the 2009, p. 171. NAEP (2007)

    Google Scholar 

  28. National Assessment Governing Board: Science Framework for the 2015 NAEP (2015)

    Google Scholar 

  29. Stevens, P.A.: Using concept maps for assessing adult learners in training situations. Ph.D. dissertation, Harvard, Cambridge

    Google Scholar 

  30. Moon, B., Ross, K., Phillips, J.: Cmap-based assessment for adult learners. In: Proceedings of the Fourth International Conference on Concept Mapping, Chile (2010)

    Google Scholar 

  31. Knollmann-Ritschel, B.E.C., Durning, S.J.: Using concept maps in a modified team-based learning exercise. Mil. Med. 180(4 Suppl.), 64–70 (2015). The Society of the Federal Health Professionals, Bethesda

    Article  Google Scholar 

  32. Strautmane, M.: Cmap-based knowledge assessment tasks and their scoring criteria: an overview. In: Proceedings of the Fifth International Conference on Concept Mapping, Valletta (2012)

    Google Scholar 

  33. Sas, J.C.: The Multiple-Choice Concept Map (MCCM): An Interactive Computer-Cased Assessment Method. Digital Scholarship@UNLV, Las Vegas (2010)

    Google Scholar 

  34. Ayala, C.C., Shavelson, R.J., Ruiz-Primo, M.A., Brandon, P.R., Yin, Y., Furtak, E.M., Young, D.B., Tomita, M.K.: From formal embedded assessments to reflective lessons: the development of formative assessment studies. Appl. Measur. Educ. 21(4), 315–334 (2008)

    Article  Google Scholar 

  35. McClure, J.R., Sonak, B., Suen, H.K.: Concept map assessment of classroom learning: reliability, validity, and logistical practicality. J. Res. Sci. Teach. 36(4), 475–492 (1999)

    Article  Google Scholar 

  36. Schau, C., Mattern, N., Weber, R.W.: Use of fill-in concept maps to assess middle school students’ connected understanding of science. Chicago (1997)

    Google Scholar 

  37. Ruiz-Primo, M.A.: Examining concept maps as an assessment tool. In: Proceedings of the Fist International Conference on Concept Mapping, Universidad Pública de Navarra, Pamplona, pp. 555–563 (2004)

    Google Scholar 

  38. Besterfield-Sacre, M., Gerchak, J., Lyons, M.R., Shuman, L.J., Wolfe, J.: Scoring concept maps: an integrated rubric for assessing engineering education. J. Eng. Educ. 93(2), 105–115 (2004). Pennsylvania

    Article  Google Scholar 

  39. Ruiz-Primo, M.A., Schultz, S.E., Li, M., Shavelson, R.J.: Comparison of the reliability and validity of scores from two concept-mapping techniques. J. Res. Sci. Teach. 38(2), 260–278 (2001). Wiley, Hoboken

    Article  Google Scholar 

  40. Himangshu, S., Cassata-Widera, A.: Beyond individual classrooms: how valid are concept maps for large scale assessment? In: Proceedings of the 4th International Conference on Concept Mapping, Universidad de Chile, Viña del Mar (2010)

    Google Scholar 

  41. Cañas, A.J., Bunch, L., Novak, J.D., Reiska, P.: CmapAnalysis: an extensible concept map analysis tool. J. Educ. Teach. Train. 1, 36–46 (2013). Universidad de Granada, Granada

    Google Scholar 

  42. Liu, J.: The assessment agent system: assessing comprehensive understanding based on concept maps. Ph.D. dissertation, Virginia Polytechnic Institute and State University, Blacksburg (2010)

    Google Scholar 

  43. Strautmane, M.: Cmap-based knowledge assessment tasks and their scoring criteria: an overview. In: Proceedings of the 5th International Conference on Concept Mapping, Valletta, Malta (2012)

    Google Scholar 

  44. Crandall, B., Klein, G.A., Hoffman, R.R.: Working Minds: A Practitioner’s Guide to Cognitive Task Analysis, pp. 55–66. MIT Press, Cambridge (2006)

    Google Scholar 

  45. Moon, B., Hoffman, R.R., Eskridge, T., Coffey, J.: Skills in concept mapping. In: Moon, B., Hoffman, R.R., Novak, J.D., Cañas, A.J. (eds.) Applied Concept Mapping: Capturing, Analyzing, and Organizing Knowledge, New York, pp. 23–46 (2011)

    Google Scholar 

  46. Hoffman, R.R., Coffey, J.W., Carnot, M.J., Novak, J.D.: An empirical comparison of methods for eliciting and modeling expert knowledge. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 46, no. 3, pp. 482–486. SAGE Publications (2002)

    Google Scholar 

  47. ReliabilityFirst: 2015 RF Regional Risk Assessment Report (2015)

    Google Scholar 

  48. Lintern, G., Moon, B., Klein, G.A., Hoffman, R.R.: Eliciting and representing the knowledge of the expert practitioner. In: Ericsson, K. et al. (eds.) Cambridge Handbook of Expertise and Expert Performance, 2nd edn., Cambridge (In press)

    Google Scholar 

  49. Cañas, A.J., Hill, G., Carff, R., Suri, N., Lott, J., Eskridge, T., Gómez, G., Arroyo, M., Carvajal, R.: CmapTools: a knowledge modeling and sharing environment. concept maps: theory, methodology, technology. In: Proceedings of the First International Conference on Concept Mapping, Pamplona, vol. 1 (2004)

    Google Scholar 

  50. Ausubel, D.P.: The Psychology of Meaningful Learning; An Introduction to School Learning, p. vi. Grune and Stratton, New York (1968)

    Google Scholar 

  51. Kinchin, I.M: The qualitative analysis of concept maps: some unforeseen consequences and emerging opportunities. In: Keynote Address at the Third International Conference on Concept Mapping, Tallinn (2008)

    Google Scholar 

Download references

Acknowledgements

Thanks to ReliabilityFirst for supporting this effort, and the ReliabilityFirst experts and their managers for making time available for our demonstration.


Author information

Authors and Affiliations

  1. Perigean Technologies LLC, Spotsylvania, VA, USA

    Brian Moon, Charles Johnston & Sana Rizvi

  2. ReliabilityFirst Corporation, Independence, OH, USA

    Carl Dister

Authors
  1. Brian Moon
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Charles Johnston
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sana Rizvi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Carl Dister
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Brian Moon .

Editor information

Editors and Affiliations

  1. Inst. for Human and Machine Cognition , Pensacola, Florida, USA

    Alberto Cañas

  2. Tallinn University , Tallinn, Estonia

    Priit Reiska

  3. Inst. for Human and Machine Cognition , Pensacola, Florida, USA

    Joseph Novak

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Moon, B., Johnston, C., Rizvi, S., Dister, C. (2016). Eliciting, Representing, and Evaluating Adult Knowledge: A Model for Organizational Use of Concept Mapping and Concept Maps. In: Cañas, A., Reiska, P., Novak, J. (eds) Innovating with Concept Mapping. CMC 2016. Communications in Computer and Information Science, vol 635. Springer, Cham. https://doi.org/10.1007/978-3-319-45501-3_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-45501-3_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-45500-6

  • Online ISBN: 978-3-319-45501-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation