Technology, Knowledge and Learning

, Volume 24, Issue 2, pp 161–168 | Cite as

Overcoming the PBL Assessment Challenge: Design and Development of the Incremental Thesaurus for Assessing Causal Maps (ITACM)

  • Philippe J. Giabbanelli
  • Andrew A. TawfikEmail author
Original research


Because of the lack of tools available to assess problem-solving skills, teachers often revert to more traditional instructional approaches (e.g. lecture-based, memorization) that fail to prepare learners for the complexity of dynamic work environments. To overcome this challenge, technology solutions are needed that accurately and efficiently assess complex problem-solving skills such as causal reasoning. Moreover, these tools must be valid and reliable so instructors can accurately assess student learning. This emergent report details the design and development of Incremental Thesaurus for Assessing Causal Maps. As will be described, the software offers three unique features: (a) analysis of causal map with little or no manipulation of the original file; (b) a growing repository of terms that supports efficient assessment and (c) ability to codify the level of concept complexity using the structure–behavior–function framework.


Problem-based learning Assessment Causal reasoning Ill-structured problem solving 


  1. Bastien, M., Heymann, S., & Jacomy, M. (2009) Gephi: An open source software for exploring and manipulating networks. In Proceedings of the international AAAI conference on weblogs and social media.Google Scholar
  2. Baur, M., & Benkert, M. (2005). Network comparison. In U. Brandes & T. Erlebach (Eds.), Network analysis (pp. 318–340). Berlin: Springer.Google Scholar
  3. Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, 18(1), 32–42.Google Scholar
  4. Chi, M., Feltovich, P., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5(2), 121–152.Google Scholar
  5. Dickinson, P. J., Bunke, H., Dadej, A., & Kraetzl, M. (2003). On graphs with unique node labels. In E. Hancock & M. Vento (Eds.), Graph based representations in pattern recognition (pp. 13–23). Berlin: Springer.Google Scholar
  6. Ertmer, P. (1999). Addressing first- and second-order barriers to change: Strategies for technology integration. Educational Technology Research and Development, 47(4), 47–61.Google Scholar
  7. Ertmer, P., & Ottenbreit-Leftwich, A. (2013). Removing obstacles to the pedagogical changes required by Jonassen’s vision of authentic technology-enabled learning. Computers & Education, 64, 175–182.Google Scholar
  8. Eseryel, D., Ifenthaler, D., & Ge, X. (2013). Validation study of a method for assessing complex ill-structured problem solving by using causal representations. Educational Technology Research and Development, 61(3), 443–463.Google Scholar
  9. Feltovich, P. J., Spiro, R. J., & Coulson, R. L. (1997). Issues of expert flexibility in contexts characterized by complexity and change. In P. Feltovich, K. Fork, & R. Hoffman (Eds.), Expertise in context: Human and machine (pp. 125–146). Cambridge, MA: MIT Press.Google Scholar
  10. Giabbanelli, P. J., & Baniukiewicz, M. (2018). Navigating complex systems for policymaking using simple software tools. In V. Mago, P. J. Giabbanelli, & E. Papageorgiou (Eds.), Advanced data analytics in health. New York, NY: Springer.Google Scholar
  11. Hmelo-Silver, C. (2013). Creating a learning space in problem-based learning. Interdisciplinary Journal of Problem-Based Learning. doi: 10.7771/1541-5015.1334.Google Scholar
  12. Hmelo-Silver, C. E., Marathe, S., & Liu, L. (2007). Fish swim, rocks sit, and lungs breathe: Expert-novice understanding of complex systems. Journal of the Learning Sciences, 16(3), 307–331.Google Scholar
  13. Hmelo-Silver, C. E., Nagarajan, A., & Day, R. S. (2002). “It’s harder than we thought it would be”: A comparative case study of expert–novice experimentation strategies. Science Education, 86(2), 219–243.Google Scholar
  14. Hmelo-Silver, C. E., & Pfeffer, M. G. (2004). Comparing expert and novice understanding of a complex system from the perspective of structures, behaviors, and functions. Cognitive Science, 28(1), 127–138.Google Scholar
  15. Hung, W. (2011). Theory to reality: A few issues in implementing problem-based learning. Educational Technology Research and Development, 59(4), 529–552.Google Scholar
  16. Hung, W. (2016). All PBL starts here: The problem. Interdisciplinary Journal of Problem-Based Learning, 10(2), 2.Google Scholar
  17. Ifenthaler, D., & Eseryel, D. (2013). Facilitating complex learning by mobile augmented reality learning environments. In R. Huang, Kinshuk, & J. M. Spector (Eds.), Reshaping learning (pp. 415–438). Berlin: Springer.Google Scholar
  18. Ifenthaler, D., & Pirnay-Dummer, P. (2014). Model-based tools for knowledge assessment. In J. M. Spector, M. D. Merrill, J. Elen, & M. J. Bishop (Eds.), Handbook of research on educational communications and technology (4th ed., pp. 289–301). New York, NY: Springer.Google Scholar
  19. Jacobson, M. J. (2001). Problem solving, cognition, and complex systems: Differences between experts and novices. Complexity, 6(3), 41–49.Google Scholar
  20. Jeong, A., & Lee, W. J. (2012). Developing causal understanding with causal maps: The impact of total links, temporal flow, and lateral position of outcome nodes. Educational Technology Research and Development, 60(2), 325–340.Google Scholar
  21. Jonassen, D. H. (2000). Toward a design theory of problem solving. Educational Technology, Research and Development, 48(4), 63–85.Google Scholar
  22. Jonassen, D. H. (2011). Learning to solve problems: A handbook for designing problem-solving learning environments (1st ed.). London: Routledge.Google Scholar
  23. Jonassen, D. H., & Hung, W. (2006). Learning to troubleshoot: A new theory-based design architecture. Educational Psychology Review, 18(1), 77–114.Google Scholar
  24. Kolodner, J. (1991). Improving human decision making through case-based decision aiding. AI Magazine, 12(2), 52–68.Google Scholar
  25. Kolodner, J., Hmelo-Silver, C., & Narayanan, N. H. (1996). Problem-based learning meets case-based reasoning. In Proceedings of the 1996 international conference on learning sciences (pp. 188–195). Evanston, Illinois: International Society of the Learning Sciences.Google Scholar
  26. Moon, B., Hoffman, R. R., Novak, J., & Canas, A. (2011). Applied concept mapping. Boca Raton: CRC Press.Google Scholar
  27. Savin-Baden, M. (2004). Understanding the impact of assessment on students in problem-based learning. Innovations in Education and Teaching International, 41(2), 221–233.Google Scholar
  28. Senocak, E. (2009). Development of an instrument for assessing undergraduate science students’ perceptions: The problem-based learning environment inventory. Journal of Science Education and Technology, 18(6), 560–569.Google Scholar
  29. Tamim, S., & Grant, M. (2013). Definitions and uses: Case study of teachers implementing project-based learning. Interdisciplinary Journal of Problem-Based Learning. doi: 10.7771/1541-5015.1323.Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2017

Authors and Affiliations

  1. 1.Department of Computer ScienceNorthern Illinois UniversityDekalbUSA
  2. 2.Department of Instructional Design & TechnologyUniversity of MemphisMemphisUSA

Personalised recommendations