Abstract
The applications of learning outcomes and competency frameworks have brought better clarity to engineering programs in many universities. Several frameworks have been proposed to integrate outcomes and competencies into course design, delivery and assessment. However, in many cases, competencies are course-specific and their overall impact on the curriculum design is unknown. Such impact analysis is important for analysing, discovering gaps and improving the curriculum design. Unfortunately, manual analysis is a painstaking process due to large amounts of competencies across the curriculum. In this paper, we propose an automated method to analyse the competencies and discover their impact on the overall curriculum design. We provide a principled methodology for discovering the impact of courses’ competencies using Bloom’s Taxonomy, Dreyfus’ model and the learning outcomes framework. We developed the Curriculum Analytics Tool (CAT) which generates the competency scores for the entire curriculum across two dimensions; Cognitive levels and Progression levels. We use the CAT to analyse the competencies of an undergraduate Information Systems Management core curriculum program. Using 14 courses and the corresponding 578 competencies, this paper shows how our method enables us to perform in-depth analysis on the curriculum by discovering the cognition and progression statistics. We further apply the tool for recommending competencies when launching new courses.
Similar content being viewed by others
References
Baker, R. S. J.d., & Yacef, K. (2009). The state of educational data mining in 2009: A Review and Future Visions. Journal of Educational Data Mining, 1(1), 3–17.
Baumgartner, I., & Shankararaman, V. (2013). Actively linking learning outcomes and competencies to course design and delivery: experiences from an undergraduate information systems program in Singapore. In IEEE global engineering education conference (EDUCON 2013). Germany: Berlin.
Bloom, B. S., Engelhart, M. D., Furst, E. J., Hilland, W. H., & Krathwohl, D. R. (1956). Taxonomy of educational objectives: handbook I: cognitive domain. New York: David McKay, 19, 56.
Brabrand, C., & Dahl, B. (2009). Analyzing CS competencies using the SOLO taxonomy. New York: In the Proceedings of the 14th annual ACM SIGCSE conference on Innovation and technology in computer science education.
Burr, L., & Spennemann, D. H. (2004). Pattern of user behavior in university online forums. In International Journal of Instructional Technology and Distance Learning, 1(10), 11–28.
Cook, D., & Swayne, D. F. (2007). Interactive and dynamic graphics for data analysis: with R and GGobi. Springer.
Dreyfus, H. L., & Dreyfus, S. E. (1986). Mind over machine: the power of human intuition and experience in the era of the computer. Oxford: Basil Blackwell.
Ducrot, J., Miller, S., & Goodman, P. S. (2008). Learning outcomes for a business information systems undergraduate program. Communications of the Association for Information Systems, 23, 6.
EU. (2014). The European qualifications framework. European Union Education and Culture DG.
Feiman-Nemser, S. (1990). Teacher preparation: structural and conceptual alternatives. In W. R. Houston (Ed.), Handbook of Research on Teacher Education (pp. 212–229). New York: McMillan.
Gnana Singh, A. A., & Leavline, E. J. (2013). Competency-based calisthenics of learning outcomes for engineering education. International Journal of Education and Learning, 2(1), 25–34.
Gottipati, S., & Shankararaman, V. (2014a). Learning analytics applied to curriculum analysis. AIS SIGED 2014 Proceedings. 2. http://aisel.aisnet.org/siged2014/2
Gottipati, S., & Shankararaman, V. (2014b). Analyzing course competencies: What can competencies reveal about the curriculum?. Proceedings of the 22nd International conference on computers in education ICCE 2014: November 30–December 4, 2014, Nara, Japan. 319–324. Research collection school of information systems.
Hartel, R. W., & Foegeding, E. A. (2004). Learning: objectives, competencies, or outcomes. Journal of Food Science Education, 3, 69–70.
Jin, H., Wu, T., Liu, Z., & Yan, J. (2009). Application of visual data Mining in Higher-Education Evaluation System (pp. 101–104). Washington, DC: In International Workshop on Education Technology and Computer Science.
Judith, G. C., Ramiah, K., Weist, E. M. G., & Shortell, S. M. (2008). Development of a Core competency model for the master of public health degree. American Journal of Public Health, 98(9), 1598–1607.
Kennedy, D. (2007). Writing and using learning outcomes: a practical guide. Cork: Quality Promotion Unit, University College Cork. Internet address http://www.nairtl.ie/.
Kennedy, D., Hyland A., Ryan N. (2009). Learning outcomes and competences: introducing bologna objectives and tools, B 2.3–3, 1–18.
Khairuddin, N.N., Khairuddin, H. (2008). Application of Bloom's taxonomy in software engineering assessments. In Proceedings of the 8th conference on applied computer science (ACS'08), Subhas C. Misra, Roberto Revetria, Les M. Sztandera, Mihaiela Iliescu, Azami Zaharim, and Hamed Parsiani (Eds.). World scientific and engineering academy and society (WSEAS), Stevens point, Wisconsin, USA, pp 66–69.
Krathwohl, D. R. (2002). A revision of Bloom's Taxonomy: an overview. Benjamin S. Bloom, University of Chicago. Theory Into Practice, 42(4), 216.
Lister, R., & Leaney, J. (2003). Introductory programming, criterion-referencing, and bloom. In Proceedings of the 34th SIGCSE technical symposium on computer science education (pp. 143–147). New York: ACM.
Mazza, R., & Milani, C. (2004). GISMO: a graphical interactive student monitoring tool for course management systems. In In international conference on technology enhanced learning, Milan (pp. 1–8).
Mostow, J., Beck, J., Cen, H., Cuneo, A., Gouvea, E., Heiner, C. (2005). An educational data mining tool to browse tutor-student interactions: Time will tell! In Proceedings of the Workshop on Educational Data Mining. 15–22.
O’Neill, G., & Murphy, F. (2010). Guide to taxonomies of learning. In UCD teaching and learning/resources. Available http://www.ucd.ie/t4cms/ucdtla0034.pdf
Passow, H. (2012). Which ABET competencies do engineering graduates find most important in their work? Journal of Engineering Education, 101(1), 95–118.
Proli, D., & Dondi, C. (2011). Analysis of learning outcomes approach implementation in European higher education. Project number: 167178-LLP-1-2009-ES-KA1-KA1EQF, European Union Education and Culture DG.
Raykova, M., Kostadinova, H., & Totkov, G. (2011). Adaptive test system based on revised Bloom's taxonomy. In Proceedings of the 12th International Conference on Computer Systems and Technologies. ACM, New York: pp. 504–509.
Rodolfa, E. R., Bent, R. J., Eisman, E., Nelson, P. D., Rehm, L., & Ritchie, P. (2005). A cube model for competency development: implications for psychology educators and regulators. Professional Psychology: Research and Practice, 36, 347–354.
Romero, C., Gutierrez, S., Freire, M., & Ventura, S. (2008). Mining and visualizing visited trails in web-based educational systems (pp. 182–185). Montreal, Canada: In international conference on educational data mining.
Scott, T. (2003). Bloom's taxonomy applied to testing in computer science classes. Journal Computer Small Colloid, 19(1), 267–274.
Shankararaman, V., & Ducrot, J. (2016). Leveraging competency framework to improve teaching and learning: a methodological approach. Journal of Education and Information Technologies, 21, 1299.
Shen, R., Yang, F., & Han, P. (2002). Data analysis center based on e-learning platform. In Workshop the internet challenge (pp. 19–28). Berlin, Germany: Technology and applications.
Vignan, S., Senthilkumar, G., Marthi, N., & Goteti, P. (2011). Integrating learning outcomes and Bloom's taxonomy in web application development course: Experiences from corporate training. In the Proceedings of IEEE Students' Technology Symposium (TechSym), January, Kharagpur, India, pp 11–16.
Wheeler, D. (2007). Using a summative assessment alignment model and the revised Bloom's taxonomy to improve curriculum development, instruction and evaluation. Doctoral dissertation: Syracuse University.
Whetten, D.A. (2007). Principles of effective course design: What I wish I had known about Learning-Centered Teaching 30 Years Ago. Journal of Management Education 31(3), 339–345, pp. 347–357.
Yoo, J., Yoo, S., Lance, C., Hankins, J. (2006). Student progress monitoring tool using treeview. In proceedings of the 37th Technical Symposium on Computer Science Education, SIGCSE’06. ACM Press. March 1–5, Houston: 373–377.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Gottipati, S., Shankararaman, V. Competency analytics tool: Analyzing curriculum using course competencies. Educ Inf Technol 23, 41–60 (2018). https://doi.org/10.1007/s10639-017-9584-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10639-017-9584-3