Educational Data Mining and Learning Analytics: Applications to Constructionist Research
- 4.4k Downloads
Constructionism can be a powerful framework for teaching complex content to novices. At the core of constructionism is the suggestion that by enabling learners to build creative artifacts that require complex content to function, those learners will have opportunities to learn this content in contextualized, personally meaningful ways. In this paper, we investigate the relevance of a set of approaches broadly called “educational data mining” or “learning analytics” (henceforth, EDM) to help provide a basis for quantitative research on constructionist learning which does not abandon the richness seen as essential by many researchers in that paradigm. We suggest that EDM may have the potential to support research that is meaningful and useful both to researchers working actively in the constructionist tradition but also to wider communities. Finally, we explore potential collaborations between researchers in the EDM and constructionist traditions; such collaborations have the potential to enhance the ability of constructionist researchers to make rich inferences about learning and learners, while providing EDM researchers with many interesting new research questions and challenges.
KeywordsConstructionism Educational data mining Learning analytics Design of learning environments Project-based learning
Berland would like to thank the Complex Play Lab for help with this work, Don Davis for editorial help, and National Science Foundation Awards #SMA-1338508 and #EEC-1331655. Baker would like to thank support from the Bill and Melinda Gates Foundation, Award #OPP1048577, and from the National Science Foundation through the Pittsburgh Science of Learning Center, Award #SBE-0836012. Blikstein would like to thank the National Science Foundation through the CAREER Award #1055130, the AT&T Foundation, and the Lemann Foundation.
- Abelson, H., & diSessa, A. (1986). Turtle geometry: The computer as a medium for exploring mathematics. Cambridge, MA: The MIT Press.Google Scholar
- Amershi, S., & Conati, C. (2009). Combining unsupervised and supervised classification to build user models for exploratory learning environments. Journal of Educational Data Mining, 1(1), 18–71.Google Scholar
- Andersen, E., Liu, Y.-E., Apter, E., Boucher-Genesse, F., & Popovic, Z. (2010). Gameplay analysis through state projection. In Proceedings of the fifth international conference on the foundations of digital games (pp. 1–8). Monterey, California: ACM.Google Scholar
- Arnold, K. E. (2010). Signals: Applying academic analytics. Educause Quarterly, 33(1), n1.Google Scholar
- Baker, E. L., Barton, P. E., Darling-Hammond, L., Haertel, E., Ladd, H. F., Linn, R. L., et al. (2010). Problems with the use of student test scores to evaluate teachers. Washington, DC: Economic Policy Institute.Google Scholar
- Baker, R.S., Corbett, A.T., Koedinger, K. R. (2004). Detecting Student Misuse of Intelligent Tutoring Systems. In Proceedings of the 7th international conference on intelligent tutoring systems (pp. 531–540).Google Scholar
- Baker, R., Corbett, A., Koedinger, K., Evenson, S., Roll, I., Wagner, A., et al. (2006). Adapting to when students game an intelligent tutoring system. In M. Ikeda, K. Ashley, & T.-W. Chan (Eds.), Intelligent tutoring systems, Lecture Notes in Computer Science (Vol. 4053, pp. 392–401). Berlin/Heidelberg: Springer. Retrieved from http://www.springerlink.com.libweb.lib.utsa.edu/content/t3103564632g7n41/abstract/.
- Baker, R., Gowda, S., & Corbett, A. (2011). Towards predicting future transfer of learning. In G. Biswas, S. Bull, J. Kay, & A. Mitrovic (Eds.), Artificial intelligence in education, Lecture Notes in Computer Science (Vol. 6738, pp. 23–30). Berlin: Springer. Retrieved from http://www.springerlink.com.libweb.lib.utsa.edu/content/41325h8k111q0734/abstract/.
- Baker, R., & Siemens, G. (in press). Educational data mining and learning analytics. To appear in Sawyer, K. (Ed.), Cambridge Handbook of the Learning Sciences: 2nd Edition. Cambridge, UK: Cambridge University Press.Google Scholar
- Baker, R., & Yacef, K. (2009). The state of educational data mining in 2009: A review and future visions. Journal of Educational Data Mining, 1(1), 3–17.Google Scholar
- Beck, J. E., & Woolf, B. P. (2000). High-level student modeling with machine learning. In G. Gauthier, C. Frasson, & K. VanLehn (Eds.), Intelligent tutoring systems, Lecture Notes in Computer Science (pp. 584–593). Berlin, Heidelberg: Springer. Retrieved from http://link.springer.com.libweb.lib.utsa.edu/chapter/10.1007/3-540-45108-0_62.
- Black, P., & Wiliam, D. (1998). Inside the black box: Raising standards through classroom assessment. London: King’s College London.Google Scholar
- Blikstein, P. (2009). An atom is known by the company it keeps: Content, representation and pedagogy within the epistemic revolution of the complexity sciences. PhD. dissertation, Northwestern University, Evanston, IL.Google Scholar
- Blikstein, P. (2011). Using learning analytics to assess students’ behavior in open-ended programming tasks. In Proceedings of the I learning analytics knowledge conference (LAK 2011), Banff, Canada.Google Scholar
- Blikstein, P. (2013a). Digital fabrication and ‘making’ in education: The democratization of invention. In J. Walter-Herrmann & C. Büching (Eds.), FabLabs: Of machines, makers and inventors. Bielefeld: Transcript Publishers.Google Scholar
- Blikstein, P. (2013b). Multimodal Learning Analytics. In: Proceedings of the III learning analytics knowledge conference (LAK 2013), Leuven, Belgium.Google Scholar
- Piech, C., Sahami, M., koller, D., Cooper, S., & Blikstein, P. (2012). Modeling how students learn to program. In Proceedings of the 43rd ACM technical symposium on Computer Science Education (pp. 153–160). Raleigh, North Carolina, USA: ACM.Google Scholar
- Conati, C., & Maclaren, H. (2005). Data-driven refinement of a probabilistic model of user affect. In L. Ardissono, P. Brna, & A. Mitrovic (Eds.), User modeling 2005, Lecture Notes in Computer Science (pp. 40–49). Berlin: Springer. Retrieved from http://link.springer.com.libweb.lib.utsa.edu/chapter/10.1007/11527886_7.
- Corbett, A. T., & Anderson, J. R. (1995). Knowledge decomposition and subgoal reification in the ACT programming tutor. In Proceedings of the 7th world conference on artificial intelligence in education.Google Scholar
- Dragon, T., Arroyo, I., Woolf, B. P., Burleson, W., Kaliouby, R. el, & Eydgahi, H. (2008). Viewing student affect and learning through classroom observation and physical sensors. In B. P. Woolf, E. Aïmeur, R. Nkambou, & S. Lajoie (Eds.), Intelligent tutoring systems, Lecture Notes in Computer Science (pp. 29–39). Berlin: Springer. Retrieved from http://link.springer.com.libweb.lib.utsa.edu/chapter/10.1007/978-3-540-69132-7_8.
- Duncan, S., & Berland, M. (2012). Triangulating Learning in Board Games: Computational thinking at multiple scales of analysis. In Proceedings of Games, Learning, & Society 8.0 (pp. 90–95). Madison, WI, USA.Google Scholar
- Dyke, G. (2011). Which aspects of novice programmers’ usage of an IDE predict learning outcomes. In Proceedings of the 42nd ACM technical symposium on computer science education (pp. 505–510). Dallas, TX: ACM.Google Scholar
- Eisenberg, M. (2011). Educational fabrication, in and out of the classroom. Society for Information Technology & Teacher Education International Conference, 2011(1), 884–891.Google Scholar
- Fosnot, C. T. (2005). Constructivism: Theory, perspectives, and practice. New York: Teachers College Press.Google Scholar
- Harel, I. (1990). Children as software designers: A constructionist approach for learning mathematics. Journal of Mathematical Behavior, 9(1), 3–93.Google Scholar
- IEDMS. (2009). International Educational Data Mining Society. Retrieved April 22, 2013, from http://www.educationaldatamining.org/.
- Jeong, H., Biswas, G., Johnson, J., & Howard, L. (2010). Analysis of productive learning behaviors in a structured inquiry cycle using hidden markov models. Manuscript submitted for publication.Google Scholar
- Levy, F., & Murnane, R. J. (2005). The new division of labor: How computers are creating the next job market. Princeton University Press.Google Scholar
- Liu, Y.-E., Andersen, E., Snider, R., Cooper, S., & Popović, Z. (2011). Feature-based projections for effective playtrace analysis. In Proceedings of the 6th international conference on foundations of digital games (pp. 69–76). Bordeaux, France: ACM.Google Scholar
- Lynch, C., Ashley, K., Pinkwart, N., & Aleven, V. (2008). Argument graph classification with Genetic Programming and C4.5 (pp. 137–146). Presented at the educational data mining 2008: 1st international conference on educational data mining, Proceedings, Montreal, Quebec, Canada.Google Scholar
- Merceron, A., & Yacef, K. (2004). Mining student data captured from a web-based tutoring tool: Initial exploration and results. Journal of Interactive Learning Research, 15(4), 319–346.Google Scholar
- Mitchell, T. M. (1997). Machine learning. New York: McGraw-Hill.Google Scholar
- Muehlenbrock, M. (2005). Automatic action analysis in an interactive learning environment. In Proceedings of the workshop on usage analysis in learning systems at the 12th international conference on artificial intelligence in education AIED 2005 (pp. 73–80).Google Scholar
- NGSS Lead States. (2013). Next generation science standards: For states, by states. Washington, DC: The National Academies Press.Google Scholar
- Ocumpaugh, J., Baker, R., Gowda, S., Heffernan, N., & Heffernan, C. (2014). Population validity for educational data Mining models: A case study in affect detection. British Journal of Educational Technology, 45(3), 487–501.Google Scholar
- Papert, S. (1972). Teaching children to be mathematicians versus teaching about mathematics. International Journal of Mathematical Education in Science and Technology. Retrieved from http://www.informaworld.com/index/746865236.pdf.
- Papert, S. (1980). Mindstorms: Children, computers, and powerful ideas. NYC: Basic Books.Google Scholar
- Papert, S. (2000). What’s the big idea? Toward a pedagogy of idea power. IBM Systems Journal. Retrieved from http://llk.media.mit.edu/courses/readings/Papert-Big-Idea.pdf.
- Papert, S., & Harel, I. (1991). Situating constructionism. Constructionism. Retrieved from http://namodemello.com.br/pdf/tendencias/situatingconstrutivism.pdf.
- Piech, C., Sahami, M., Koller, D., Cooper, S., & Blikstein, P. (2012). Modeling how students learn to program. In Proceedings of the 43rd ACM technical symposium on computer science education, Raleigh, North Carolina, USA.Google Scholar
- Roll, I., Aleven, V., McLaren, B. M., & Koedinger, K. R. (2011). Improving students’ help-seeking skills using metacognitive feedback in an intelligent tutoring system. Special Section I: Solving information-based problems: Evaluating sources and information Special Section II: Stretching the limits in help-seeking research: Theoretical, methodological, and technological advances, 21(2), 267–280. doi: 10.1016/j.learninstruc.2010.07.004.Google Scholar
- Romero, C., & Ventura, S. (2010). Educational data mining: a review of the state of the art. Systems, Man, and Cybernetics, Part C: Applications and Reviews, IEEE Transactions on, 40(6), 601–618.Google Scholar
- Sao Pedro, M. A. S., Baker, R. S. J. D., & Gobert, J. D. (2012). Improving construct validity yields better models of systematic inquiry, even with less information. In J. Masthoff, B. Mobasher, M. C. Desmarais, & R. Nkambou (Eds.), User modeling, adaptation, and personalization, Lecture Notes in Computer Science (pp. 249–260). Berlin, Heidelberg: Springer. Retrieved from http://link.springer.com.libweb.lib.utsa.edu/chapter/10.1007/978-3-642-31454-4_21.
- Sao Pedro, M. A. S., Gobert, J. D., & Raziuddin, J. J. (2010). Comparing pedagogical approaches for the acquisition and long-term robustness of the control of variables strategy. In Proceedings of the 9th international conference of the learning sciences (Vol. 1, pp. 1024–1031). Chicago, IL: International Society of the Learning Sciences.Google Scholar
- Schneider, B. & Blikstein, P. (2014). Unraveling students' interaction around a tangible interface using multimodal learning analytics. In Proceedings of the 7th international conference on educational data mining. London, UK.Google Scholar
- Sherin, B. (2012). Using computational methods to discover student science conceptions in interview data. In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 188–197). ACM.Google Scholar
- Sherin, B. (under review). A computational study of commonsense science: An exploration in the automated analysis of clinical interview data.Google Scholar
- Stamper, J., Eagle, M., Barnes, T., & Croy, M. (2011). Experimental evaluation of automatic hint generation for a logic tutor. In G. Biswas, S. Bull, J. Kay, & A. Mitrovic (Eds.), Artificial intelligence in education, Lecture Notes in Computer Science (Vol. 6738, pp. 345–352). Berlin/Heidelberg: Springer. Retrieved from http://www.springerlink.com.libweb.lib.utsa.edu/content/f0w1t2642k4t6128/abstract/.
- Stevens, R., Satwicz, T., & McCarthy, L. (2008). In-game, in-room, in-world: Reconnecting video game play to the rest of kids’ lives. The ecology of games: Connecting youth, games, and learning, 9, 41–66.Google Scholar
- Tabanao, E. S., Rodrigo, M. M. T., & Jadud, M. C. (2011). Predicting at-risk novice Java programmers through the analysis of online protocols. In Proceedings of the seventh international workshop on computing education research (pp. 85–92). Providence, Rhode Island, USA: ACM.Google Scholar
- Vee, M. H. N. C., Meyer, B., & Mannock, K. L. (2006). Understanding novice errors and error paths in Object-oriented programming through log analysis. In Proceedings of workshop on educational data mining at the 8th international conference on intelligent tutoring systems (ITS 2006) (pp. 13–20).Google Scholar
- Walonoski, J.A., Heffernan, N.T. (2006): Prevention of off-task gaming behavior in intelligent tutoring systems. In Proceedings of the 8th International Conference on Intelligent Tutoring Systems (pp. 722–724). Berlin: Springer.Google Scholar
- Wilensky, U. (1996). Making sense of probability through paradox and programming: A case study of connected mathematics. In Y. Kafai & M. Resnick (Eds.), Constructionism in practice (pp. 269–296). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
- Wilensky, U. (1999). NetLogo. Retrieved from http://ccl.northwestern.edu/netlogo.
- Witten, I. H., Frank, E., & Hall, M. A. (2011). Data mining: Practical machine learning tools and techniques. Morgan Kaufmann.Google Scholar
- Worsley, M. (2012). Multimodal learning analytics: Enabling the future of learning through multimodal data analysis and interfaces. In Proceedings of the international conference on multimodal interfaces, Santa Monica, CA.Google Scholar
- Worsley, M., & Blikstein, P. (2011). What’s an Expert? Using learning analytics to identify emergent markers of expertise through automated speech, sentiment and sketch analysis. In Proceedings for the 4th annual conference on educational data mining.Google Scholar
- Worsley, M., & Blikstein, P. (2012). An Eye For Detail: Techniques for using eye tracker data to explore learning in computer-mediated environments. In Proceedings of the international conference of the learning sciences (pp. 561–562). Sydney, Australia.Google Scholar
- Worsley, M., & Blikstein, P. (2013). Towards the Development of Multimodal Action Based Assessment. In Proceedings of the III learning analytics knowledge conference (LAK 2013), Leuven, Belgium.Google Scholar
- Worsley, M., & Blikstein, P. (in press). Analyzing engineering design through the lens of learning analytics. Journal of Learning Analytics.Google Scholar
- Yu, L., & Liu, H. (2004). Efficient feature selection via analysis of relevance and redundancy. The Journal of Machine Learning Research, 5, 1205–1224.Google Scholar
- Zahn, C., Krauskopf, K., Hesse, F. W., & Pea, R. (2010). Digital video tools in the classroom: Empirical studies on constructivist learning with audio-visual media in the domain of history. In Proceedings of the 9th international conference of the learning sciences (Vol. 1, pp. 620–627). Chicago, Illinois: International Society of the Learning Sciences.Google Scholar