Another 25 Years of AIED? Challenges and Opportunities for Intelligent Educational Technologies of the Future
- 1.6k Downloads
This paper attempts an analysis of some current trends and future developments in computer science, education, and educational technology. Based on these trends, two possible future predictions of AIED are presented in the form of a utopian vision and a dystopian vision. A comparison of these two visions leads to seven challenges that AIED might have to face in the future: intercultural and global dimensions, practical impact, privacy, interaction methods, collaboration at scale, effectiveness in multiple domains, and the role of AIED in educational technology. The paper discusses these challenges and the associated risks and opportunities.
KeywordsAied Educational technologies Future Risks Challenges
Thinking realistically, any attempt to look forward on 25 years of future research in a field that is only 25 years of age itself1 and at the same time influenced by rapidly evolving technologies – like AIED – is hopeless and absurd. Many of such attempts, conducted by prominent people, have failed and are, as a collection, evidence of the difficulty of the endeavor by themselves. This includes famous quotes of Thomas Watson, chairman of IBM, who in 1943 predicted a world market for five computers (Morgan & Langford, 1981), or Ken Olson, the president of Digital Equipment Corporation, who in 1977 stated that he sees no reason for an individual to have a computer in their home (Cerf & Navasky, 1984).
Yet, even in the light of this difficult situation, it is still worth to look into the future of AIED, as this special issue does. The 25th anniversary of the journal is a formal one and an inviting occasion to do so. There have been examples of future predictions on technology that turned out to be surprisingly accurate (e.g., Bush, 1945; Weiser, 1991). And indeed, some of the AIED research has matured and taken a steady development over time even in the “rebellious youth years” of the field. As such, there may actually be a chance that the future changes of the field may not be disruptive and entirely unpredictable but follow the lines of past developments and the trends on the horizons that we can currently observe. A look into the future of AIED as an interdisciplinary field necessarily needs to take into account the anticipated future developments of “external” (i.e., educational and technological) fields as well as “internal” trends within the field itself. This paper is an attempt to do just this – it starts by listing some noteworthy aspects (in the sense of relatively stable states or current development trends) of technology, education, and AIED. After that, these trends are interpreted in the form of two contrasting future visions, a utopian one and a dystopian one. The paper concludes with a comparative analysis of these visions, leading to seven important risks, challenges and opportunities that the AIED field will likely need to deal with in the next 25 years in order to be successful. There is certainly no way in which any of these can be “right” or “wrong” in a scientific way, and there is no way to prove or falsify them now. The intended contribution of the paper is to think through possible future developments and thereby imagining and opening for discourse a possible future, the associated actions that the AIED field may (or may not) take, and the possible impact of those actions on future developments of the field. In retrospect, this paper – as well as all the others in the volume – might be a nice historical resource to reconsider in a few years from now in order to compare the actual developments of AIED to the then-discussed possible future.2
Current Trends and Developments
As argued, factors outside of AIED will most definitely influence the future of AIED. This is not a new insight – some of the very core aspects of today’s AIED would be unimaginable without developments in technology, the cognitive and learning sciences, or educational practice. Just think of sensor technologies that made new forms of affective tutors possible (Calvo & D’Mello, 2011), graphics technologies that enable wholly new forms of interactive feedback that were unimaginable 25 years ago (Core et al., 2006), or cognitive architectures like ACT-R (Anderson, 2013) that have led to successful ITSs that are deployed in many schools. This use in turn influences research through the empirical insights gained (Koedinger et al., 1997), to name a few examples. In the following, we list some of today’s technology and education trends, followed by selected AIED-internal aspects that might have a bearing on the future of the field.
Technological Trends and Developments Relevant to AIED
preserving and archiving our digital cultural heritage,
enabling a secure, private, free and trustful use of the internet of the future,
addressing systemic risks in global IT supported networks through appropriate intervention strategies,
developing ubiquitous interaction methods that allow people of all ages and education to benefit from communication and information tools and to participate in education, culture and politics via IT, and
creating methods for guaranteeing the reliability of systems
This list is certainly debatable and it has been debated within that association. It would possibly be a different list if it were constructed in other countries, yet it does have some overlap with a list of Grand Challenges proposed by the UK Computing Research committee in 2008.4 The list contains some Computer Science challenges that are directly relevant for AIED. Specifically, the challenge of creating ubiquitous and universal interaction methods has direct implication for educational technologies, as do networks and policies that enable the reliable processing of educational data within AIED systems while respecting privacy.
In addition to these selected challenges for the future of Computer Science (which the field is likely to address), an aspect of the CS’s present that will likely have an impact on AIED is the increasing importance of data. Buzzwords like “Big Data” or “Industry 4.0” are currently being used to describe the fact that the amount of data collected by IT systems is not only getting greater with resulting implications for storage and analysis, but also has a value of its own. This is even true in fields where data is not the key asset by itself such as in social networks. The value of data, as argued, lies in its aggregation and in analysis, along with the possible insights gained from these processes. The latter can be helpful for improving industrial production, or even for understanding societal processes and identifying social issues (Pentland, 2014). The value of data for AIED is evident, for example in its function to build student models based on student interaction data. Yet, the field is probably just beginning to understand the larger value of long-term data collections such as the PSLC DataShop (Koedinger et al., 2010) in order to contribute to a data driven science of learning and the related adaptive educational technologies.
Educational Trends and Developments Relevant to AIED
Compared to Computer Science and technology development, learning science (and certainly educational practice!) changes less rapidly. In the past decades, we observed development trends from behaviorism through cognitivism to constructionism. These have had their impact on educational technologies – from early behaviorist Computer Assisted Instruction systems to platforms that feature self-directed learning. A tendency in educational research and practice with implications for AIED is from the focus on the individual through small group learning settings to the consideration of larger communities of learners. Arguably, AIED work that takes into account collaborative learning is not new, with modeling methods for learning groups dating back at least 20 years (Hoppe, 1995) and several collaborative learning methods having been proposed in AIED (e.g., Walker et al., 2011). Yet, the challenge (and opportunity) that unfolds for AIED research with massive scale online learning settings currently named MOOCs is still enormous and includes aspects such as forming and supporting effective learning groups with participants from different cultures in dynamic settings involving dynamically joining and leaving students.
In addition to scale, another present educational challenge relevant for AIED relates to global dimensions. The Global Learning XPrize,5 a current competition (worth $15 million) about developing software solutions for children in developing countries that help them teach themselves basic reading, writing and math skills, expresses this nicely. The call is not about AIED per se, but illustrates the need for effective (and, thus, probably adaptive) learning software that works globally. Unfortunately, some insights from within the AIED field suggest that tutoring systems seem to be culturally dependent (Ogan & Johnson, 2015). This makes the solution of the problem more difficult but not less urgent.
In terms of the trends in learning and teaching as monitored by the OECD, an important aspect for teachers which is relevant for AIED is professional development: according to TALIS (Teaching And Learning International Survey) 2009, teachers require professional development most in aspects related to inclusion, followed by ICT teaching skills.6 Yet, such training often is not available. Most survey respondents will likely have had in mind ICT technologies such as tablets or interactive whiteboards when answering the OECD questionnaire, yet the general insight probably holds: teachers need to be able to understand the technology proposed for their classroom (and the benefits that come with it); otherwise, they are simply less likely to use it. This stresses the importance of easy, ubiquitous and universal interaction methods mentioned in the previous subsection, yet with a focus on teachers as users with their central role in the classroom. Research such as (De Groot et al., 2007) illustrates how AIED systems can consider teacher needs and requirements in technology enhanced learning settings by providing AI supported analyses of the students’ actions that help the teacher maintain an overview of the class. Still, much more apparently needs to be done to successfully deploy AIED technologies in classrooms and thereby support the transfer from research to practice. Currently, both the teachers’ levels of technical knowledge and the quality of the technical school infrastructures are often not sufficient to allow for an educationally effective use of even basic information technology in the classroom (e.g., see a recent BITKOM Study for Germany7 or the results of the comparative International Computer and Information Literacy Study8), not even speaking of advanced AIED systems.
Educational Technology Trends and Developments
Time-to-Adoption horizon of one year or less: Bring Your Own Device (BYOD) and Flipped Classroom
Time-to-Adoption horizon of two to three years: Makerspaces and Wearable Technology
Time-to-Adoption horizon of four to five years: Adaptive Learning Technologies and The Internet of Things.
For a field that is 25 years old and claims to have had some impact on school practice already, it is at least remarkable to be attested a time horizon of at least 4 years to adoption. Admittedly, this listing is also a signal that AIED is not completely off the radar of other educational technology researchers.
The mentioning of wearable technology in the Horizon report list is also interesting from an AIED perspective and can be can be interpreted in the light of the general technology trend of ubiquitous interaction methods: will we wear adaptive technology systems in the future?
Another clear trend observable within the educational technology research communities (which can also be seen as the learning technology variant of a general technology trend) is the focus on data driven research and system design methods. Research communities such as SOLAR (Society of Learning Analytics Researchers) and the Educational Data Mining Society have emerged in the last years. EDM started as a workshop held in conjunction with AIED or ITS conferences – and today is a conference in its own right that partially competes with its origins (at least in terms of attracting attendants, but also slightly in terms of underlying research philosophy). As Pea (2014) put it, the goal of personalized learning at scale can only be reached by establishing a field of Learning Analytics. This should certainly not be misinterpreted as a suggestion to forget about past AIED and now head off to completely new realms. Many relatively mature AIED methods for modeling, analyzing and interpreting student behavior and responding to it automatically will very likely turn out to be useful for this proposed new field – in any case, the focus on data driven analytics methods as one pathway for future educational technology activities is remarkable.
In terms of educational technology challenges, the Global Learning XPrize mentioned before also touches upon a second issue in addition to global applicability, namely the open source nature of the software to be designed as part of the challenge. Should AIED software be open source? Via long-term data collections and technologically stable systems that fulfil the requirements of everyday use, the AIED field can benefit from companies that provide stable adaptive tutoring systems with continuous support to schools – something that Universities with the typical research prototypes produced there can hardly deliver. Authoring effective tutoring systems is costly (Murray, 1999; Mitrovic & Koedinger, 2009) and may require different versions for different clienteles (Ogan & Johnson, 2015). Currently, only a few companies invest into fully-fledged ITSs. At the same time, as expressed in the XPrize call, other stakeholders favor free and open source solutions. But what would AIED business models look like that combine these requirements?
Two Future Scenarios
The trends and developments sketched above make certain lines of future development more likely than others. For instance, the complete disappearance of educational technology is not likely to happen based on what we know today. But what about the future development of AIED specifically in the next years to come? The following two, admittedly extreme, scenarios about students who are close to graduating from school in 25 years from now may serve as a means for thinking about possible futures of AIED and about decisions that AIED researchers and developers might be faced with in the future.
The Utopian Vision
Ana: Isn’t it cool that we can get feedback on our math tasks even on Sundays? My grandfather told me that in his youth that wasn’t possible – unless your parents or so had time and knew any of the stuff. Yes, there were books and so on, and he heard of some computer-assisted instruction, but that was only for freaks. Not as we have now.
Ben: Yes, I agree. I love that I can use these tutors whenever I want to. They really helped me over the years. I think I would not have made it to this point in school without them. It’s actually cool they don’t forget a thing about me. Here, look – it’s recommending me to check some older exercises on geometry again. It’s probably what I need to solve the tasks our teacher gave us to prepare for the exam – I was always struggling with these construction tasks, circles and so on.
Ana: (smiles) Just good that all our data is safe. Imagine our teacher could see when and where we learn.
Ben: (laughs) Yeah. Not the official recommendation way. But these tutors keep us on track it seems. Well, they can be annoying with their hints and messages and the like. But at least it’s fun with the games and the cool emotion stuff embedded, and there is also always this community that helps you if you don’t even get the fifth tutor message.
Ana: (grins) Last month I got five kudos there because I helped someone out online. I got to play a really fun game for that. Actually, what if all that didn’t exist – it must have been hard for teachers back then, before all that was developed!
Ben: Yep. My father used to be a teacher. But he quit – too much stress, with all these demands put on them. Good idea to leave no child behind he said, but then teachers were the ones to tailor their courses to fit every single student’s need. Impossible, my father said. And now look at our teacher – seems she really has time for some of our class.
Ana: She once told me that she gets a great overview of the class and of who needs her. There is a global research community developing all these tools, she said. Long time ago that wasn’t possible with fragmented disciplines in many countries, but they seemed to have worked it out. That’s why so many schools and companies got interested. Did you know the researchers also get our data to improve the tools?
Ben: Oh. So these scientists know we’re always in the park at Sundays and do our homework just a few minutes before school?
Ana: No. I also asked her that – well not in your words of course. The data is used to make the tools better, but they don’t need our names for that. So they detect if many of us fail on these grammar tasks for example, but not who – and I think they then figure out why, and change the tutor.
Ben: You know what? I think that’s what happened last year. This old art tutor really was a pain in the neck last year. But then they just had this new system, this was much cooler. It made me really get these perspectives in drawing. Look at what I made this year with this sketching tool and the help of the tutor!
Ana: (looks) Nice – for a boy (smiles). Hey, look, it recognized my smile and gave your design a kudo! Cool.
Ben: Strike! That goes in my portfolio and, who knows, may help me get a job in that media design company. Actually really nice that the kudos and so on don’t get lost if they switch tools. Oh, look, it’s now recommending us a break. Huh, did we really learn for four hours now?
Ana: Seems so. Time flies. But let’s call it a day. Even if fun – usually there’s only so much you can learn before your mind turns to jelly. (both leave)
The Dystopian Vision
Dan: I can’t get these math tasks on that list. What’s a cosine again?
Eve: Hm – don’t really know. Math is not my field.
Dan: Can’t our tutoring tool help us? Let’s try. (searches) Yes, here are some tasks on cosines. It also has some worked examples. (reads) Ok, I think I got it. It’s actually not so hard.
Eve: You really know how these tools work. Too bad they are not so common in France where I came from last year. I never really used them until last year when we moved to the US. Our teacher in France said she didn’t like them in their class.
Dan: Oh, I didn’t know that. Why did she say that?
Eve: Well, she said the tutors were made for US education. Which is different from French education. And then the language, she wanted the stuff in French, with French examples and so on. Otherwise it just wouldn’t work, she said. And I guess she was also concerned about her job.
Dan: Typical European (grins). But this job thing is actually not that wrong. I saw stats that in the last 10 years, 50 percent of the math and science teachers were fired. They really saved money with this, and they justified it by quoting famous scientists who claimed some sigma effect stuff I did not understand – but essentially it was saying these tutoring tools help us as well as humans do. So we need fewer teachers and have computer hours.
Eve: Good that these tutors are only there for math and such. I do occasionally like my teacher (smiles).
Dan: Yes, even if this exam prep list is really hard. Did you know there once were attempts to have tutors also for ethics, art and sport? My sister told me that. But all this was buggy she said. Weird feedback all the time, not really helpful. Even dangerous. You had to be careful with what you say.
Eve: (surprised) Even careful? Why?
Dan: Well, ain’t that clear? So think about it, what happens with the stuff you say to your tutor?
Eve: Well… it’s… stored?
Dan: Sort of. It all gets to this company who calculates if our solutions are OK or not. But they need to make money, so occasionally they seem to sell our data to whoever pays. And there were also all these hacks sometimes where people stole that data. My sister told me that one of her classmates did not get a job because of that. You know, things she said to this ethics learning machine. It sort of misinterpreted her words and not only gave silly feedback but profiled her as extreme and dangerous. Also she played with the tutor a lot and was profiled as a “gamer”. And then all that life-long profile data went to that employer somehow. So no job offer for her. Nothing she could do.
Eve: But that does not happen now anymore?
Dan: Well, so they say. I’m always thinking twice before typing in anything, but for math it’s relatively safe. Many are weak in math, and nobody’s really radical when it comes to math, not even in these online forums (smiles).
Eve: Agree. Anyway, that explains why these forums are so empty. (pause) My grandfather once told me that not only companies but also researchers build these tutors. He was a professor himself and used to work in that field, but said he left it and moved to other fields many years before he retired – they just did not get any really global community going he said: no real impact, only theory and academic battlegrounds. That was not his way of doing research. He sounded really frustrated.
Dan: As we are with math now. (listens) Oh no, the library closes. We need to leave. Let’s hope they still have this good book on French culture. I need it for the exams.
Eve: Well, I can help with that one without books or tutors I guess. (smiles, both leave)
Discussion and Conclusion
The – somewhat exaggerated – future visions with Ana and Ben on the one side and Dan and Eve on the other, together with the general trends in computer science and education discussed before, reveal some challenges that the AIED field will likely have to face in the next years to come. We can see them as risks or opportunities – and in some sense, they are both, depending on how the field positions itself relative to them. This paper discusses seven such challenges in the following.
The first challenge relates to intercultural and global dimensions of AIED. We currently observe that the internet penetration rate in developing countries is dramatically increasing, with Africa being the continent with a lowest rate of 26 % but an increase of more than 6000 % in the last 15 years.9 It seems predictable that in 25 years (and likely in less than that), the technical requirements will be available for delivering AIED to almost every student in the world. But, as argued, research results and practical experience tell us that it is often not possible to just take a piece of educational software that is successful in one cultural context and then expect that this software will also be effective in another deployment in a different cultural context. This is due to several reasons, including language and curriculum content as obvious factors but also educational cultures and typical ways of teacher-student interaction that educational software should be adapted to (especially if, as many AIED systems do, the system provides some “human teacher like” feedback). The research field will have to continue to investigate the intercultural differences between learning and classroom cultures and the consequences this has on AIED system design and implementation. If not, then Eve’s statement may turn out to be right in the future. An opportunity along with a possible intercultural use of AIED systems certainly is that a truly international AIED research community beyond WEIRD (Western, Educated, Industrialized, Rich and Democratic) countries could have much more practical impact than a scenario that focuses on a few countries only. It could have effects in countries where good AIED systems could really be of help due to problems in the educational systems (most importantly, not enough teachers, not enough learning resources and large classes).
This leads to the second challenge: practical impact. We currently observe that some AIED systems have an outreach to a considerable number of schools. These are often deployed in cooperation with commercial initiatives within the AIED field, such as Carnegie Learning or Alelo. These relations to schools and to industry are important for the field to grow. Today’s scenery in this regard is radically different to the one 25 years ago in that empirical data and larger scale deployment play a greater role. These are probably good signs. Still, for the field to have even greater impacts on educational practice in the future, it needs to continue in this way and get more stakeholders on board (unlike Eve’s previous teacher). In order to actually make their way into regular usage at schools, AIED system deployments need to meet the criteria of schools – such as available technical support and troubleshooting, stability and continuous availability. Research test prototypes developed as part of a PhD thesis often fail to meet these criteria. This calls for more professional methods, for instance via commercial spin-offs of AIED academics. It is evident that a larger clientele would be desirable for these to be profitable, which makes the intercultural challenge even more important.
If we have companies that run AIED systems, then the third challenge of privacy gains even more importance. Observing the current trends towards data driven computing and research in many application fields including educational technology and the resulting value that data has, the appropriate handling of students’ learning data becomes a central concern. No one wants to share the experiences of Dan’s sister. On the other hand, the benefits of AIED systems acting as lifelong learning companions (and, thus, life-long collectors of learning data) are visible in Ana and Ben’s experiences. Privacy regulations exist beyond AIED, albeit these laws and policies differ between countries. These regulations will get more important for the field as its outreach into practice and its reliance on collected data grows, and as the public’s concerns about privacy grow. Specifically for AIED, investigation of best practices of the typically involved stakeholders (e.g., students, parents, teachers, administrators, researchers, system providers) and their privacy related requirements will be important. While all this is true also for non-commercial AIED research and practice, the situation gets even more complicated with commercial use. How can commercial AIED systems be profitable under strict privacy policies that, for instance, prohibit giving student data away to anyone? Advertisements may often not be a real option in learning software, and schools or parents oftentimes have no money to pay great fees for licenses, especially in developing countries. The use of learner data to pay for the system development and deployment costs, enabling making the system itself available “for free”, may seem attractive, but clear rules should be defined, implemented, controlled and made transparent here.
Interaction methods are the fourth challenge in this list. Due to general developments in the Human-Computer Interaction field, the student’s interaction with today’s AIED systems is already easier and more intuitive than it used to be decades ago. This tendency is likely to continue further. In line with the Internet of Things and ubiquitous interaction methods, we will probably need to re-think the way students interact with AIED systems. This involves higher-level design methods (see Ben’s comments on games, and their use of kudos) as well as basic interaction techniques that allow students to enter data into the system and receive feedback conveniently (affect detection, speech, touch, etc). Compared to other interactive software systems that might use the same techniques, an AIED specific challenge is how to analyze the noisy data that is received via rich input channels in order to provide feedback as student models are built (see Dan’s comment on the ethics tutor). It is easier to automatically interpret a foreign language sentence typed in by a student and to consequently give guidance in an ITS than to do the same with verbal input due to the still error-prone nature of today’s speech recognition technology (e.g., background noise, accents, dialects, multiple speakers, words with the same pronunciations, wrong pronunciations of foreign language learners).
The fifth challenge in this list is related to the fourth and addresses collaboration at scale. We have already observed online classes reach global dimensions and scaling up to several hundred thousand students, yet not much (if any) AIED is involved in most of these current Massive Open Online Course projects, which typically feature rather simple models of educational technology: video and multiple choice quizzes, together with some discussion forums. Educational research is investigating community based learning principles. There are opportunities for intelligent techniques that make learning together possible at distance and at scale, with personalized guidance on content and on the collaboration process – even intertwined with each other (as put by Ana and Ben). The AIED field has developed foundations that should allow for addressing these issues. Yet, the demonstrated results are not mature enough today to fully exploit their potential to support instructors and participants of large-scale online classes. The fact that such support for collaborative learning is needed is known though (Kollar et al., 2006).
The sixth challenge is at the core of AIED research: effectiveness in multiple domains. If AIED systems are to help Ben get his kudos for the design job, they need to advance beyond the known realms. The AIED field knows how to design and construct effective learning tools. It has developed methods and standards to evaluate them. But still, a great portion of today’s AIED research is in something we may call its “comfort zone”, where student and domain modeling is feasible with known and established techniques such as Constraint Based Modeling (Ohlsson, 1994) or Model Tracing (Koedinger et al., 1997) or variations thereof. Addressing other domains and tasks where the developed modeling techniques cannot be applied easily, such as in many ill-defined domains (Pinkwart et al., 2009), is a challenge that is worth addressing. Students could probably benefit from personalized automated feedback on their ethics essays or on their actions in virtual reality surgery training just as much as they do while learning geometry with ITSs today. Yet, this moving out of the “comfort zone” may require further developments in other scientific fields such as natural language processing technology or knowledge representation.
Finally, the seventh challenge addresses the role of AIED in educational technology. The field of Human-Computer Interaction has the CHI conference; the graphics community similarly has SIGGRAPH as the prime venue. Both have an enormous prestige, a considerable size and an integration effect in the sense that these conferences and communities are “the place to be” (or publish). There is, at present, nothing similar for the educational technology sector. Instead, there have been tendencies towards smaller, more focused conferences and communities – such as CSCL and EDM having separated from AIED and ITS, AIED and ITS being similar but not the same, and LAK as an additional community in competition to all the others when it comes to the question where to publish and what conference to attend. Arguably, EDM and LAK also represent different, more data-driven, approaches to solving educational problems with technology. In related areas, the ICALT conference has recently changed towards a multiconference model, with a subconference on CSCL that aims at attracting more technical papers that have a hard time getting accepted in the review process at the empirically oriented CSCL conference. Diversity is not necessarily bad: A big “melting pot” community also comes with disadvantages. And certainly, academic debates about future orientations are needed for a field to advance. However, a challenge for the global educational technology research community will be to handle these interplays between different directions while avoiding too much fragmentation that could lead to a lack of overall impact. Independently of whether we will have a large-scale global educational technology research community in the future or not, AIED should definitely not be an island in educational technology research 25 years from now, but should have established connections to other relevant strands of research that could make use of the scalable personalization that strong AIED research offers. As argued in this paper, the potential benefits (and beneficiaries) are manifold.
- Anderson, J. R. (2013). The architecture of cognition. Psychology Press.Google Scholar
- Bush, V. (1945). As we may think. The Atlantic Monthly, 176(1), 101–108.Google Scholar
- Calvo, R. A., & D'Mello, S. K. (Eds.) (2011). New perspectives on affect and learning technologies. New York: Springer.Google Scholar
- Cerf, C., & Navasky, V. (1984). The experts speak: The definitive compendium of authoritative misinformation. Pantheon Books.Google Scholar
- DE Groot, R., Drachman, R., Hever, R., Schwarz, B. B., Hoppe, U., Harrer, A., DE Laat, M., Wegerif, R., McLaren, B. M., & Baurens, B. (2007). Computer supported moderation of e-discussions: the ARGUNAUT approach. In Proceedings of the 8th iternational conference on Computer supported collaborative learning (pp. 168–170). International Society of the Learning Sciences.Google Scholar
- Hoppe, H. U. (1995). The use of multiple student modeling to parameterize group learning. In J. Greer (Ed.), Proceedings of AI-ED'95, 7th world conference on artificial intelligence in education (pp. 234–241). Charlottesville, Virginia: AACE.Google Scholar
- Johnson, L., Adams Becker, S., Estrada, V., & Freeman, A. (2015). NMC horizon report: 2015 higher education edition. Austin, Texas: The New Media Consortium.Google Scholar
- Koedinger, K. R., Anderson, J. R., Hadley, W. H., & Mark, M. A. (1997). Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education, 8, 30–43.Google Scholar
- Koedinger, K. R., Baker, R. S., Cunningham, K., Skogsholm, A., Leber, B., & Stamper, J. (2010). A data repository for the EDM community: the PSLC DataShop. Handbook of Educational data Mining, 43.Google Scholar
- Mitrovic, A., & Koedinger, K. R. (Eds.) (2009). International journal of artificial intelligence in education, special issue on authoring intelligent tutoring systems, 19(2). Google Scholar
- Morgan, C., & Langford, D. (1981). Facts and fallacies: A book of definitive mistakes and misguided predictions. St. Martin’s Press.Google Scholar
- Murray, T. (1999). Authoring intelligent tutoring systems: an analysis of the state of the art. International Journal of Artificial Intelligence in Education, 10, 98–129.Google Scholar
- Ogan, A., & Johnson, W. L. (Eds.) (2015). International journal of artificial intelligence in education, special issue on culturally aware educational technologies, 25(2). Google Scholar
- Pea, R. (2014). The learning analytics workgroup. A report on building the field of learning analytics for personalized learning at scale. Available at https://ed.stanford.edu/sites/default/files/law_report_complete_09-02-2014.pdf
- Pentland, A. (2014). Social physics: How good ideas spread-The lessons from a New science. Penguin.Google Scholar
- Pinkwart, N., Lynch, C., Ashley, K. D., & Aleven, V. (Eds.) (2009). International journal of artificial intelligence in education, special issue on aied applications for ill-defined domains, 19(3 + 4). Google Scholar