Automated Expert Modeling for Automated Student Evaluation

  • Robert G. Abbott
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4053)


This paper presents automated expert modeling for automated student evaluation, or AEMASE (pronounced “amaze”). This technique grades students by comparing their actions to a model of expert behavior. The expert model is constructed with machine learning techniques, avoiding the costly and time-consuming process of manual knowledge elicitation and expert system implementation. A brief summary of after action review (AAR) and intelligent tutoring systems (ITS) provides background for a prototype AAR application with a learning expert model. A validation experiment confirms that the prototype accurately grades student behavior on a tactical aircraft maneuver application. Finally, several topics for further research are proposed.


Receiver Operating Characteristic Curve Machine Learning Technique Student Evaluation Intelligent Tutor System Subject Matter Expert 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Centre, C.A.L.L.: The after action review process, learning more from our training. Army Lessons Learned Centre Dispatches 6(3) (1999)Google Scholar
  2. 2.
    DeGrosky, M.T.: Improving after action review (aar) practice. In: Eighth International Wildland Fire Safety Summit. (2005)Google Scholar
  3. 3.
    Meliza, L.: After action review: Methods and tools (2004) (retrieved on December 2, 2005),
  4. 4.
    Jensen, R., Nolan, M., Chen, D.Y.: Automatic causal explanation analysis for combined arms training aar. In: Proceedings of the Industry/Interservice, Training, Simulation and Education Conference (I/ITSEC) (2005)Google Scholar
  5. 5.
    Cohen, P.A., Kulik, J.A., Kulik, C.C.: Educational outcomes of tutoring: A meta-analysis of findings. American Educational Research Journal 19, 237–248 (1982)Google Scholar
  6. 6.
    Corbett, A.T.: Cognitive computer tutors: Solving the two- sigma problem. In: Bauer, M., Gmytrasiewicz, P.J., Vassileva, J. (eds.) UM 2001. LNCS, vol. 2109, pp. 137–147. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  7. 7.
    Burns, H., Capps, C.: Foundations of intelligent tutoring systems: An introduction. In: Polson, M.C., Richardson, J.J. (eds.) Foundations of Intelligent Tutoring Systems, pp. 1–18. Lawrence Erlbaum Associates Publishers, Mahwah (1988)Google Scholar
  8. 8.
    Murray, T.: Authoring intelligent tutoring systems: An analysis of the state of the art. International Journal of Artificial Intelligence in Education 10, 98–129 (1999)Google Scholar
  9. 9.
    Welch, G., Foxlin, E.: Motion tracking: no silver bullet, but a respectable arsenal. IEEE Computer Graphics and Applications 22(6), 24–38 (2002)CrossRefGoogle Scholar
  10. 10.
    Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco (1992)Google Scholar
  11. 11.
    Srinivasa, N., Ahuja, N.: A topological and temporal correlator network for spatiotemporal pattern learning, recognition and recall. IEEE Transactions on Neural Networks 10(2), 356–371 (1999)CrossRefGoogle Scholar
  12. 12.
    Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer, Heidelberg (2001)zbMATHGoogle Scholar
  13. 13.
    Sutton, R., Barto, A.: 3.5 The Markov Property. In: Reinforcement Learning, pp. 61–65. MIT Press, Cambridge (1998)Google Scholar
  14. 14.
    Kornecki, A., Hilburn, T., Diefenbach, T., Towhidnajad, M.: Intelligent tutoring issues for atc training system. IEEE Transactions on Control Systems Technology 1(3), 204–211 (1993)CrossRefGoogle Scholar
  15. 15.
    Hess, S.M., MacMillan, J., Sefarty, D., Elliot, L.: From cognitive task analysis to simulation: Developing a synthetic team task for awacs weapons directors, Air Force Research Laboratory (1999)Google Scholar
  16. 16.
    Kohavi, R.: A study of cross-validation and bootstrap for accuracy estimation and model selection. In: IJCAI, pp. 1137–1145 (1995)Google Scholar
  17. 17.
    Zweig, C.: Receiver-operating characteristic (roc) plots: a fundamental evaluation tool in clinical medicine. Clin. Chem. 39(4), 561–577 (1993)Google Scholar
  18. 18.
    Ali, K., Langley, P., Maloof, M., Sage, S., Binford, T.: Improving rooftop detection with interactive visual learning. In: Proceedings of the Image Understanding Workshop, pp. 479–492. Morgan Kaufmann, San Francisco (1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Robert G. Abbott
    • 1
  1. 1.Sandia National Labs MS 1188AlbuquerqueUSA

Personalised recommendations