Investigating the feasibility of using digital representations of work for performance assessment in engineering

  • P. John Williams


This paper reports on the results of a 3-year study conducted at the Centre for Schooling and Learning Technologies (CSaLT) at Edith Cowan University in collaboration with the Curriculum Council of Western Australia which concerns the potential to use digital technologies to represent the output from assessment tasks in the senior secondary course, Engineering Studies. The general aim of this study is to explore the potential of various digitally-based forms for external assessment for senior secondary courses in terms of manageability, cost, validity and reliability. The problem being addressed was the need to provide students with assessment opportunities in new courses, that are on one hand authentic, where many outcomes do not lend themselves to being assessed using pen and paper over a three hour period, while on the other hand being able to be reliably and manageably assessed by external examiners. That is, the external assessment for a course needs to accurately and reliably assess the outcomes without a huge increase in the cost of assessment. A computer managed examination was designed that consisted of a design task that was broken down into a number of timed activities. Students were paced through each activity, recording their input in the form of a portfolio. Input consisted of text, graphics through a camera, video and voice. The exam outputs were uploaded to a online repository. The students’ work was marked by external assessors using a standards based rubric that allowed the students work to be ranked though Rasch Modelling.


Digital assessment Pairs marking Computer managed learning High stakes assessment Online learning Elearning 


  1. American Educational Research Association, American Psychological Association, & National Council of Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: AERA.Google Scholar
  2. Barrett, H. C. (2005). Researching electronic portfolios and learner engagement: The REFLECT Initiative. Google Scholar
  3. Biggs, J. B. (2003). Teaching for quality learning at university. Buckingham: The Open University Press.Google Scholar
  4. Burns, R. B. (1996). Introduction to research methods. South Melbourne, Australia: Addison Wesley Longman Australia Pty. Limited.Google Scholar
  5. Campbell, A. (2008). Performance enhancement of the task assessment process through the application of an electronic performance support system. Unpublished Doctorate, Edith Cowan University, Perth.Google Scholar
  6. Cisco, et al. (2009). Transforming education: assessing and teaching 21st century skills. Assessment and teaching of 21st century skills. Retrieved September 8, 2011 from
  7. Curriculum Council. (2007). Engineering studies syllabus. Perth, Western Australia: Curriculum Council.Google Scholar
  8. Dede, C. (2003). No cliche left behind: Why education policy is not like the movies. Educational Technology, 43(2), 5–10.Google Scholar
  9. Garmire, E., & Pearson, G. (Eds.). (2006). Tech Tally: Approaches to assessing technological literacy. Washington: National Academy Press.Google Scholar
  10. Kimbell, R. (2004). Design & Technology. In J. White (Ed.), Rethinking the school curriculum: Values, aims and purposes (pp. 40–59). New York, London: Routledge Falmer.Google Scholar
  11. Kimbell, R., & Wheeler, T. (2005). Project e-scape: Phase 1 report. London: Technology Education Research Unit, Goldsmiths College.Google Scholar
  12. Kimbell, R., Wheeler, T., Miller, A., & Pollitt, A. (2007). e-scape: e-solutions for creative assessment in portfolio environments. London: Technology Education Research Unit, Goldsmiths College.Google Scholar
  13. Lane, S. (2004). Validity of high-stakes assessment: Are students engaged in complex thinking? Educational measurement. Issues and Practice, 23(3), 6–14.CrossRefGoogle Scholar
  14. Lesgold, A. (2009). Better schools for the 21st century. Retrieved 9/6/2009, from
  15. Lin, H., & Dwyer, F. (2006). The fingertip effects of computer-based assessment in education. TechTrends, 50(6), 27–31.CrossRefGoogle Scholar
  16. McGaw, B. (2006). Assessment to fit for purpose. In Paper presented at the 32nd annual conference of the international association for educational assessment, Singapore.Google Scholar
  17. Messick, S. (1994). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher, 23(2), 13–23.Google Scholar
  18. Patton, M. Q. (1990). Qualitative evaluation and research methods. Newbery Park, California: SAGE Publications, Inc.Google Scholar
  19. Pollitt, A. (2004, June 2004). Let’s stop marking exams. Paper presented at The International Association for Educational Assessment Conference, Philadelphia.Google Scholar
  20. Richardson, H. C., & Ward, R. (2005). Developing and implementing a methodology for reviewing E-portfolio products. Wigan, UK: Joint Information Systems Committee.Google Scholar
  21. Ridgway, J., McCusker, S., & Pead, D. (2006). Report 10: Literature review of E-assessment. Bristol: Futurelab.Google Scholar
  22. The British Psychological Society. (2002). Guidelines for the development and use of computer-based assessments. Leicester, UK: The British Psychological Society.Google Scholar
  23. The Council of the International Test Commission. (2005). International guidelines on comptuer-based and internet delivered testing. Granada, Spain: The Council of the International Test Commission.Google Scholar
  24. Wiggins, G. (1998). Educative assessment: Designing assessments to inform and improve student performance. San Francisco, CA: Jossey-Bass.Google Scholar
  25. Willig, C. (2001). Introducing qualitative research in psychology adventures in theory and method. Buckingham: Open University Press.Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2011

Authors and Affiliations

  1. 1.University of WaikatoHamiltonNew Zealand
  2. 2.Edith Cowan UniversityPerthAustralia

Personalised recommendations