Validation of a free response test of deep learning about the normal swallowing process

Abstract

A test of core knowledge within a healthsciences topic was developed and validatedusing the partial credit model of Raschanalysis (Masters 1982). The SOLO Taxonomy(Biggs and Collis 1982) was used as thetheoretical basis to determine speech pathologystudents' depth of understanding of keyfeatures of a complex physiological process(swallowing) as depicted in their writtendescriptions. Rasch analysis revealed thosefeatures that fitted the empirical model andthose that needed to be revised or discarded.Students' mastery of the retained features wasused to calibrate scaled scores that could beemployed for a variety of comparative purposes.This study provides evidence that use of theSOLO Taxonomy can be extended to provideprecise and reliable quantitative scores.Consequently valid tests of student knowledgein a range of distinct topic areas can bedesigned to reflect both the quality of studentlearning as well as its quantity, a featurethat offers potential for assessment ingeneral.

This is a preview of subscription content, access via your institution.

References

  1. Adams, R.J. and Khoo, S.T. (1997). Quest: The Interactive Test Analysis System [statistical analysis]. Melbourne: Australian Council for Educational Research.

    Google Scholar 

  2. Aldous, C., Leeder, S.R., Price, J., Jervie Sefton, A.E. and Teubner, J.K. (1997). ‘A selection test for Australian Graduate Entry Medical Schools’, Medical Journal of Australia 166, 247.

    Google Scholar 

  3. Anderson, L.W. and Sosniak, L.A. (1994). Bloom's Taxonomy: A Forty Year Retrospective. Chicago: National Society of the Study of Education.

    Google Scholar 

  4. Andrich, D. (1997). ‘Rating analysis’, in Keeves, J.P. (ed.), Educational Research, Methodology, and Measurement: An International Handbook. Oxford: Pergamon, pp. 874-880.

    Google Scholar 

  5. Bereiter, C. and Scardamalia, M. (1996). ‘Rethinking learning’, in Olson, D.R. and Torrance, N. (eds.), Handbook of Education and Human Development: New Models of Learning, Teaching and Schooling. Cambridge, MA: Basil Blackwell.

    Google Scholar 

  6. Biggs, J.B. (ed.) (1991). Teaching for Learning. The View from Cognitive Psychology. ACER.

  7. Biggs, J.B. and Collis, K.F. (1982). Evaluating the Quality of Learning: The SOLO Taxonomy. New York: Academic Press.

    Google Scholar 

  8. Bloom, B.S. (ed.), Engelhart, M.D., Furst, E.J. Hill, W.H. and Krathwohl, D.R. (1956). Taxonomy of Educational Objectives I: Cognitive Domain. New York: McKay.

    Google Scholar 

  9. Cox, (1976). ‘How did you guess? Or, what do multiple choice questions measure?’ Medical Journal of Australia 1, 88.

    Google Scholar 

  10. Damjanov, I., Fenderson, B., Veloski, J. and Rubin, E. (1995). ‘Testing of medical students with open-ended, uncued questions’, Human Pathology, 26(4), 362-365.

    Google Scholar 

  11. Dudley, H.A.F. (1973). ‘Multiple-choice questions: time for a second look?’ Lancet 11, 195.

    Google Scholar 

  12. Harden, R.Mc.G., Brown, R.A., Biran, L.A., Dallas Ross, W.P. and Wakeford, R. (1976). ‘Multiple choice questions: to guess or not to guess’, Medical Education 10, 27.

    Google Scholar 

  13. Hattie, J. Jaeger, R.M. and Bond, L. (1999). ‘Persistent methodological questions in educational testing’, in Iran-Nejad, A. and Pearson, P.D. (eds.), Review of Research in Education, Vol 24. Washington D.C: American Educational Research Association, pp. 393-446.

    Google Scholar 

  14. Hattie, J. and Purdie, N. (1998). ‘The SOLO model: Addressing fundamental measurement issues’, in Dart, B. and Boulton-Lewis, G. (eds.), Teaching and Learning in Higher Education. ACER Press, pp. 145-176.

  15. Hayes, J. and Flower, L. (1986). ‘Writing research and the writer’, American Psychologist, 1106-1113.

  16. Heywood, J. (1989). Assessment in Higher Education (second edition). New York: Wiley.

    Google Scholar 

  17. Killoran, J. (1992). ‘In defence of multiple-choice questions’, Social Education 56(2), 106-108.

    Google Scholar 

  18. Krathwohl, D., Anderson, L., Airasian, P., Cruickshank, K., Mayer, R., Pintrich, P. and Raths, J. (2000). A Taxonomy of Learning for Teaching: A Revision of Bloom's Taxonomy of Educational Objectives. New York: Addison-Wesley-Longman.

    Google Scholar 

  19. Lam, R. (1996). ‘The theory and construction of ordered outcome items’, in Biggs, J.B. (ed.), Testing: To Educate or To Select? Hong Kong Educational Publishing Co.

  20. Masters, G.N. (1982). ‘A Rasch model for partial credit scoring’, Psychometrika 47(2), 149-174.

    Google Scholar 

  21. McCloskey, K.I. and Holland, R.A.B. (1976). ‘A comparison of student performances in answering essay-type and multiple choice questions’, Medical Education 10, 38-45.

    Google Scholar 

  22. Newble, D.I. Baxter, A. and Elmslie, R.G. (1979). ‘A comparison of multiple-choice tests and free-response tests in examinations of clinical competence’, Medical Education 13, 263-268.

    Google Scholar 

  23. Newble, D.I. and Jaeger, K. (1983). ‘The effect of assessments and examination on the learning of medical students’, Medical Education 17, 165-171.

    Google Scholar 

  24. Nightingale, P., Wiata, I.T., Toohey, S., Ryan, G., Hughes C. and Magin, D. (1996). Assessing Learning in Universities. Professional Development Centre, University of New South Wales.

  25. Scholten, I. (2001). Growth of Students’ Knowledge of the Normal Swallowing Process. A Research paper prepared in fulfilment of the requirements in the topic Research Studies C (EDES 98123), within the Doctor of Education program, School of Education, Flinders University, Adelaide: Australia.

  26. Scholten, I. (2000). ‘What students do and don't know about swallowing’, ACQuring Knowledge in Speech, Language and Hearing 2(2), 56-58.

    Google Scholar 

  27. Scholten, I. (1999). Student learning of essential aspects of the swallowing process. Paper presented at the Dysphagia Research Society, Burlington, Vermont.

    Google Scholar 

  28. Thorndike (1997). In Keeves, J.P. (ed.), Educational Research, Methodology, and Measurement: An International Handbook. Oxford: Pergamon, pp. 330-343.

    Google Scholar 

  29. Van der Vleuten, C.P.M., Norman, G.R. and de Graaff, E. (1991). ‘Pitfalls in the pursuit of objectivity: issues of reliability’, Medical Education 25, 110-118.

    Google Scholar 

  30. Wiggins, G. (1992). ‘Creating tests worth taking’, Educational Leadership, May, 26-33.

  31. Willett, J.B. (1997). ‘Change, measurement of’, in Keeves, J.P. (ed.), Educational Research, Methodology, and Measurement: An International Handbook. Oxford: Pergamon, pp. 327-334.

    Google Scholar 

  32. Wright, B.D. and Masters, G.N. (1982). Rating Scale Analysis. Chicago: Mesa Press.

    Google Scholar 

Download references

Author information

Affiliations

Authors

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Scholten, I., Keeves, J.P. & Lawson, M.J. Validation of a free response test of deep learning about the normal swallowing process. Higher Education 44, 233–255 (2002). https://doi.org/10.1023/A:1016322828592

Download citation

  • assessment
  • deep learning
  • dysphagia
  • Rasch analysis
  • SOLO taxonomy
  • student learning
  • swallowing