Abstract
Two instructional designers were provided with formative evaluation data from the sources recommended in the literature and were asked to revise a piece of instructional text while thinking aloud. Their verbal protocols were recorded, transcribed, segmented and coded, using a coding scheme derived from Newell and Simon's (1972) problem-solving model. Relevant codes were analyzed for evidence of the characteristics of expert performers identified by Glaser and Chi (1988) and Shanteau (1992). Results were unclear when the number of coded segments in each category was counted, but examination of the content of the segments showed clearly that one of the research participants demonstrated more attributes of expertise than the other, based on the identified characteristics.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Alexander, P.A., Shallert, D.L., & Hare, V.C. (1991). Coming to terms: How researchers in learning and literacy talk about knowledge.Review of Educational Research, 61(3), 315–343.
Bordonaro, T. (1993).A comparison of the effectiveness, cost and efficiency of four formative evaluation conditions. Unpublished master's thesis, McGill University, Montreal, Canada.
Chase, W.G., & Simon, H.A. (1973). Perception in chess.Cognitive psychology, 4, 55–81.
Chi, M.T.H., Feltovich, P.J., & Glaser R. (1981). Categorization and representation of physics problems by experts and novices.Cognitive Science, 5, 121–125.
deGroot, A. (1966). Perception and memory versus thought: Some old ideas and recent findings. In B. Kleinmuntz (Ed.),Problem solving (pp. 1950). New York: Wiley.
Dick, W., & Carey, L. (1990).The systematic design of instruction. (3rd ed.). Glenview, IL: Scott, Foresman & Company.
Ericsson, K.A. (1996). The acquisition of expert performance. In K.A. Ericsson (Ed.),The road to excellence: The acquisition of expert performance in the arts and sciences, sports and games (pp. 1–50). Mahwah, NJ: Erlbaum.
Ericsson, K.A., & Polson, P.G. (1988). A cognitive analysis of exceptional memory for restaurant orders. In M.T.H. Chi, R. Glaser, & M.J. Farr (Eds.),The nature of expertise (pp. 23–70). Hillsdale, NJ: Erlbaum.
Ericsson, K.A., & Simon, H.A. (1980). Verbal reports as data.Psychological Review, 87(3), 215–251.
Ericsson, K.A., & Simon, H.A. (1993).Protocol analysis: Verbal reports as data (Revised edition). Cambridge, MA: MIT Press.
Ericsson, K.A., & Smith, J. (1991). Empirical study of expertise: prospects and limits. In K.A. Ericsson & J. Smith (Eds.),Toward a general theory of expertise: Prospects and limits (pp. 1–38). Cambridge: Cambridge University Press.
Flower, L., Carey, L., & Hayes, J.R. (1985).Diagnosis in revision: The experts' opinion (Technical Report No. 27). Pittsburgh, PA: Carnegie-Mellon University, Communications Design Center.
Glaser, R., & Chi, M.T.H. (1988). Overview. In M.T.H. Chi, R. Glaser, & M.J. Farr (Eds.),The nature of expertise (pp.xv-xxi).Hillsdale, NJ:Erlbaum.
Hayes, J.R. (1989).The complete problem solver (2nd ed.). Hillsdale, NJ: Erlbaum.
Holyoak, K.J., (1991) Symbolic connectionism: Toward third generation theories of expertise. In K.A. Ericsson & J. Smith (Eds.)Toward a general theory of expertise: Prospects and limits (pp. 301–335). Cambridge: Cambridge University Press.
Le Maistre, C. (1994).The priorities established among data sources when instructional designers revise written materials. Unpublished doctoral thesis, McGill University, Montreal, Canada.
Le Maistre, K. & Weston, C. (1996). The priorities established among data sources when instructional designers revise written materials.Educational Technology Research and Development, 44(1), 61–70.
McAlpine, L. (1987). The think-aloud protocol: A description of its use in the formative evaluation of learning materials.Performance and Instruction, 26(8). 18–21.
Nelson, W.A., & Orey, M.A. (1991, April).Reconceptualizing the instructional design process: Lessons learned from cognitive science. Paper presented at the Annual Meeting of the American Educational Research Association, Chicago, IL. (ERIC Document Reproduction Service No. ED 334 268)
Newell, A., & Simon, H.A. (1972).Human problem solving. Englewood Cliffs, NJ: Prentice-Hall Inc. (Chapter 14, pp. 787–868).
Orey, M.A., & Nelson, W.A. (1993). Development principles for intelligent tutoring systems: Integrating cognitive theory into the development of computer-based instruction.Educational Technology Research and Development, 41(1), 59–72.
Patel, V.L., & Groen, G.J. (1991). The general and specific nature of medical expertise: A critical look. In K.A. Ericsson & J. Smith (Eds.)Toward a general theory of expertise: Prospects and limits (pp. 93–125). Cambridge: Cambridge University Press.
Perez, R.S., Johnson, J.F., & Emery, C.D. (1995). Instructional design expertise: A cognitive model of design.Instructional Science, 23, 321–349.
Phelps, R.H., & Shanteau, J. (1978). Livestock judges: How much information can an expert use?Organizational Behavior and Human Performance, 21, 209–219.
Pirolli, P. (1992).Knowledge and processes in design: DPS final report. University of California at Berkeley, Graduate School of Education. (ERIC Document Reproduction Service No. ED 354 252)
Rahilly, T.J. (1991).An analysis of three learner-based formative evaluation conditions. Unpublished master's thesis, McGill University, Montreal, Canada.
Rowland, G. (1992). What do instructional designers actually do? An initial investigation of expert practice.Performance Improvement Quarterly, 5(2), 65–86.
Rowland, G. (1993). Designing and instructional design.Educational Technology Research and Development, 41(1), 79–91.
Russell, D.M., & Pirolli, P. (1992).Computer assisted instructional design for computer-based instruction.Final report. Berkeley, CA: National Center for Research in Vocational Education. (ERIC Document Reproduction Service No. ED 354 872)
Shanteau, J. (1992). The psychology of experts: An alternative view. In G. Wright and F. Bolger (Eds.),Expertise and decision support (pp. 11–23). New York: Plenum Press.
Simon, H.A., & Lea, G. (1974). Problem solving and rule induction: A unified view. In L.W. Gregg (Ed.),Knowledge and cognition. Potomac, MD: Erlbaum.
Smith, P.L., & Wedman, J.F., (1988). Read-think-aloud protocols: A new data source for formative evaluation.Performance Improvement Quarterly, 1(2), 13–22.
Stepp, S.L., & Shrock, S.A. (1991).The validity of a multiple-choice, paper and pencil instrument in discriminating between masters and nonmasters of instructional design. (ERIC Document Reproduction Service No. ED 335 015)
Tessmer, M., & Wedman, J.F. (1990). A layers-of-necessity instructional development model.Educational Technology Research and Development, 38(2), 77–85.
Tremblay, D. (1994).An analysis of the role of expert reviewers in formative evaluation. Master's thesis, McGill University, Montreal, Canada.
Tripp, S.D., & Bichelmeyer, B. (1990). Rapid prototyping: An alternative instructional design strategy.Educational Technology Research and Development, 38(1), 31–44.
Waldron, J.S. (1973).Instructional development unit: A guide to organizing an instructional development unit in health science. Bethesda, MD: National Institute of Health (DHEW). (ERIC Document Reproduction Service No. ED 125 625)
Weston, C., Le Maistre, C., McAlpine, L., & Bordonaro, T. (1997). The influence of the participants in formative evaluation on the improvement of learning from instructional materials.Instructional Science, 25, 369–386.
Winn, W. (1990). Some implications of cognitive theory for instructional design.Instructional Science, 19, 53–69.
Author information
Authors and Affiliations
Additional information
This paper was originally presented at the Annual Meeting of the American Educational Research Association, Chicago, IL, March 1997.
The author acknowledges, with gratitude, the helpful comments made by Cynthia Weston, Wayne Nelson, and three reviewers on earlier drafts of this paper.
This research was supported in part by grants from the Social Sciences and Humanities Research Council of Canada (SSHRC) and Fonds pour la formation des chercheurs et l'aide a la recherche (FCAR).
Rights and permissions
About this article
Cite this article
Le Maistre, C. What is an expert instructional designer? Evidence of expert performance during formative evaluation. ETR&D 46, 21–36 (1998). https://doi.org/10.1007/BF02299759
Issue Date:
DOI: https://doi.org/10.1007/BF02299759