Modeling the Problem-based Learning Preferences of McMaster University Undergraduate Medical Students Using a Discrete Choice Conjoint Experiment
Received: 04 February 2005 Accepted: 12 January 2006 DOI:
Cite this article as: Cunningham, C.E., Deal, K., Neville, A. et al. Adv Health Sci Educ Theory Pract (2006) 11: 245. doi:10.1007/s10459-006-0003-6 Abstract Objectives: To use methods from the field of marketing research to involve students in the redesign of McMaster University’s small group, problem-based undergraduate medical education program. Methods: We used themes from a focus group conducted in an electronic decision support lab to compose 14 four-level educational attributes. Undergraduate medical students completed a discrete choice experiment composed of 15 web-administered, partial-profile, conjoint-choice tasks. Results: Latent class analysis revealed two segments with different preferences. Segment 1, (86% of students), preferred a problem-based approach with more small group tutorial sessions led by expert tutors who facilitated the tutorial process without teaching didactically. Segment 2, (14% of students), preferred more large group lectures, explicit learning objectives, expert tutors who taught didactically, and streaming options based on learning preferences. Although Segment 1 preferred smaller tutorial groups, simulations predicted these students would trade increases in tutorial group size for a conceptually integrated program that included tutorial problems based on core curriculum concepts, greater integration of the content of clinical skills training sessions and the tutorial curriculum, and a link between clerkship patient selection and the program’s curriculum. A majority of both segments would accept a more conceptually integrated program if the savings associated with increases in tutorial group size was reinvested in web-enhanced tutorial processes and computer-simulated health care problems. Conclusions: Most students preferred a small group, web-supported, problem-based learning approach led by content experts who facilitated group process. Students favored a program in which tutorial group problems, clinical skills training sessions and the patients selected for clerkship activities were more closely linked to core curriculum concepts. References Allenby, G.M., Arora, N., Ginter, J.L. 1995 Incorporating prior knowledge in the analysis of conjoint studies Journal of Marketing Research 32 152 162 CrossRef Google Scholar Arora, N., Allenby, G., Ginter, J.L. 1998 A hierarchical Bayes mode of primary and secondary demand Marketing Science 17 29 44 Google Scholar Arulampalam, W., Naylor, R., Smith, J. 2004 Factors affecting the probability of first year medical students dropout in the UK: A logistic analysis for the intake cohorts of 1980–92 Medical Education 38 492 503 CrossRef Google Scholar Barbour, R.S. 1999 The case for combining qualitative and quantitative approaches in health services research Journal of Health Services & Research Policy 4 39 43 Google Scholar Bearman, M., Cesnik, B., Liddell, M. 2001 Random comparison of ‘virtual patient’ models in the context of teaching clinical communication skills Medical Education 35 824 832 CrossRef Google Scholar Colliver, J.A. 2000 Effectiveness of problem-based learning curricula: Research and theory Academic Medicine 75 259 266 CrossRef Google Scholar Das, M., Mpofu, D.J.S., Hasan, M.Y., Stewart, T.S. 2002 Student perceptions of tutor skills in problem-based learning tutorials Medical Education 36 272 278 CrossRef Google Scholar DeGrave, W.S., Dolmans, D.H.J.M., Vleuten, C.P.M. 1999 Profiles of effective tutors in problem-based learning: Scaffolding student learning Medical Education 33 901 906 CrossRef Google Scholar DeSarbo, W.S., Ramaswamy, V., Cohen, S.H. 1995 Market segmentation with choice based conjoint analysis Marketing Letters 6 37 148 CrossRef Google Scholar Dev, P., Montgomery, K., Senger, S., Heinrichs, W.L., Srivastava, S., Waldron, K. 2002 Simulated medical learning environments on the internet Journal of the American Medical Informatics Association 9 437 447 CrossRef Google Scholar Dhalla, I.A., Kwong, J.C., Streiner, D.L., Baddour, R.E., Waddell, A.E., Johnson, I.L. 2002 Characteristics of first-year students in Canadian medical schools Canadian Medical Association Journal 166 1029 1035 Google Scholar Dolmans, D.H.J.M., Gijselaers, W.H., Moust, J.H.C., Grave, W.S., Wolfhagen, I.H.A.P., Vleuten, C.P.M. 2002 Trends in research on the tutor in problem-based learning: Conclusions and implications for educational practice and research Medical Teacher 24 173 180 CrossRef Google Scholar Dolmans, D.H.J.M., Wolfhagen, I.H.A.P., Scherpbier, A.J.J.A., Vleuten, C.P.M. 2001 Relationship of tutors’ group-dynamics skills to their performance ratings in problem-based learning Academic Medicine 76 473 476 CrossRef Google Scholar Eva, K., Reiter, H.J., Rosenfeld, J., Norman, G. 2004 The relationship between interviewers’ characteristics and ratings assigned during a multiple mini-interview Academic Medicine 79 602 609 CrossRef Google Scholar Farrow, R., Norman, G.R. 2003 The effectiveness of PBL: The debate continues. Is meta-analysis helpful? Medical Education 37 1161 1132 CrossRef Google Scholar Hendry, G.D., Ryan, G., Harris, J. 2003 Group problems in problem-based learning Medical Teaching 25 609 616 CrossRef Google Scholar Holmes, D.B., Kaufman, D.M. 1994 Tutoring in problem-based learning: A teacher development process Medical Education 28 275 283 Google Scholar
Houldon, R.L., Collier, C.P., Frid, P.J., John, S.L. & Pross, H. (2001). Problems identified by tutors in a hybrid problem-based learning curriculum.
76: 81 Hubal, R.C., Kizakevich, P.N., Guinn, C.I., Merino, K.D., West, S.L. 2000 The virtual standardized patient. Simulated patient-practitioner dialog for patient interview training Studies in Health Information Technology 70 133 138 Google Scholar
Huber, J. & Miller, R. (1999). Dealing with product similarity in conjoint simulations.
Sawtooth Software Research Paper Series
. Retrieved November 1, 2004, from
http://www.sawtoothsoftware.com/download/techpap/prodsim.pdf Hudson, J.N. 2004 Computer-aided learning in the real world of medical education: Does the quality of interaction with the computer affect student learning? Medical Education 38 887 895 CrossRef Google Scholar Khoo, H.E. 2003 Implementation of problem-based learning in Asian medical schools and student’s perceptions of their experience Medical Education 37 401 409 CrossRef Google Scholar Kulatunga-Moruzi, C., Norman, G.R. 2002 Validity of admissions measures in predicting performance outcomes: The contribution of cognitive and non-cognitive dimensions Teaching and Learning Medicine 14 34 42 CrossRef Google Scholar Lenk, P.K., DeSarbo, W.S., Green, P.E., Young, M.R. 1996 Hierarchical Bayes conjoint analysis: recovery of partworth heterogeneity in reduced experimental designs Marketing Science 15 173 191 Google Scholar Lohfeld, L., Neville, A., Norman, G. 2005 PBL in undergraduate medical education: A qualitative study of the views of Canadian residents Advances in Health Sciences Education 10 189 214 CrossRef Google Scholar Maudsley, G. 1999 Roles and responsibilities of the problem based learning tutor in the undergraduate medical curriculum British Medical Journal 318 657 661 Google Scholar Moffat, K.J., McConnachie, A., Ross, S., Morrison, J.M. 2004 First year medical student stress and coping in a problem-based learning medical curriculum Medical Education 38 482 491 CrossRef Google Scholar Morgan, A., Shackley, P., Pickin, M., Brazier, J. 2000 Quantifying patient preferences for out-of-ours primary care Journal of Health Services Service, Research and Policy 5 214 218 Google Scholar Neufeld, V.R., Woodward, C.A., MacLeod, S.M. 1989 The McMaster MD program: A case study of renewal in medical education Academic Medicine 64 423 432 CrossRef Google Scholar Neville, A.J. 1999 The problem-based learning tutor: Teacher? Facilitator? Evaluator? Medical Teacher 21 393 401 CrossRef Google Scholar
Newman, M. & The Pilot Review Group (2003).
A Pilot Systematic Review and Meta-analysis on the Effectiveness of Problem-based Learning. Newcastle: Learning and Teaching Subject Network for Medicine, Dentistry, and Veterinary Medicine. http://www.ltsn-0.ac.uk/resources/features/pbl Norman, G.R., Schmidt, H.G. 1992 The psychological basis of problem-based learning: A review of the evidence Academic Medicine 76 215 216 Google Scholar Norman, G.R., Schmidt, H.G. 2000 Effectiveness of problem-based learning curricula: Theory, practice and paper darts Medical Education 34 721 728 CrossRef Google Scholar Orme, B.K. 2006Getting Started with Conjoint Analysis: Strategies for Product Design and Pricing Research Research Publishers Madison Google Scholar
Osman, L.M., Mckenzie, L., Cairns, J., Friend, J.A., Godden, D.J., Legge, J.S. & Douglas, J.G. (2001). Patient weighting of importance of asthma symptoms.
56: 138–142 Patterson, M. 2003Partial Profile Discrete Choice: What’s the Optimal Number of Attributes. Sawtooth Software Conference Proceedings Sawtooth Software Sequim Google Scholar Payne, J.W., Bettman, J.R., Johnson, E.J. 1993The Use of Multiple Strategies in Judgment and Choice Lawrence Erlbaum Associates Hillsdale, NJ, England Google Scholar Phillips, K.A., Johnson, R., Maddala, T. 2002 Measuring what people value: A comparison of “attitude” and “preference” surveys Journal of Health Services Research 37 1659 1679 CrossRef Google Scholar
Reznich, C.B. & Werner, B. (2004). Facilitators’ influence on student PBL small group session on line information resource use: A survey. Downloaded from
BMC Medical Education
Riesenberg, L.A., Biddle, B.W., Erney, S.L. 2001 Medical student and faculty perceptions of desirable primary care teaching site characteristics Medical Education 35 660 665 CrossRef Google Scholar Ryan, G., Dolling, T., Barnet, S. 2004 Supporting the problem-based learning process in the clinical years: evaluation of an online Clinical Reasoning Guide Medical Education 38 638 645 CrossRef Google Scholar Ryan, M., Gerard, K. 2003 Using discrete choice experiments to value health care: Current practice and future prospects Applied Health Economics and Policy Analysis 2 55 64 Google Scholar Ryan, M., Scott, D.A., Reeves, C., Bate, A., Teijlingen, E.R., Russell, E.M., Napper, M., Robb, C.M. 2001 Eliciting public preferences for healthcare: A systematic review of techniques Health Technology Assessment 5 1 186 Google Scholar Salkeld, G., Ryan, M., Short, L. 2000 The veil of experience: Do consumers prefer what they know best? Health Economics 9 267 270 CrossRef Google Scholar
Sawtooth Software Inc (2004a).
The CBC Latent Class Technical Paper (version 3). Retrieved November 1, 2004, from http://www.sawtoothsoftware.com/download/techpap/lctech.pdf
Sawtooth Software Inc (2004b).
The CBC/HB System for Hierarchical Bayes estimation (version 3.2). Sawtooth Software Technical Paper Series. Retrieved November 1, 2004, from http://www.sawtoothsoftware.com/download/techpap/hbtech.pdf Schmidt, H.G. 1993 Foundations of problem-based learning: some explanatory notes Medical Education 27 422 432 CrossRef Google Scholar Schmidt, H.G. 1994 Resolving inconsistencies in tutor expertise research: does lack of structure cause students to seek tutor guidance? Academic Medicine 69 652 662 CrossRef Google Scholar Stanek, E.J., Oates, M.B., McGhan, W.F., Denofrio, D., Loh, E. 2000 Preferences for treatment outcomes in patients with heart failure: Symptoms versus survival Journal of Cardiac Failure 6 225 232 CrossRef Google Scholar Stark, P. 2003 Teaching and learning in the clinical setting: A qualitative study of the perceptions of students and teachers Medical Education 37 975 982 CrossRef Google Scholar Streiner, D.L., Norman, G. 2003Health Measurement Scales: A Practical Guide to their Development and Use Oxford Medical Publications New York Google Scholar Stromso, H., Grottum, P., Lycke, K.H. 2004 Changes in student approaches to learning with the introduction of computer-supported problem-based learning Medical Education 38 390 400 CrossRef Google Scholar Svoboda, T., Henry, B., Shulman, L., Kennedy, E., Rea, E., Ng, W., Wallington, T., Yaffe, B., Gournis, E., Vicencil, E., Asrur, S., Glazier, R.H. 2004 Public health measures to control the spread of the severe acute respiratory syndrome during the outbreak in Toronto The New England Journal of Medicine 350 2352 2361 CrossRef Google Scholar Tabachnik, B.G., Fidell, L.S. 2001Using Multivariate StatisticsFourth Harper Collins College Publishers New York Google Scholar Willis, S.C., Jones, A., Bundy, C., Burdett, K., Whitehouse, C.R., O’Neill, P.A. 2002 Small-group work and assessment in a PBL curriculum: A qualitative and quantitative evaluation of student perceptions of the process of working in small groups and its assessment Medical Teacher 24 495 501 CrossRef Google Scholar Woodward, C.A., Ferrier, B.M. 1982 Perspectives of graduates two or five years after graduation from a three-year medical school Journal of Medical Education 57 294 302 Google Scholar