Medical students’ learning approaches and examination outcomes: longitudinal and cross-sectional studies in a problem-based system

Entwistle learning approaches are an evidence-based lens for analysing and improving student learning. Quantifying potential effects on attainment and in specific medical curriculum types merits further attention. This study aimed to explore medical students’ learning approaches in an integrated, problem-based curriculum, namely their validity, reliability, distribution, and how they change with student progression; their association with satisfaction; their association with cumulative attainment (examinations). Within the pragmatism paradigm, two series of mixed-methods questionnaires were analysed multi-cross-sectionally and longitudinally. Of seven surveys of Liverpool medical students (n ~ 115 to 201 responders, postal) and one of prospective medical students (n ~ 968 responders, on-campus), six included Entwistle 18-item Short RASI—Revised Approaches to Studying Inventory and six included ‘satisfaction’ items. Comparing four entry-cohorts, three academic years (9-year period), four levels (year-groups), and follow-ups allowed: cross-tabulation or correlation of learning approaches with demography, satisfaction, and attainment; principal components analysis of learning approaches; and multiple regression on attainment. Relatively high deep and strategic approach and relatively low surface approach prevailed, with strategic approach predominating overall, and deep and strategic approach waning and surface approach increasing from pre-admission to mid-Year 5. In multivariable analysis, deep approach remained associated with sustained (cumulative) high attainment and surface approach was inversely associated with passing Year 1 examinations first time (adjusted odds ratio = 0.89, p = 0.008), while higher ‘satisfaction’ was associated with higher strategic and lower surface approach but not with attainment. This study illuminates difficulties in maintaining cohesive active learning systems while promoting deep approach, attainment, and satisfaction and dissuading surface approach.


Background
Deep, surface, and strategic learning approaches are an evidence-based lens to study complex health professions education. Discussing them critically may well clarify ways to improve educational practice and student attainment (Dinsmore and Alexander 2012), consistent with education scholarship goals ["improving the readiness… for unknown and unexpected futures" (Eva 2020, p. 492)]. Learning approaches are patterns of motivation, strategy, self-regulation, and behaviour that vary with personal characteristics, context, discipline, curriculum, content, task, workload, and assessment (Baeten et al. 2010;Entwistle and Ramsden 1983;Kember et al. 2008;Lonka et al. 2004;Sadlo and Richardson 2003;Scouller 1998). While ideally, all learners want to learn better (Kytle 2012), some struggle to flex their learning (Fabry and Giesler 2012) or to 'meet minds' with educators (Karagiannopoulou and Entwistle 2019). Deep approach requires flexibility, which resonates with the scholarships of education and of discovery, i.e. connecting, communicating, and constructing knowledge (Maconachie 2002).
Entwistle built his deep-surface-strategic theoretical framework on Marton and Säljö's (1976a, b) seminal qualitative evidence of university students studying long texts using deep or surface approaches depending on how they perceived tasks and outcomes. Entwistle and Ramsden (1983) added strategic approach .
The evidence base uses various Entwistle instruments (Entwistle and McCune 2004) and Biggs' (1979) Study Process Questionnaire (SPQ) (Biggs et al. 2001). Entwistle's (2000) 18-item Short RASI (Revised Approaches to Studying Inventory ASI) is just one way of measuring the three subscales on an individual: • Deep approach intrinsically seeks meaning (intends to understand); is interested in and relates ideas; seeks evidence; seeks patterns and underlying principles. • Surface approach is extrinsically motivated to avoid disapproval or failure (intends to learn what is required) and cope with course; memorizes and reproduces in wrong situations; lacks purpose; is syllabus-focussed. • Strategic approach competes to achieve; adapts, is organized; manages time; is assessment-focussed.
Of note, these deep-surface-strategic terms label the approaches rather than the individuals (Coffield et al. 2004a, b;Coffield 2008) and, while deep and surface scores tend to be inversely related, strategic is arguably a subdomain (Biggs et al. 2001;Kember and Leung 1998) closer to deep approach. Deep approach should integrate "the whole with its purpose… the parts into a whole… the task with oneself" (Entwistle 2018, pp. 70-71) to give meaning, connect knowledge, and use personal experience, respectively.
Illustrating the versatility of Entwistle learning approaches for different countries, disciplines, and purposes, the University of Helsinki reported institutionwide use of a modified 12-item Entwistle Approaches to Learning and Studying Inventory (ALSI) complementing another Entwistle inventory about the perceived learner-educator environment. Measuring learning approaches helped to counsel students, enhance their deep approach (seek understanding, seek evidence, and relate ideas), and investigate learning in many disciplines. The other inventory identified student problems with stressful workloads, time management, or disorganized studying, and together these instruments prompted curriculum redesign, sharing of good practice, and staff development.

Evidence and theory
Evidence and theory about learning approaches span higher education. Health professions education should benefit from further exploration though of how much learning approaches adapt, their association with satisfaction, and their association with attainment:

Change with progression
Many university disciplines prefer deep approach, but promoting it is an enduring challenge (Asikainen and Gijbels 2017;Baeten et al. 2010;Mattick and Knight 2007;Reid et al. 2005) requiring constructive alignment with student experience (Biggs et al. 2001). Asikainen and Gijbels's (2017) systematic review of 43 longitudinal studies dispelled assumptions that deep approach must automatically improve just by attending university. Deep approach may well wane while surface approach increases (Entwistle and Ramsden 1983), yet longitudinal evidence suggests that medical students using a deep approach do likewise as doctors and are happier career-wise (McManus et al. 2004).

Association with satisfaction
In theory, deep or strategic approach should enhance complex practice and satisfaction. Deep approach has been associated positively with student satisfaction about educators, curricula, and collegiate experience generally (Prosser and Trigwell 1990;Nelson Laird 2008). Better learning possibly "[transforms structures learnt from the educator] into a personally satisfying form" (Entwistle and Entwistle 1991, p. 223).

Association with attainment
Ultimately though, student attainment is crucial and should reflect meaning-directed learning. For example, Vermunt's (2005) meaning-directed learning (akin to deep approach and particularly age-and discipline-dependent) is associated with good attainment.
Besides prior academic performance, aptitude (intellect), and sociodemographic factors (age, sex, socioeconomic status), there are numerous likely predictors of undergraduate 'grade-point average' (GPA) (Richardson et al. 2012). The learning approaches domain (deep, strategic, surface) accounted for only 3/42 non-intellectual predictors that Richardson et al. (2012) identified, overlapping with four other psychological domains (personality, motivation, self-regulation, and learning strategies) and psychosocial context. Learning approaches and personality might mediate how ability affects academic performance (Chamorro-Premuzic and Furnham 2008), but quantifying possible attainment effects of learning approaches, particularly deep, is elusive (Lindblom-Ylänne and Lonka 1999;McCune and Entwistle 2011). Recommended research strategies include multivariable analysis of individual follow-up from (pre)-entry to graduation while acknowledging the learning context (Richardson et al. 2012). Specific academic curricula need exploring to ensure that assessments across higher education are "rewarding engaged critical thinking and connections [being built]" (Herrmann et al. 2017, p. 40).
In medicine, Lindblom-Ylänne and Lonka (1999) found that students whose study orientation embraced deep approach achieved significantly better preclinical and clinical grades. While deep approach may be "a necessary, but not a sufficient, condition for productive studying" (Lonka et al. 2004, p. 307), success in medicine might also require the persistence of strategic approach, required for "organized studying and effort" (Entwistle 2018, p. 239). Furthermore, students must also self-regulate well, conceive knowledge relativistically, and not be derailed by traditional medical learning environments (Lindblom-Ylänne and Lonka 1999). Some individuals who are 'reproduction-orientated' (towards memorizing rather than constructing knowledge) and 'externally regulated' (dependent on others to control learning) show atypical scoring patterns across their learning approaches and conceptions of knowledge. Such 'dissonant study orchestrations' are associated with suboptimal performance and can include high-scoring on both deep and surface approach, induced by misleading curriculum cues (Lindblom-Ylänne and Lonka 1999;Entwistle 2018). Unexpected scoring across learning approaches should prompt researchers to consider how the academic environment might derail deep approach and confuse students (Entwistle 2018). Considering the academic context (educational philosophy and curriculum implementation) is crucial when researching learning approaches, particularly aspects that might favour one or more of the three approaches. Researchers should also consider how students perceive their knowledge and their educator's role as further potential influences.
Other evidence has been unconvincing or unsupportive. For example, on multiple regression, Leiden et al.'s (1990) Years 1-3 Nevada medical students showed no significant association between deep, strategic, or surface approaches (from ASI) and grade-point average (GPA) or National Board of Medical Examiners (NBME) Part I scores. This was despite correlations, respectively, being (albeit unquantified and non-significant): positive, positive, negative. Assessment outcomes in conventional curricula tend to correlate negatively with surface approach rather than positively with deep approach (Clarke and McKenzie 1994). Strategic approach might even outperform deep approach (Ward 2011), fostering academic success for some medical students and less anxiety (e.g. Year 1 anatomy, Cipra and Müller-Hilke 2019) plus better objective structured clinical examination (OSCE) performance (Year 3, Martin et al. 2000).
Educational context, problem-based curricula, and medical students Reid et al. (2012) urged further exploration by curriculum type, particularly problem-based, after finding medical students' learning approaches hardly changed over a hybrid programme where PBL focussed on promoting deep approach. Theoretically 'whole-system' problem-based curricula (Dangerfield et al. 2007) should improve deep approach (Baeten et al. 2010), based on improving active learning by constructing flexible knowledge, critical reasoning, self-directed learning, collaborative skills, and intrinsic motivation (Hmelo-Silver and Eberbach 2012). Aligning with deep approach, students discuss how concepts and principles relate, apply these to authentic scenarios, and integrate resources, knowledge, and skills (Dolmans et al. 2016), but evidence of this encouraging better learning approaches is mixed after early promise. Coles (1985) found learning approaches worsened significantly by the end-of-Year 1 in a conventional medical curriculum, but surface approach decreased significantly in a problem-based curriculum (from a similar baseline) while deep approach held steady. Cross-sectionally, Newble and Clarke (1986) found that Years 1 and 3 had significantly higher deep and lower surface approaches (but similar strategic approach) in a problem-based versus conventional curriculum. While in the former, deep learning scored similarly between Years 3 and 5/6, in the latter deep learning increased significantly and progressively over Years 1, 3, and 5/6. Dolmans et al.'s (2016) systematic review responded to "inconsistency and ambiguity" (p. 1089) in research about deep and surface approaches across higher education (e.g. Baeten et al. 2010). They concluded that PBL does indeed improve deep approach, even though average effect-sizes (Cohen's d) were small and nonsignificant-at + 0.11 standard deviations for deep approach (positive effect in 11/21 studies, negative effect in four, no effect in six, and more positive for whole curriculum implementations). Good student experiences of the occupational therapy educational environment are associated significantly with deep approach, positively, but the association is stronger with surface approach, negatively (Sadlo and Richardson 2003). Discouraging surface approach may indeed be more important than promoting deep approach (Ward 2011). Chung et al. (2015) found that Year 1-3 medical students' deep approach was unchanged after case-based learning. Regarding wider implications, Cipra and Müller-Hilke (2019) found that surface approach was associated significantly with trait anxiety (r s = 0.50) in a problem-based curriculum, risking burnout from suboptimal attainment.
Besides curriculum ethos and setting, assessments might change students' preferred learning approaches (Baeten et al. 2010;Scouller 1998). Heavy assessment might even promote surface approach [especially immediately pre-examination (Newble and Clarke 1986)] and strategic approach (Ramsden 1988). Multiplechoice tests might promote surface approach, whereas essay-type coursework promotes deep approach, but both show surface approach associated negatively with performance (Scouller 1998).

In summary
Priorities in learning approaches research include more longitudinal analyses (Chonkar et al. 2018;Dolmans et al. 2016;Herrmann et al. 2017), how much medical students change their dominant learning approach (Ward 2011), the elusive relationship between deep approach and attainment (Entwistle 2018) for specific disciplines and curriculum types (Herrmann et al. 2017), and the relevance of satisfaction (Rienties and Toetenel 2016). We might therefore ask: How do learning approaches change and relate to satisfaction and attainment in an integrated, problem-based curriculum?, expecting that such whole-system designs should improve learning, attainment, and satisfaction.

Aim
To explore medical students' learning approaches in an integrated, problem-based curriculum, namely (a) their validity, reliability, distribution, and how they change with student progression, (b) their association with satisfaction, (c) their association with cumulative attainment (examinations).

Setting
The 5-year Liverpool problem-based curriculum admitted medical student entrycohorts (n ~ 200 to 270), by two-person interview, between 1996 and 2013. Problem-based learning (PBL) in groups of 7-10 was the main learning mode, particularly in Years 1-2. A problem-based philosophy of active learning used clinically relevant scenarios for context, complemented by early clinical simulation and contact (clinical skills, communication skills, clinical placements); spiral curriculum; community-orientation; and integration across subject boundaries with no preclinical-clinical divide. The goal was more meaningful learning and attainment and less overload. Focussed on critical understanding, written examinations tested core content by multiple-choice, extended-matching, and short-answer items (including key clinical features). OSCEs assessed clinical and communication skills; a Year 4 clinical examination involved patients. Liverpool 4-year graduate-entry programme started in 2003 for n ~ 29 students, combining 2 years into one bespoke year before students joined main Year 3. From 2006, a satellite campus at another university also delivered the 5-year Liverpool degree for n ~ 50, before becoming its own degree-awarding medical school 12 years later. Initial national notification of these satellite places came too late to start a new selection process, so inaugural entrants were mostly applicants just missing the interview score for a Liverpool main-campus place.
Despite the community orientation, only 16% of 'subsequently admitted' prospective medical students expressed their career intention as general practice (Maudsley et al. 2010). Of these though, three-quarters retained that intention when surveyed 5 years later (2006/2007) (n = 74 double responders who remained in cohort by Year 5 or intercalated another degree between Year 4-5). Both those surveys and several similar contemporaneous surveys had also captured Entwistle learning approaches, allowing this current study to collate evidence comprehensively across and within four entry-cohorts.

Design
Situated within the pragmatism paradigm (Creswell 2003;Maudsley 2011) and receiving appropriate approvals (see "Ethical approval" statement), this study extracted learning approaches and related data from mixed-methods questionnaires designed for two study-series by the author. The S-series of six questionnaires S1-S6 (e.g. Maudsley et al. 2007Maudsley et al. , 2008Maudsley et al. , 2010  This new analysis now collates and explores learning approaches data from six of the above questionnaires and context about satisfaction from six and about 'ideal tutor' perceptions from four. Questionnaire cover letters outlined the research, assured confidentiality, and stated that the unique identifier allowed reminders plus linking to other (outcome) information. The questionnaires sought parental occupation (coded medical or not), postcode for Townsend score (1999 and 2001 entry-cohorts) (Hoare 2003), and ethnicity of United Kingdom (UK)-'Home' students [as per Office for National Statistics (ONS) censuses: 1991, 2011 (Laux 2019)]. Class lists allowed cross-checking or supplementation of demographic data about age, sex, and 'Home' (UK plus European Community unless otherwise stated) versus 'overseas' status.
Three item sets used 5-point Likert items (•Agree = 5 •Agree somewhat = 4 •Unsure = 3 •Disagree somewhat = 2 •Disagree = 1), as follows: • Entwistle Short RASI (18-item) learning approaches (from Approaches and Study Skills Inventory for Students (ASSIST), with permission) in six questionnaires, with each subscale scoring /30, six items per subscale (items listed in e-Appendix 2), e.g. Deep subscale: "Ideas in course books or articles often set me off on long chains of thought of my own". • Three items designed as proxies for 'satisfaction' (piloted in S3, then used unchanged): Five questionnaires used all three; a sixth (K3) used item 3 only should…': Open-ended answers in a prior study (end-of-Year 1, S2, 1999 entry-cohort, Maudsley et al. 2008) informed a 38-item pilot-set for S3, used unchanged in S5 and S6 (Maudsley 2005), then reduced to a 24-item set for K1 (Maudsley 2009) (e-Appendix 1), e.g. "Give us the faculty learning objectives". These four questionnaires generated regression-based z-scores (mean = 0, standard deviation = 1) on the two top components from principal components analysis (data not shown, see Maudsley 2005Maudsley , 2009), i.e. "tell me what to learn" and "help me with how to learn" (e-Appendix 1), which in K1, for example, had Cronbach's alpha of 0.7 and 0.6, respectively, for items loading at ≥ 0.4.
Data checking (e.g. distributions, missing data) and description (using IBM-SPSS-24 and StatsDirect-3) involved simple frequencies and cross-tabulations. Comparison of response rates (%) was by cohort, age, sex, and (non-)White British.

Analysis
Analysis of learning approaches (using statistical significance at p < 0.05) explored (e-Appendix 4) the following: Using forced-entry multiple logistic regression (IBM-SPSS-24), several models provided odds ratios (ORs) for 5-year students' cumulative examination outcomes. The dependent variable was 'Progressed in-cohort by passing all examinations with(out) resit(s)'. To avoid overfitting (Babyak 2004) and drawing on Field (2000) and Sperandei (2014), choice of number and type of independent variables heeded the following: • Indicative number ~ n/10, where n = smallest number of students passing with(out) resit(s) • Expected associations from literature; associations noted in targeted crosstabulations; possible explanatory power (Cox and Snell R 2 , Nagelkerke R 2 ); avoidance of multicollinearity (checking variance inflation factors via multiple linear regression using each independent variable as the dependent variable) • Stability [Hosmer-Lemeshow p value; unstandardized coefficients with p value; model significance (Omnibus p)] and small subgroups • Intuitive importance Forced-entry multiple linear regression (IBM-SPSS-24) on 'cumulative pointsscore' considered ~ n/10 variables (n = overall sample), analysis of variance (ANOVA) for model p value, and adjusted R 2 for potential variation explained. The most stable and intuitive model of B-coefficients was chosen.

Survey response
Across eight studies, response rates varied markedly (Table 1; e-Appendices 1-footnote, 3, 5) from 31 to 91%. Differences between responders and nonresponders were minimal. Table 1 Overall response rates for eight questionnaire surveys of Liverpool medical students: all except two* included a section for Entwistle Short RASI learning approaches Entwistle follow-up data are indicated as: S3→S6: n = 160 paired (responded both at start-and end-ofyear 1); S4→K2: n = 74 paired (responded as applicants and again at mid-year 5) RASI Revised Approaches to Studying Inventory (RASI) from Approaches and Study Skills Inventory for Students † The labels refer to two overlapping study series, the earlier S-series, S1-S6 (with only S4-S6 data reported here) and the later K-series, K1-K4 (all contributing to the current study). See e-Appendix 1 for items included. Maudsley et al. (2010)

Context: Student satisfaction and perceptions of the tutor
From six surveys, cross-sectionally in three entry-cohorts and longitudinally in a fourth (start-to-end-of-Year 1, then to end-of-Year 5), the vast majority of responders 'agreed + agreed somewhat' that, given their time again, they would still do Medicine (e-Appendix 3). The cross-sectional downtrend in the four entry-cohorts was significant ( Just under one-half of declining satisfaction (0.34 of the 0.78 above) was potentially attributable to Medicine itself. A problem-based design rather than locality or implementation potentially underpinned the remainder (0.65-0.34 = 0.31 versus 0.78-0.65 = 0.13). 'Satisfaction' with Medicine in the Liverpool problem-based curriculum did not change significantly in paired mid-to-end-of-Year 1 data (p = 0.190) (e-Appendix 3).
In all three surveys containing the tutoring items, 'ideally… tutor should tell me what to learn' correlated negligibly with 'Medicine' satisfaction but (mostly highly) significantly and negatively with problem-based and Liverpool problem-based satisfaction items (r s 2 potentially only explaining 4-8% though). The lower the expectations for PBL tutors to 'tell me what to learn', the more satisfaction (e-Appendix 3).

Validity and internal consistency
Learning approaches showed modest-satisfactory internal reliability over the six surveys. Cronbach alpha was highest for strategic approach (Table 2)    In principal components analysis, the 18-item model had good factorability (Kaiser-Meyer-Olkin (KMO) measures = 0.7-0.8), validating its use. Items mostly loaded as expected in the best (3-component) solution, showing modest-satisfactory reliability (e-Appendix 2).

Distribution
Learning approaches showed similar distributions in all six surveys (Table 2;  Cross-sectionally, strategic approach predominated in about one-half to two-thirds of responders, surface approach predominated least (1.0%) at pre-entry, and deep approach predominated most for start-of-Year 1 (40.8%) (e-Appendix 5).
Mean pre-entry scores (Table 2) (and proportion with subscale predominating and overall points proportion for subscale: data not shown) did not differ significantly between responders subsequently admitted or not. Of admitted responders, similar proportions of 2002 entrants (n = 220) or deferred 2003 entrants (n = 18) had each learning approach predominating [data not shown]. Deferred entrants' overall proportion of points for strategic approach appeared slightly lower (39.3% versus 41.7%), but p = 0.056 (e-Appendix 5). Mean strategic score appeared lower, but p = 0.079 if equal variances assumed (Table 2-footnote).

Relationship between satisfaction and learning approach
In all four cohorts, strategic approach correlated significantly and positively (r s 2 potentially explaining 4-19%) and surface approach significantly and negatively (potentially explaining 3-24%) with the satisfaction items.

Overview of summative examination outcomes available in the three entry-cohorts
Most 1999 entrants still in-cohort by end-of-Year 4 had passed all examinations first time at all three levels (end-of-Y1, mid-Year 3, end-of-Y4): 118/184 (64.1%).

Overview of summative examination outcomes by subgroups in the three entry-cohorts
Of the 1999 entrants, cumulative performance by end-of-Year 4 (passing versus resitting) did not differ significantly by: age, male, White British, Home status (versus 'international'), medical parent(s) [data not shown], affluence of England and Wales resident postcodes at entry, or 'would still do Medicine in this Liverpool problem-based curriculum' (e-Appendix 6-footnote). Mean regression-based score about ideal tutor (from principal components analysis, mid-Year 3) did not differ significantly either: -'ideally… tutor should tell…': −0.12 (n = 88 with no resits) versus 0.14 (n = 53) (t(139) = −1.46, p = 0.146) -'ideally… tutor should help…': 0.13 versus −0.18 (t(91.7) = 1.78, p = 0.079, assuming unequal variances, as Levene's Test: F = 4.34, p = 0.039). Statistical evidence was insufficient to support a suggestion here that 'passing all examinations first time throughout' was associated with viewing the ideal tutor as 'helping' with learning (rather than 'telling me what to learn').
Those passing everything without resits were more likely to be:

Overview of bivariable analysis of examination outcomes by learning approach in the three entry-cohorts
In bivariable analysis, passing without resits was associated with learning approach in all three cohorts (e-Appendix 6). For the 1999 entrants, at each of the three assessment levels the general pattern was higher mean deep and strategic approaches and lower surface approach (mid-Year 3 scores) if passing first time versus resitting [data not shown], but this was non-significant. End-of-Year 1 examinations gave the strongest evidence, with slightly lower mean surface approach of 15.3 (n = 123) versus 17.0 (n = 32) (t (153)

For the 2006 entrants staying in-cohort plus peers with them in
Year 5, their strategic and surface approaches (measured end-of-Year 5 or end-of-intercalationyear) showed no clear pattern of association (r s ) with the 'cumulative points-score', a top three-quarters 'cumulative points-score', or whether they had avoided resitting examinations throughout. A small positive association with deep approach was non-significant-higher mean deep approach versus a top three-quarters 'cumulative points-score' had the smallest p value, i.e. 21.2 (n = 67) versus 18.9 (n = 12) (t(77) = 1.92, p = 0.058: 95% CI on 2.26 difference −0.08, 4.61).
Overall in the bivariable analysis above, surface (negatively) and deep approach (positively) had the strongest associations with attainment (e-Appendix 6 summary table).

Multiple regression analysis on assessment outcomes
In the three entry-cohorts studied for 'progressing in-cohort' (the 2001 entrants through to end-of-Year 1 outcomes; the 1999 and the 2006 entrants cumulatively through to end-of-Year 4), the best multiple logistic regression models for 'passing all examinations without resits' and percentage variation possibly explained (without obvious collinearity) were as follows: the satellite campus (negatively) were significantly associated with the 'cumulative points score'. According to standardized beta, 'effect sizes' were 0.2-0.3 standard deviations per one standard deviation change in those four independent variables. Assuming similarity to Cohen's d (McGough and Faraone 2009) would deem this 'weak-modest', but maybe educational researchers interpret this as stronger (Kraft 2020) or even controversial (Simpson 2021). The model 'explained' about 18.1% of variation (p = 0.004, n = 77).
Age and satisfaction did not enhance any of the four models presented and nor did affluence in the two models with the available data [data not shown].

Summary of association between learning approaches and summative examination outcomes
Overall, in the bivariable and multivariable analysis above, deep approach was associated with Year 1-4 sustained high examination attainment and surface approach was associated inversely with Year 1 outcome.
In bivariable analysis (above and e-Appendix 6) • In the two entry-cohorts with analysis of Year 1 examination performance, the 2001 entrants showed it to be significantly inversely associated with surface approach, and the 1999 entrant data suggested similarly (albeit p = 0.053). • In the two entry-cohorts with analysis of sustained Year 1-4 examination performance, the 1999 entrants showed it to be significantly associated with deep approach, and the 2006 entrant data suggested similarly (albeit p = 0.058) for a top three-quarters Year 1-4 'medical school performance' score.
In multivariable analysis (above and e-Appendix 7a-d), progress through end-of-Year 1 examinations without resits remained significantly inversely associated with surface approach when adjusted for deep approach, Home status (which remained a significant predictor), and being male (2001 entry-cohort). In two further multivariable models, sustained Year 1-4 attainment remained significantly associated with deep approach when using • 'progress without resits' (adjusted for surface approach, 'ideally… tutor should tell…', Home status, and male) (1999 entry-cohort) or • 'cumulative points score for all assessments' (available only for this most recent 2006 entry-cohort). Of note, after adjustment, the 'cumulative points score' also remained inversely associated with medical parent(s), the satellite campus (which accepted slightly lower-scoring interviewees that initial year), and overseas ('not Home') status but not with strategic or surface approach or being male.

Discussion
This study of medical students found relatively high deep and strategic approach and relatively low surface approach in four entry-cohorts, with strategic approach predominating overall, and deep and strategic approach waning and surface approach increasing from pre-admission to mid-Year 5. Deep approach was associated with sustained (cumulative) high attainment and surface approach was inversely associated with passing Year 1 examinations first time, while higher 'satisfaction' was associated with higher strategic and lower surface approach but not with attainment. Effect sizes and changes were relatively modest but, despite some Short RASI items being ill-suited to this specific curriculum, learning approaches were consistent. Pre-entry and Year 1 measures are important. Baseline learning approaches probably influence how students: perceive their educational context and tasks; choose learning approaches; and perform academically (Arnold and Feighny 1995).
The conceptual lexicon of deep-strategic-surface approaches remains crucial in discussing how best to learn (Coffield et al. 2004a, b) and capturing complexity in higher education, including "everyday idiosyncrasy"  p. 104).
The current study had strategic approach highest-scoring pre-entry, unlike Leiden et al.'s (1990) Year 3 medical students having strategic approach significantly higher than Years 1 and 2 (but that was cross-sectional). Strategic and deep approach then remained relatively high in both Year 5s studied, despite decreasing significantly (while surface learning increased significantly) longitudinally in individuals by mid-Year 5. Improving on medical students' relatively high entrylevel deep approach might be difficult-"What scores are 'deep enough' at each stage…?" anyway (Chung et al. 2015, p. 210). The literature reports very little such follow-up of individual medical students and pre-entry applicants for longerterm assessment outcomes as Richardson et al. (2012) recommended.
Of the significant r s correlations for each approach with itself across this preentry-to-mid-Year 5 period, deep = 0.36, strategic = 0.38, surface = 0.26, the first two were similar to McManus et al. (1998): 0.37, 0.34, 0.42 (all p < 0.001). McKee et al. (2009) reported similar but mostly non-significant changes crosssectionally between first and final year for nursing and medical students completing anonymous unlinked surveys-deep and strategic decreased slightly and surface approach increased.
In the current study, the only Year 1 change was strategic approach increasing significantly, whereas Tooth et al.'s (1989) Year 1 increased surface approach and decreased deep and strategic significantly and Reid et al.'s (2005) Year 2 decreased deep and strategic significantly. Whatever the change, most might occur in the first semester (Fabry and Giesler 2012). Concerning the longer follow-up here, after 4 years, intercalating responders scored significantly more on deep approach than non-intercalating Year 5, as for McManus et al. (1999), but Year 5 who had intercalated the previous year were in-between, suggesting a transient effect.
Reported research use of this specific Short RASI is unusual. Using the 2013 updated Short RASI with general practitioners (n = 544) across the UK, Curtis et al. (2018) found their mean deep, strategic, and surface approaches (/30) to be 20.9 (significantly higher), 20.2 (similar), 13.9 (significantly lower) versus general practice specialty registrars (n = 461): 20.2, 20.0, 15.2. Corresponding scores for end-of-Year 5 Liverpool medical students (e-Appendix 3) differed significantly, respectively, 21.1, 23.4, 15.3. Liverpool students thus approximated to GPs on deep approach and GP registrars on surface approach but out-scored both on strategic approach, possibly from having just completed an intense year of presenting a required portfolio of clinical and academic evidence for final summative assessment.

Learning approaches and satisfaction
In the current study, 'satisfaction' was unchanged during the Year 1 of individual follow-up but then declined significantly by mid-Year 5. Medicine itself possibly explained one-half of that decline. Satisfaction was significantly inversely associated with expecting tutors to 'tell…' you what to learn (albeit only 4-8% of variation), unlike how PBL is meant to be. Only one-half of end-of-Year 5 agreed (somewhat) that: …I would still do Medicine in this Liverpool problem-based curriculum, but this cohort's contemporaneous annual NSS result (Higher Education Funding Council for England 2011) was significantly more positive, with 70% being 'satisfied'.
Turkish medical students with deep approach predominating [on Biggs et al.'s Revised 2-Factor SPQ (2001)] were significantly more satisfied with PBL than if surface approach predominated (Gurpinar et al. 2013). In Baeten et al.'s (2010) review, more than the environment itself, students' perceptions of it (satisfaction with tutor, workload, or relevance) were positively associated with deep approach prevailing, supported by Gustin et al.'s (2018) path analysis.
In the current study, across four entry-cohorts, strategic and surface approaches correlated significantly positively and negatively, respectively, with satisfaction. Deep approach correlated significantly (positively) only for the earliest entry-cohort though, whereas Nelson Laird (2008) (using a comparable concept of satisfaction: "If you could start over again, would you go to the same institution you are now attending?") found such association consistently in senior students across all university disciplines.
Crucially, the current study showed no clear association between satisfaction and attainment, consistent with the large-scale findings across 151 Open University modules (Rienties and Toetenel 2016) and thus reinforcing that finding with medical students.

Learning approaches and summative examination outcomes
The 2001 entry-cohort had start-of-Year 1 surface approach significantly inversely associated with Year 1 examination performance as Papinczak (2009) also found in a problem-based curriculum, whereas Mattick et al. (2004) found a significant Year 1 association with deep approach for 'progress testing' performance.
The current study finding that deep approach was associated with sustained (cumulative) high attainment adds more insight to a scattered and tentative evidence base. On meta-analysis, Watkins (2001) and Richardson et al. (2012) found weak-modest correlations between university students' deep (positively), strategic (positively), and surface (negatively) approaches and academic scores. May et al.'s (2012) Year 4 higher-performing medical students on clinical examination scored significantly higher on deep approach (especially patient interaction and patient satisfaction) and lower on surface approach. McManus et al. (1998) found that, unlike learning approaches at application, Year 5 SPQ deep, strategic, surface approaches correlated (minimally but) significantly positively, positively, negatively with combined academic and clinical final examination performance.
Using Parpala and Lindblom-Ylänne's (2012) LEARN inventory and adjusting for age, sex, and year of study, Herrmann et al.'s (2017) multilevel model found surface approach inversely related to examination performance but variably so across social science disciplines. Unlike the current study, deep approach did not predict academic performance, but 'organized effort' (an extension of strategic approach) did, similar to Tooth et al. 's (1989) Year 1 medical students.
That the 2006 entrants' Year 1-4 'cumulative points score' remained inversely associated (after adjustment) with having medical parent(s) was interesting; that entry-cohort had also not shown association between medical parentage and medical school admission (Maudsley et al. 2010). Maybe medical parents misadvised about non-traditional examinations.
In this study, age, 'satisfaction', and affluence did not enhance the models despite (on bivariable analysis) students passing Year 1 examinations without resits in one entry-cohort being significantly slightly older and all four entry-cohorts having, respectively, strategic and surface approach correlating significantly positively and negatively with satisfaction. In contrast, Richardson et al.'s (2012) meta-analysis found older, female, and more affluent students performed significantly better academically, but each weighted r was only ~ 0.1. May et al.'s (2012) female medical students scored significantly higher on strategic approach (but similarly to males on deep and surface), which also correlated significantly with patient satisfaction in a clinical examination. Females' better study habits may also improve academic achievement (Alzahrani et al. 2018).
In this study, evidence in one cohort of higher sustained attainment by White British students fell short of statistical significance, but 'overseas' status remained a significant negative predictor of end-of-Year 1 and of sustained Y1-Y4 examination performance in models adjusted for relevant learning approaches in different cohorts. Isik et al.'s (2018) evidence among Amsterdam medical students suggested that poorer attainment in ethnic minority groups might reflect assessment type, for example, if it rewards strategic approach but they either used deep or errant strategic approach, but the current study adjusted for some learning approaches. Overall, well-documented poorer academic attainment in non-white UK medical students is under scrutiny as a complex inequality probably partly arising from negative stereotyping and suboptimal social learning networks (Claridge et al. 2018;Woolf et al. 2008Woolf et al. , 2012. Woolf et al.'s (2011) meta-analysis found that examiner bias or candidate communication skills were unlikely reasons as machine-marked written examinations showed similar-sized inequalities. In the current study, 'examinations' combined machine-marked items and anonymously marked items besides face-to-face practical examinations.

'So what?' for this type of curriculum
Why does this evidence still matter? Firstly, it cannot be assumed that medical students will necessarily improve their learning approaches just because the curriculum is problem-based. In the current study, while learning approaches were not associated with selection to medical school, deep and strategic approaches then waned significantly longitudinally and surface approach increased (pre-entry-to-mid-Year 5). The problem-based design and philosophy prevailed most though in Years 1 and 2 (weakening in senior clinical placements), and another entry-cohort did maintain deep approach longitudinally in Year 1. Balasooriya et al. (2009) found a similarly complex and polarized student response to an integrated, self-directed teamworking curriculum, whereby deep approach decreased and surface approach increased in some students, associated with intolerance of uncertainty and of integrated learning across body systems. Indeed, successful curriculum integration is tricky, requiring explicit support (Chipamaunga and Prozesky 2019). While Sadlo and Richardson (2003) found significantly higher deep approach and lower surface approach in problem-based versus subject-based curricula, in occupational therapy across several countries, their data were cross-sectional.
Secondly, students using much surface learning may well be uncomfortable with problem-based curricula, but satisfaction is not a key determinant of sustained high attainment. National attention on 'satisfaction'-focussed league tables might distract from using educationally robust ways of challenging students out of surface approach. Papinczak's (2009) high-scoring students on both deep and strategic approaches appeared protected in PBL, being more positive and less stressed about the experience, but a 'metacognitive intervention' (Papinczak et al. 2008) did not reverse worsening self-efficacy and deep and strategic approaches over Year 1.
Thirdly, showing that deep approach was associated with sustained (cumulative) high attainment complements an evidence base requiring more clarity about the key curriculum design features. Gustin et al.'s (2018) path analysis suggested that curriculum integration surpasses PBL in promoting deep approach, with educational context and students perception' of it possibly explaining one-quarter of variance in deep approach.

Strengths and limitations
As Dinsmore and Alexander (2012) recommended, the current study used a clear definition of learning approaches, from Entwistle's conceptual framework, and explored these within a specified type of learning environment. Furthermore, several entry-cohorts showed acceptable construct validity (by principal components analysis) with modest-satisfactory reliability (Cronbach's alpha) (Lance et al. 2006) of the Short RASI-the instrument has featured little in the literature. This study explored not just: "what is the relation between levels of processing and learning outcomes" but also "for whom, at what point in development, in what situations, and for what end?" (Dinsmore and Alexander 2012, p. 522).
Percentage questionnaire response ranged from low-modest to excellent, and responders were suitably representative overall. Applicant response rate was similar, for example, to McManus et al.'s (1998) postal response of 92%. It was a strength that this 'domain'/discipline-specific (Lonka et al. 2004) study of medical students involved multiple entry-cohorts, year-groups, and academic years and undertook some paired comparisons. Good governance rightly constrained research access though, i.e. when and how often different research projects could study specific entry-cohorts, but researching within everyday educational practice involves such compromises, as accommodated within a 'horses-for-courses' pragmatism paradigm. Only one medical school was studied, using self-reported data, with longitudinal comparisons mostly two-point and some rather small analytical subgroups. Relatively small sample sizes limited how many variables the multivariable models could legitimately test and regression analyses were not hierarchical to reflect likely complexity.
Analysed outcomes merged quite different assessment modes and contents, and potential selection bias lay in studying only those remaining in-cohort of the 5-year curriculum but not leavers, year-repeaters, or 4-year programme entrants. Despite some Short RASI items being unsuited to this curriculum though, a coherent pattern of modest associations emerged, even when measuring some learning approaches after the index assessment. Here, the two entry-cohorts followed for sustained examination performance unfortunately excluded the entry-cohort followed from pre-entry. Both multivariable models had rather weak explanatory potential (4-variable, including deep and surface: 7-9%; 5-variable, including deep: 13-18%) but were consistent in magnitude with Richardson et al.'s (2012) meta-analysis, where the deep-strategic-surface combination could explain 9% of variance in 'gradepoint average. Data were unavailable to adjust for many key influences, e.g. ability, prior academic qualifications, personality. The datasets for the three entry-cohorts were unsuitable to combine for a multilevel mixed analysis with entry-cohort as the clustering variable, and no suitable common assessment measure was available for repeated measures analysis.

Conclusion
In-depth analysis of learning approaches illuminated difficulties in maintaining cohesive active learning systems while promoting deep approach, attainment, and satisfaction and dissuading surface approach. This study provides further evidence about 'How much?' learning approaches might change and 'With what outcome?'.
High-quality learning has many barriers (Mattick and Knight 2007). How to prompt more deep approach in an integrated, self-directed curriculum based on small-groupwork is unclear and must be balanced with support and structure for struggling (Balasooriya et al. 2009) or dissatisfied students. While curriculum-assessment alignment should dissuade students (Biggs et al. 2001;Ramsden 2005) from the "unreflectiveness… unrelatedness… memorization" of surface approach (Entwistle 2018, p. 71), it is debatable by how much, particularly in different stages and contexts (and how much deep or strategic approach is required).
Research agendas should focus on the mechanisms that might explain-the relationships between learning approaches, satisfaction, and attainment; how struggling subgroups might be supported; and how learning systems might reassure students and improve satisfaction about learning and attaining in educationally robust ways.