Introduction

Evidence-based practice (EBP) is at the core of contemporary patient-centred healthcare [1]. It requires competency in understanding the research process and critical appraisal skills to evaluate research studies. EBP has been shown to improve clinical outcomes, decrease variation in patient outcomes, increase patient safety and reduce healthcare costs [1, 2]. Research competence also empowers health professionals to actively partake in and contribute to their research field and evaluate their own practice, rather than simply remaining as consumers of research publications [3]. However, two worldwide trends are evident: declining participation of undergraduates in health research, and diminishing number of practising health professionals involved in research, both of which form barriers to meaningful EBP, and need to be addressed with urgency [4, 5].

To better engage health professionals in research, early involvement has been shown to be advantageous [4]. Research training in health professional courses such as medical and allied health degrees encourages interest in research activities and fosters positive attitudes towards research through greater confidence and understanding of required research skills and competencies. This underpins a positive experience of being directly involved in research [6, 7]. Early research education also correlates with greater interest in a career as a clinician-scientist, with health professionals more likely to pursue research alongside clinical work [4, 5, 8]. Such early training in research is beneficial in promoting EBP, and effective in promoting future health professionals’ engagement in clinical research [9].

Project-based learning is currently the status quo to consolidate learning of research methods, through putting into practice active learning strategies to enhance deeper level understanding of a topic. This coupled with small improvement in students’ perceived understanding of research methods [10, 11] suggests a positive role for project-based interventions in research training. However, project-based learning alone does not appear to be sufficient for holistic understanding and development of research skills, and additional complementary interventions are likely recommended [10,11,12].

One promising style of intervention that can complement project-based research training is a centralised hub of e-learning resources that is freely accessible. Callaghan et al. [13] demonstrated benefit of having a comprehensive and centralised hub of resources which students can use at the time of encountering an issue with their research project [13]. Such a hub could help students grasp links between concepts, and use it to fill knowledge gaps supervisors are unable to cover [13]. With recent advancements in e-learning and m-learning (mobile-learning) as new methods for delivery, students can easily access a reservoir of knowledge that is not only versatile to access, but also more engaging [14, 15].

In 2017, The University of Western Australia (UWA) developed the ‘Health and Medical Sciences Research Modules’, (hereafter ‘The Modules’): a centralised hub for research training, suitable for health professional students from all clinical and social health disciplines. The contents are derived mainly from tertiary hospital and government research training resources developed for health clinicians in WA (see Appendix 1 for detailed description of the contents). It uses practical tips and real-world examples to complement theories, and aims to link UWA students with peers from health disciplines, and industry experts including clinicians. The Modules have been designed as a complementary resource for face-to-face teaching, and their components can be tailored to meet the needs of different student groups.

Although uptake of the Modules has increased, an evaluation of students’ experiences with content and delivery of the Modules has yet to be conducted. Therefore, this study seeks to identify the perception of future health professionals, who are in tertiary education, towards the centralised E-learning hub named ‘Health and Medical Sciences Research Modules’. The three key objectives are:

  1. 1.

    To explore the role the Modules can play in supporting students in their current research training courses.

  2. 2.

    To explore student perception of the selection of topics and content quality within the Modules.

  3. 3.

    To explore student engagement with Modules content.

Methods

Ethics approval for this research project was granted by The University of Western Australia Human Research Ethics Committee (RA/4/20/5508). COREQ, a consolidated criteria for qualitative study reporting, was used to inform reporting of this study [16].

Procedures and Participants

A qualitative descriptive study design was used to observe students’ experiences and perceptions; to describe at manifest level a participant’s experiences in his or her own words; and to capture the range of responses to a phenomenon [17,18,19]. Participants were recruited from 3rd- and 4th-year Doctor of Dental Medicine (4 years of postgraduate study), and 2nd-year Master of Pharmacy (2 years of postgraduate study) at UWA in Perth, Western Australia. Both courses offered access to the Modules, integrated into their teaching curriculum, in different formats. Briefly, the pharmacy students completed one topic from the Modules per week, followed by a face-to-face tutorial on the section content the week after (i.e. the envisaged usage of the Modules), and assessed via a mid-semester exam. The dentistry students self-selected topics from the Modules without any complementary face-to-face teaching from the faculty, assessed via an open-book multiple-choice exam.

Two sampling methods were used, stratified purposeful sampling, and snowball sampling. For the former, participants were recruited across two different disciplines, dentistry and pharmacy, in order to capture major variations [20]. This was done through two main pathways. For stratified purposeful sampling, staff members from both disciplines were asked to send out a standardised recruitment email to students, with two reminder emails. For snowball sampling, student societies were asked to distribute a standardised recruitment email, with four reminder emails. As interviews were conducted concurrently with recruitment, snowball sampling was conducted by asking participants at the end of each interview whether they knew anyone else who may be interested in participating in the study. The potential candidates were then approached via email. No volunteers were excluded.

Recruitment and interviews were undertaken until data saturation was achieved, defined as the point when no new information emerged during the semi-structured interview [21]. An initial sample size target of 10 to 15 was agreed upon by the research team, in accordance with commonly utilised qualitative study sample sizes [22, 23].

Interviews

The semi-structured interview guide was written with expertise of the research team and literature on writing interview guides, aligned with the objectives of the research [24]. The guide contained six questions: first question was an ice breaker, while the rest progressed logically through the three main objectives of the study. Suggested follow-up questions were included for each main question, as well as prompts to help participants with understanding. Please refer to Appendix 2 for the interview guide.

After training with a researcher with substantial qualitative research experience, skills such as active listening, reflecting, summarising, probing, use of silence, and how to avoid leading and double-barrelled questions were covered, to ensure comprehensive and trustworthy data [25]. Interviews with students were conducted using the online platform Zoom (by RQ) due to special circumstances of the COVID-19 pandemic. Each interview was initiated with a video chat to establish rapport; then, videos were switched to audio, which signalled commencement of interview. Only audio was recorded. At the conclusion of each interview, demographic data were collected, including sex, age, discipline, year of study, and highest education level obtained. This was done to provide additional context on participants’ backgrounds. Each interview lasted 15–20 min. No repeat interviews were conducted. Feedback for interview techniques were sought from each interviewee, at the conclusion of each interview. This assisted ongoing improvement of the researcher’s interview skills and the interview guide, and served to ensure quality and reliability of data.

Analysis

Demographic information (health discipline, age, gender, prior research experience) was coded by the interviewer (RQ) and frequencies and percentages were reported. Interviews were analysed thematically using the framework method [26]. Its matrix output allows systematic comparison of data across interview transcripts as well as within individual cases [26], to enable effective generation of themes.

Following completion of each interview, audio recording was transcribed verbatim by the interviewer, to provide a permanent record of what was and was not said, and allow for early immersion in the data. Line by line coding was done with the computer-assisted qualitative data analysis software (QSR Nvivo). The interviewer (RQ) and an additional researcher (KL) coded the first transcript independently, codes were compared, and a working analytical framework was formed. RQ then inductively coded the remaining transcripts using the working analytical framework. Following coding completion, KL independently coded one more transcript, and any discrepancies were discussed and resolved, to ensure quality of the analytical framework. Data were subsequently charted into the framework matrix with assistance of QSR Nvivo, and one matrix was generated for each category. Themes were generated from the framework matrix by reviewing the data set, and making connections between and within participants, categories and codes. The approach to thematic analysis was then divided further into manifest and latent levels, which reflect how themes are identified. Manifest level themes were derived from patterns directly observable in the statements made by participants, while latent level themes were identified from patterns underlying the information such as potential meaning behind why participants made certain statements.

Methods of Quality Assurance

A variety of Lincoln and Guba’s methods of quality assurance were used to guide this study [27]. Analyst triangulation was used during coding process, as two researchers coded the first transcript independently and then reconvened to develop the working analytical framework [27]. This was done again with the last transcript to revise the quality of the analytical framework and to reconcile any adjustments. This principle of triangulation enhances credibility of the study [27]. Any changes to the working analytical framework were recorded in an audit trail, thereby ensuring transparency, and enhancing study dependability and confirmability [27]. The interviewer created annotations and memos — which were recorded in the audit trial — throughout the coding process, to allow noting of interesting ideas, concepts, impressions, and any emerging themes and sub-themes. This assisted the interviewer to be more aware of researcher biases, and to practise reflexivity which increases credibility and confirmability of the paper [27]. Furthermore, detailed descriptions of findings were provided to increase study transferability, and negative cases were investigated to revise and confirm emerging themes, to improve credibility of the results [27].

Results

Overview

Eleven participants (seven females and four males) between ages of 22 and 24 completed the interview, including five third-year postgraduate dentistry students and six second-year postgraduate pharmacy students. Data saturation was reached by the 3rd interview for dentistry participants, and 5th interview for pharmacy participants; however, recruitment was continued to confirm saturation. Six out of the eleven participants had no prior research experience (55%), and five had some prior experience (45%). Seven out of the eleven participants were working on a quantitative study alongside the Modules (64%), and four were working on a qualitative study (36%).

Notable themes which reflect common trends in student responses are described below. Quotations are identified as DP (dentistry participant), or PP (pharmacy participant) Table 1.

Manifest Level Themes

Table 1 depicts the description and exemplary quotes for the manifest level themes
Full size table

Latent Level Themes

Content Volume and Balance

Participants wished for the Modules content to be more succinct, tailored to users’ background, and to adopt a better balance in the content focus.

Distilling the Content

Dentistry students who used the Modules without complementary face-to-face teaching described the amount of content as overwhelming and lengthy given the time they had to complete the Modules, and proposed the information be condensed. Of note, the solution of including summary sheets of different topics was suggested numerous times and could also help as a reference when needed. DP1 expressed: people tune out because the content is just too much. DP3 also stated: fundamentally the Modules has to be amended, like not so long…maybe they can condense it.

Poor Balance in Content Focus

Both pharmacy and dentistry students with research backgrounds indicated sections of the Modules overlapped with prior knowledge, and found the information repetitive. A few admitted to relying on past experience, and skipping through the lectures and slides. DP2 suggested the solution of being able to select which sections to study, and leaving the content they have learnt before already. DP2 said: if like at the start, there were options to select what you were interested in learning or building upon from past experience, then, it would have been more tailored and more effective. Many students, while acknowledging the difficulty in capturing broad topics, expressed preference for more details in certain topics. Statistics was mentioned the most as an example. PP5 said: I feel like the statistics one was very basic. I feel like after I did the statistics, I still don’t know so much about it.

Relevance of Content to Project

Both pharmacy and dentistry students wanted to know how the Modules content was relevant to their research projects. Conversely, where content was perceived as not relevant to the project, participants wanted to know how they were important to their future practice more broadly.

Tailoring of Content to Research Projects

Both dentistry and pharmacy students favoured sections of the Modules related to their research project. For instance, most students enjoyed the topic of literature searching and found it applicable for their literature review. Similarly, students doing qualitative research favoured the qualitative content, and vice versa. Conversely, they found themselves disengaged from the irrelevant sections, and wished for an option to customise the Modules. DP2 expressed: even something as simple as like having the option of selecting whether your research requires human ethics approval or consideration of those factors as opposed to purely lab work and not dealing with any biological material. DP3 offered the solution of instead of having blanket Modules proscribed for everyone’s training, to select out the Modules more suited for certain types of research projects, for instance Modules catered for laboratory-based studies.

Real-time Application to Research Project

Some dentistry students suggested the idea of having allocated time to work on specific topics, with the presence of research supervisors, so they can ask their supervisors questions regarding the Modules and also apply the knowledge in real time to their projects. Similarly, dentistry students also suggested having guidance from seniors who have done similar research projects, to assist them to apply the Modules to their respective projects. DP3 mentioned: further complement with research coordinator guidance would be perfect, for a holistic research skills learning. It will be even better if you have seniors guiding you as well…because seniors tend to have done similar research project.

Like Topics Relevant to Future Practice

For Modules content not applicable to the students’ respective research projects, dentistry and pharmacy students enjoyed them, when they could see the content’s relevance in their research skill development. Some examples include ethics, research governance, and how to critically appraise studies. For example, PP1 was appreciative of his newly developed ability to critically analyse studies, which PP1 used to analyse trials of new medications in healthcare. PP1 commented: I think it is a useful skill to have picked up. It is definitely a skill I’d be using in my career, when news headlines and information come up. Of note, PP4 suggested the inclusion of more real-world examples in the Modules, so students can place the Modules content into the perspective of future practice. PP4 mentioned: for qualitative, maybe they could have shown us an example of a qualitative interview.

Alignment and Sequencing

Participants wanted the Modules to be aligned with their in-class tutorials and assessments, and their project.

Alignment Between Modules and Face-to-Face Tutorials

Pharmacy students who received face-to-face teaching the week after learning Modules content found they were able to reinforce their knowledge from the Modules, and to apply them in the face-to-face setting. PP2 said: that really solidified anything that might have been a bit unclear from the online teaching. They found some tutorials to be a good complementary match to the Modules; statistics, qualitative and quantitative research, and critical appraisal tutorials were mentioned as examples. PP6 recalled: I just opened my notes for the critical appraisal one, and I would feel that was also, the tutorial was an extension of the Modules. Because in the tutorial, all we did was critically appraise two articles using what we’d learnt in the Modules…So I feel like that’s a complementary match, like they complemented each other.

Pharmacy students also pointed out, sometimes there was a misalignment between the Modules and face-to-face tutorials. They recalled the Modules content on research and governance never had a face-to-face tutorial to consolidate the learning, and at times face-to-face tutorials were mismatched in their content compared to the Modules. PP4 commented: sometimes, even if we have a tutorial, it’s not similar to the Modules work. There were also times when students found too much overlap between the face-to-face tutorials and the Modules. PP3 reported: perhaps if they just sort of make sure that the content they covered was not going to be the same, or if it is going to be the same content then just tell people it will be. Then if people feel like they've already got a good idea of it they don't need to go and listen to it again.

Confusion with Accessibility of Content/Direction from the Unit Coordinator

Pharmacy students who had a mid-semester exam based on the Modules and face-to-face tutorials experienced confusion regarding assessability of the content. They proposed for more direction from the faculty coordinator on which content is assessable, and which is not, given the significant volume of content. PP4 mentioned: so, I think in the midterm, everyone was a bit like, we don’t know what’s in it…Maybe more guidance or something, from the actual unit coordinator.

Interactivity

Participants wished for the content to be more interactive, rather than simply listening, e.g. incorporation of interspersed quizzes, and use of discussion forums.

Use of Quizzes

Many pharmacy and dentistry students proposed inclusion of quizzes interspersed with learning content, to reinforce knowledge and to increase interactivity. PP5 commented: I’d rather not someone talk for an hour, but more they talk for a bit, they give you a little mini-quiz. And then it’s interactive in that sense. A few students also noted this can add to the incentive of learning the content properly.

Discussion Forums

Most students weren’t aware of the function of discussion forum. Those who knew saw it as a good way to ask questions, as many students found it difficult to have their questions answered through other online E-learning platforms. They also proposed a discussion forum as a way to stimulate ideas, to enhance peer learning and to keep in touch with fellow students. Interestingly, PP4 suggested creating an interdisciplinary discussion board, involving dentistry, pharmacy, audiology students, and more. Furthermore, PP6 proposed making posting on discussion boards mandatory, as a way to incentivise students to learn the online content properly: everyone put one thing you’ve learnt about it up. At least then people would be more likely to watch them, because they’d know they’d have to put this thing up.

Discussion

The present study suggests overall, when used as designed (as demonstrated by the pharmacy curriculum), to complement face-to-face teaching with clearly outlined learning objectives, the Modules were well received and utilised. But when used as a standalone additional resource without accompanying instruction (as demonstrated by the dentistry curriculum), they were not. In addition, the issue of content oversaturation, the importance of highlighting the relevance of Modules content to participants’ research projects or skills learning, the need for alignment of face-to-face teaching with the Modules content, and the importance of greater interactivity all emerged as topics which can be addressed to improve the student learning experience and satisfaction. These topics should be evaluated when considering further implementation of centralised e-learning hubs to complement project-based learning, across Australia and worldwide.

Content Saturation

While most of the students found the Modules content to be well structured, of high quality, with a broad selection of topics, many wished for the content to be more succinct. The dentistry students especially reported the amount of information to be overwhelming, given the amount of time they had to finish the Modules and the lack of complementary face-to-face teaching. A growing body of evidence recognises the importance of curricular content management in health professions education. The Institute of Medicine has specifically noted the over-crowding of curricula as one major challenge of health education reform [28]. With a curriculum saturated with content, students often resort to memorisation and are left with no time to develop a deep understanding, or higher order thinking skills such as critical thinking and problem solving [29]. Diekelmann in his study on nursing education captured the frustrating and overwhelming impact of excessive content processing, memorisation and assignments on student learning experience [30]. Similar negative impacts on learning were also described in the current study, and students lacking prior research experience especially struggled with the heavy content. The dentistry students who were given the Modules without face-to-face support from the faculty also indicated the learning experience to have been especially overwhelming, and admitted to skimming through contents, and poor recall of information. Numerous contributing factors to content saturation have been identified; they include the shift from industrial age to information age, teacher-centred pedagogy, content repetition and gap between academia and practice [31]. The complex interplay of the numerous contributing factors makes content saturation difficult to tackle; however, the application of cognitive load theory (CLT) could offer possible solutions [32].

The participants’ experience with content saturation resonates with CLT, first developed by John Sweller in 1991 [32]. CLT describes learning and cognitive load to be determined by three important factors — characteristics of the learner, complexity of the information or task (known as intrinsic cognitive load (CL)), and presentation of the information (known as extraneous CL) [33, 34]. While expertise of the learner and intrinsic CL are more difficult to address, extraneous CL is widely recognised as the most addressable of the triad of CLT and can potentially be used to tackle the issue of content saturation. Either the Modules or the faculty curriculum can improve their methodology of information delivery. Potential strategies suggested by this current study include summary sheets at the end of each topic, chunking learning into small manageable units, and receiving regular support such as question and answer time from the faculty. However, further studies are required to explore the methodologies of information delivery, and the impact they can have on reducing extraneous CL and tackling the issue of content saturation, and subsequently improving the student learning experience.

Relevance of Content

Both pharmacy and dentistry students wanted to know the necessity of learning the content before delving into them. They wanted to know how the Modules content applied to their own research project, or to their future practice more broadly. This highly goal- and task-orientated learning approach reflects the evolving body of evidence behind the adult learning theory or andragogy [35]. A mosaic of models, principles and theories have been developed to compose the twenty-first-century knowledge base of adult learning, with most well-known being Malcolm Knowels’ theory of andragogy, which provides six guidelines for andragogy, three of which are highly pertinent to the findings of the current study [36]:

  1. 1.

    Adults want to clarify the necessity of learning the content before doing so.

  2. 2.

    Adults are more motivated to learn information which enable them to cope more effectively in real-life situations.

  3. 3.

    Adults are task or problem centred in their orientation to learning.

The students’ approach to the Modules was very pragmatic and task-orientated; some even proposed the idea of completing the Modules topics with the presence of their research supervisors, so they can apply the knowledge in real time to their own projects. When they were unable see the relevance to their own project, they looked for intrinsic motivation to learn by asking how the content would enable them to cope better in real-life situations in the future. For instance, one pharmacy student commented on how the ability to critically appraise studies can assist with the analysis of emerging headlines of new medications in healthcare. Those study findings, with support from Knowels’ theory, suggest the importance to incorporate motivational strategies to encourage students who are adult learners [37]. Such motivational strategies can involve highlighting the importance of the topics, demonstrating the relevance to the students’ tasks and lives in general, and providing practical information with real-world examples [37]. However, further evaluations are required to examine how andragogy can be used to create interventions that will improve student learning experience, for the wider implementation of the e-learning hub.

Alignment and Sequencing

The pharmacy and dentistry students enjoyed the flexibility of online learning, and the variety of medium of content delivery. Furthermore, the pharmacy students believed completing the online Modules topics followed by complementary face-to-face tutorials really assisted with knowledge consolidation. This format of the E-learning platform as a supplement to face-to-face research education aligns with the ‘flipped classroom’ model of teaching [37]. The ‘flipped classroom’ breaks the traditional classroom/homework paradigm, as it involves the utilisation of internet technology such as uploaded videos to deliver course content at home, and what is usually considered as homework such as assigned problems is done interactively in class instead [37]. In recent years, there has been a considerable increase in flipped classroom use, and although there is limited research into this approach, many of its core aspects of learning are well supported by research [38, 39]. A few examples include the allowance of more active learning exercises in class, such as teamwork, case studies and debates, therefore stimulating higher order thinking and improving learning outcomes [40]. The flipped classroom model also enables teachers to dedicate more time to individual students, assisting them with better understanding of course matters [41, 42]. The pharmacy students found the flipped classroom style of learning gave them a chance to practise what they learnt online in real time, to consolidate their learning and to clarify any confusion they had; this aligns with the evidence behind flipped classroom learning. The dentistry students who were given the Modules on their own found the learning experience to be overwhelming, which suggests an online platform alone is insufficient. A well-measured balance between the virtual and live components of a flipped classroom is invaluable. The current study findings and the evidence behind flipped classroom learning suggest that regular face-to-face tutorials to complement online learning are crucial, as is the alignment between the tutorials and online content, to improve student learning experience and satisfaction. This indicates that while consistent feedback to the learners is important, so is support for the instructors, as this would enable them to provide the best complementary tutorials which meet the learners’ needs.

Strengths and Limitations

Difficulties were encountered during the recruitment process. Despite numerous staggered recruitment emails sent to the dentistry and pharmacy faculty coordinators and student societies throughout 2020 and 2021, there was a suboptimal response rate from the students. This meant snowball sampling was more heavily relied on, which predisposed the study to a more homogenous sample that could impact the transferability of the study. Furthermore, recruitment and interviews were undertaken until data saturation was confirmed, which ended up being eleven participants. Despite achieving data saturation, with this small sample size, generalisability is a limitation of the study. However, this is an inherent limitation of qualitative research, but also not the aim of qualitative research, as qualitative research seeks to delve deeper into human experiences to learn about the breadth of experiences.

The less structured nature of semi-structured interviews meant the data collection process was more prone to researcher bias. However, an attempt to address this was done through field notes in the format of annotations and memos, made throughout data collection and analysis. This assisted the researcher to be more reflective and aware of potential biases, thereby increasing study credibility and confirmability.

Furthermore, although the nature of qualitative studies is subjective, we used several strategies to ensure the rigor of the study. They include methods of quality assurance that closely abide by Lincoln and Guba’s evaluative criteria [27], and the use of the framework method to enable a systemic method of analysis.

Finally, the platform did not deliver in instances where the link between the online Modules and the face-to-face discipline-specific teaching was clearly made. This is another limitation of the study, but also an avenue for future research, where the impact on research training by an integrated research platform can be explored.

Further Research

Based on the students’ feedback, the Modules can be updated, and a larger study can be launched to recruit a more multidisciplinary cohort. This would enable the exploration of the broader application of an e-learning platform, its advantage and usefulness in general, how its design can be optimised, and why other institutions should provide one. Resources should also be developed to support unit/course coordinators to use the Modules appropriately, as a resource to complement face-to-face teaching, with research to explore these updates in a more multidisciplinary cohort.

Additionally, this study identified the importance of addressing content saturation. The alignment between the studying findings and the CLT suggests that using strategies to reduce the extraneous CL is a promising way to reduce content saturation. However, further studies are required to examine the relationship between the CLT, and the way it can be applied to reduce the content saturation and CL of the students. Moreover, this study also identified the necessity to cater for the adult learning style, for instance through adopting a more task- and goal-orientated educational approach. Further studies should investigate how andragogy can be applied to improve the student research learning experience, for instance by incorporating motivational strategies that highlight the relevance of e-learning content to specific tasks and goals.

Finally, the present study sought to capture the qualitative views and experiences of students enrolled in a health-related discipline. Inherent in the design of qualitative research, it is not clear which views/experiences are more prevalent, nor which factors influence a particular view. We therefore recommend future studies that utilise a survey design, with a larger sample to address these unresolved questions.

Conclusion

The present study suggests that the students’ perception of the quality of the ‘Health and Medical Sciences Research Modules’ (a research training e-learning platform developed for health professional students) was very dependent on the way they were integrated with the course curriculum. Overall, when used as designed (as demonstrated by the pharmacy curriculum), to complement face-to-face teaching with clearly outlined learning objectives, the Modules were well received and utilised. But when used as a standalone additional resource without accompanying instruction (as demonstrated by the dentistry curriculum), they were not. Therefore, our study suggests that resources should be developed to support teaching staff/faculty to use the Modules more appropriately, as a resource to complement face-to-face teaching.