1 Introduction

Typically in tertiary teaching, reading lists (RLs) provide students with references to required readings and other materials for their course work [1, 2]. They have long been a part of tertiary education as a pedagogical tool and for tracking the use of copyrighted materials [1, 3, 4]. Traditionally RLs used to contain references to print-based materials such as books, chapters, journals, articles, proceedings, websites, blogs and magazines. The online version of RLs now contain additionally a significant amount of non-textual information such as videos, audio recordings and other resources. They are therefore often referred to as Resource Lists [5]. Educators have noted the opportunity for managing and tracking reading materials in digital libraries [6,7,8] and for integrating digital libraries in academic learning environments [9,10,11,12]. RLs are often integrated into an academic library’s offerings [13, 14], and academics are supported by academic liaison librarians in managing lists. Therefore, these lists represent an important channel of communication between academics, students and librarians, and they have a critical role to play in transforming students into autonomous learners [15]. Copyright Licensing New Zealand [16] requires all universities in New Zealand to provide software solutions to enable electronic reporting on copyrighted material. To meet these reporting obligations with CLNZ, all eight New Zealand universities adopted RLs systems in 2015.

Previous studies found that the RLs are under-used in their role as a pedagogical tool [1, 3, 17, 18]. All of these studies were user focused (academics or students) and not based on log analyses (see Table 1). Therefore, we investigate this further, beyond the user study, by a transaction log analysis. We previously explored pedagogical support features provided in RL systems [19]. We observed that the ‘notes’ feature was often used for pedagogical support for students. Academics left notes to students to guide their reading. Importantly, it gives an opportunity to academics to increase their own voice in the list [18]. As examples by explaining why a particular resource is valuable, what it covers, why it is included and what the student will gain from looking at it, how the list works, their expectations of the students in terms of engagement with resources, or quite simply, which chapter to read in an ebook [20]. In the research reported here, we are interested in understanding if and how academics used this ‘note’ feature to guide the students in their teaching. We believe this understanding will help to improve the utilization of the RLs as a pedagogical supportive tool.

Table 1 Academics’ experiences of use of the RLs: summary of the studies
Full size table

In this article, we explore academics’ experiences with the RLs and the use of RLs’ notes. We seek answers to three specific research questions:

RQ 1: What are academics’ experiences with creating RLs?

RQ 2: What are academics’ experiences of linking resources in RLs?

RQ 3: How were academics using notes in RLs?

The remainder of the article is organized as follows: Section 2 gives an overview of related work on academics perception and experience with RLs. We then explain our study method (Sect. 3), and present the results of our study and data analysis (Sects. 4 and 5). In the discussion in Sect. 6, we compare our study insights with those of related work. The final section presents conclusions and recommendations from our study.

2 Literature review

The use of RLs in tertiary teaching across individual universities [1, 17, 21] as well as within parts of a university [13, 18, 22,23,24] has been well reported. A number of studies include academics’ perceptions of the RLs [1, 22, 24]. Fewer studies have reported on academics’ experiences on use of RL features [2, 5, 24]. A common focus across the literature is the identification of significant hurdles for academics to usefully engage with RLs.

2.1 Academics’ experiences of RLs

The willingness of academics to engage in RLs creation seemed to vary across the different studies. Cross [5] at Nottingham Trent University highlighted that staff time constraints were a key barrier to the uptake of the RLs at their institute. Beasley [22] found that familiarity with the system, staff time constraints, and perceived usefulness of the system were also hindrances at the University of Auckland. Krol [13] discussed resistance and lack of interest by academics. Despite RLs being created for all courses with the help of library staff, the academics’ engagement with the RLs creation remained low due to a cited lack of time.

Most studies identified significant hurdles for academics to overcome in order to usefully engage with RLs. While Zhu [17] found that the academics valued the facility of the sharing of copyright material via the RLs, 40% were dissatisfied with the overall RLs’ functionality, stability and ease of use. Consultations with staff at the University of Manchester identified the need for improved functionality of the system as well as integration into the learning management software, better support for users, and marketing to their users of the potentials and capabilities of the system [21]. Neill & Musto [23] found that academics at the Dublin Business School wished for better integration of RLs with their learning management system and also identified time constraints as the main barrier for academics to use the RLs. Other factors highlighted as hindrances to RLs uptake were the discipline and lecturing experience of the academics. Taylor [18] agreed with her colleague Devine [25] in arguing that the RLs need to go beyond being a repository of teaching materials but should become a teaching tool in its own right. However, in what way RLs and a learning management system would integrate has not been addressed.

Academics also reported concerns that the RLs may not provide enough cost benefits for them and their students. Brewerton’s [1] study at Loughborough University found that some academics were not convinced that their efforts in maintaining the RLs were appropriate in comparison with the perceived benefit to the students. Cameron & Siddall [24] even noted concerns voiced by academics about RLs effectively “spoon-feeding” students and observed a lack of effective communication between librarians and academics.

2.2 Academics’ experiences of the RLs features

Adolphus [20] highlighted that the initial set-up of a reading list has become highly complex and takes a significant amount of time. A similar issue was identified by Cameron & Siddal [24]: all of their study participants agreed that setting up multiple lists was extremely time-consuming, taking “forever” to do, and each list involved a “tremendous amount of work,” that was “off-putting and daunting”. Importantly, they observed that the amount of set-up and maintenance requirements differed significantly depending on the individual academic’s discipline.

Thompson et al. [26] found in their study at the University of Wolverhampton that students preferred lists which are structured into key reading/titles for specific weeks, specific topics/subject areas and a single core text with background/supplementary readings. Brewerton [1] and Siddall [2] also found that students benefited from lists that are well-structured, rather than an alphabetical list of references. Similarly, Siddall & Rose [3] noted that well-structured and annotated lists that included course-relevant explanations and signposting were found to be helpful by students and helped build their study confidence.

Secker [27] found that lists which are enriched with commentary, notes and explanations are pedagogically valuable and constitute an important learning resource. Adolphus [20] observed that the note feature could be used to include a variety of texts into the reading lists that address different student abilities. He further recommended that academics use the note feature to explain why a particular resource is valuable, what it covers, why it is included and what the student will gain from looking at it. Taylor [18] highlighted the use of the note feature to personalize reading lists, to explain how the list works, their expectations of the students in terms of engagement with resources, the importance of texts, or quite simply, which chapter to read in an ebook.

The literature also touches on the resource-linking features found in RLs. The ‘Bookmarks’ feature allows academics to capture the available information from online resources and presents it in an easy to edit format, ready to save and add to the lists. Cross [5] identified in his study at the Nottingham Trent University, for a large amount of online material not yet bookmark compatible, only basic information (URL, and page Title tag data) is extracted. He suggests that the bookmarklet feature needs a significant amount of sustained intervention to manually add the missing metadata and to create sustainable authentication-aware links. Bookmarking full-text documents was also seen as an issue by McGuinn et al. [28] in their study at the University of Huddersfield, and they suggest that this feature needs to be further developed. Zhu [17] also highlighted that academics dissatisfaction with the features like Bookmarks largely affects their intention to use the RLs at the Auckland University of Technology.

One resource linking feature that prompted positive feedback from many academics was the ‘content digitization’ service, which allows academics to request copyright-cleared articles and chapters be made available online via the RLs [18]. This has greatly expanded the range of material available to students electronically (including articles outside the library subscriptions). From an academic and library point of view, it is a triumph for both copyright law and online library subscription usage statistics.

Some studies suggest new features for the RLs. For example, Zhu [17] reported that academics want to have a feature in RLs that allows students to ‘submit resources’. McGuinn et al. [28] suggest a more user-friendly interface to the RLs (mobile-friendly), and features such as download, e-mail and personalization.

2.3 Identified research gap

We observe that many of the available publications are reports reflecting on an institute’s journey. Table 1 provides an overview of the discussed studies on academics’ perceptions of use of the RLs. Out of the ten studies, only two studies included academics’ perception of a specific feature. Of these two Siddall [2] focused on the book ordering feature, whereas Cameron & Siddall [24] focused on labels feature. In addition, only two studies used a detailed log analysis. Of these two, Beasley [22] focused predominantly on a single semester. Krol [13] covered a four-year period (2016–2019) but limited the study to a single faculty (Computing and Engineering). Both of these studies explored the uptake of the RLs at particular universities. None of the studies focused on analyzing the log data of the use of a specific feature.

3 Method

This section describes the study context, method, data collection, data preparation and pre-processing for analysis. Our study employs a mixed-methods approach [29] including interviews with academics and a transaction log analysis.

3.1 Institutional context

The University of Waikato had eight faculties at the time of the study: Arts and Social Sciences (FASS), Education (FEDU), Science and Engineering (FSEN), Waikato Management School (WMS), Maori and Indigenous Studies (FMIS), Computing and Mathematical Sciences (FCMS), Health, Sport and Human Performance (FHSHP) and Law (FLAW) (see Appendix 1 for particularities of the each of faculty). RLs are typically created for each course instance, being assigned to different semesters and years, such as Summer Schools S, and G, Semesters (Trimester) A and B, whole year D courses, and Semester C (all other periods). Most students attend Semesters A and B, with fewer in Summer Schools S and G. D and C are rarely used, mostly for postgraduate studies. Courses are taught under different levels. There are six such levels available. Level 0 for foundation or bridging students, Level 1/100, Level 2/200, Level 3/300, Level 4/400, and Level 5/500/5 + for postgraduate courses.

3.2 Study method

Our study consists of two phases. The first phase of our mixed-methods study consisted of interviews with academics. The second phase of the study used a transaction log analysis of the “lecturer notes” (notes given by the lecturer to the students for each linked item) of Waikato Reading Lists (WRL) for the year 2020.

The purpose of the “notes” log analysis was to gain an in-depth understanding of how academics engage with a pedagogical support feature of the WRL. We here describe the data collection process for the two phases of our study.

3.2.1 Phase 1: interviews

The study population for the interviews consisted of the UOW academics who were involved with WRL as list creators in at least one case. The survey invitation request was emailed to them in October 2021. We received 19 positive responses from the academics by representing 6 faculties. A total of 19 interview sessions were conducted via Zoom within the stipulated time period. The interview questions comprised closed and open-ended questions and were constructed under three main themes (see Appendix 2):

  • Experience of the WRL set-up and linking materials

  • Experience of the use of notes feature

  • Perceptions and suggestions to improve WRL

Due to the inclusion of both open-ended and closed-ended questions in the interview questionnaire, a series of manual procedures were implemented to preprocess the responses for subsequent analysis. Responses obtained from the closed-ended questions were initially categorized based on criteria encompassing ratings, prevalent comments, and affiliation with respective faculties of the academics. Then, we analyzed and presented them in a structured way using various tools like charts, graphs, and tables. In the case of open-ended question responses, a thematic analysis approach grounded in word/theme occurrences was employed for evaluation. A theme, in this context, pertains to a discernible pattern within the data, functioning to describe, organize, and potentially interpret various facets of the phenomenon (Boyatzis, 1998).

3.2.2 Phase 2: log analysis

The raw data from the WRL transaction logs were automatically processed and collated into tables of summary data. In this study, a selection of these data received from the UOW library covering 1 January 2016 to 31 December 2020 is discussed. The collected data represent all RLs items (book, chapter, article, journal or other) which were linked by, or for, all the faculties concerning the teaching and assessment periods. In addition, data were collected for the following:

  • Courses for which a list was created

  • Lists for which an item was linked

  • Bibliographic details of the item

  • The year and the semester for which the item was linked

  • The creator (academic staff or academic liaison librarian)

  • Mode of the item (online or physical)

  • Type of the item (book, chapter, article, journal or other)

  • Specific notes indicated with the item (to students or the library)

Any lists that were deleted at some point from the WRL do not appear in the logs. This is a rare occurrence. The lists are deleted, usually, only if they are duplicates and the regular practice is for all lists to be archived annually. Hence, the statistics for RLs can be assumed to be complete. Table 2 describes the preparation of the transaction log data for analysis.

Table 2 Preprocessing steps for transaction log data analysis
Full size table

4 Results and analysis of interviews

The interviews investigated the experiences and perceptions of academics with the use of WRL under three sections; reading list set-up and linking process, use of notes feature, and perception and suggestions to improve the WRL. We interviewed 19 participants, representing 6 of 8 faculties. We did not receive an equal number of participants from each faculty. The faculties were represented as follows: FASS (7), WMS (4), FEDU (3), FLAW (2), FCMS (1), FSEN (1), FMIS (0) and FHSHP (0).

4.1 Academics’ experiences with the WRL set-up and linking resources

We here discuss the academics’ experiences of creating reading lists and linking digital resources in their lists.

4.1.1 Set-up of the RLs

First, we asked academics ‘how easy was it to create a reading list?’, and received mixed feedback (see Fig. 1). Three participants did not answer the question as they were not directly engaged with reading list creation, they sought help from the liaison librarians for the initial set-up of their reading lists. Six of 16 respondents (with 2 strongly) said it was easy to set up, 4 were neutral, and 6 experienced it as difficult.

Fig. 1
figure 1

Ease of creating a reading list, feedback sorted by faculty (n = 16)

Full size image

We did not identify any patterns across the faculties. All respondents provided further reasons for their ratings (respondents were permitted to give more than one reason). Five of 16 respondents commented that they found ‘it was self-explanatory, easy to navigate and straightforward to use’. Thirteen of 16 respondents gave negative feedback such as it's complicated and easily forget how to do it (6) not intuitive (4), not user-friendly (3) and time-consuming (3).

Next, we questioned, ‘how easy is it to remember the process of creating reading lists?’. Eight of 16 respondents gave negative affirmation to the question, 5 gave a positive response and 3 remained neutral (see Fig. 2). From the detailed feedback of 16 respondents (they were permitted to give more than one reason), we see that 10 felt ‘if WRL is not used all the time, it's not that easy to keep remembering the process’. Additional critical comments were the complicated processes (3), not intuitive (1) and time-consuming (1). Positive feedback included that ‘if you're using it all the time, it's easy’ (3).

Fig. 2
figure 2

How easy is it to remember the process of creating reading lists?, feedback sorted by faculty (n = 16)

Full size image

4.1.2 Linking resources in RLs

As WRL provides a variety of options to add reading materials to the lists, we asked academics how they add materials to the reading lists (see Fig. 3). More than half of the respondents (10 of 19) indicated that they commonly used the bookmark browser extension feature to include teaching materials in their lists. Six of 19 respondents used the add resource feature in the WRL, whereas 8 respondents sent their resources to the library to add into their lists on behalf of them. 2 of 19 respondents used the digitization request feature in the WRL. We found that the bookmarking feature were common across four faculties except FLAW and FSEN. Respondents from the FLAW and FSEN expressed that they send their resources to the library to add to their lists.

Fig. 3
figure 3

How academics add material to the reading lists, feedback sorted by faculty (n = 19, more than one option allowed)

Full size image

The main reason was that this option is easier for them as the library team is doing it better. For example, one respondent mentioned that “…I just gave them the citation…and of course they were all accessible in the library, so they would do it on their own… they've been really, really great…’’. However, only respondents from the FCMS said that she used all the options to add materials to the lists. According to her, options such as add resource and bookmark extension are intuitive and fairly easy to use. Further she mentioned that the “digitization feature is easy to use if you know exactly what part of the book you want to digitize, else it's difficult”.

Next, with respect to academics' responses on how they add materials (as in Fig. 3), we requested them to express their views of the interfaces that they engaged with when adding materials. We received the following further responses (see Table 3).

Table 3 Academics responses on the interfaces used for adding materials
Full size table

When asked about the ease of browsing the contents in the interface when viewing the linked materials, many respondents (15 of 18) were positive (see Fig. 4) and 3 remained neutral. None of the academics gave negative responses. All participants provided further feedback on their rating. Participants whose rating was positive said browsing the contents in the interface is fairly easy and they haven't found any difficulties. However, out of them, 4 participants who rated it as easy, experienced some issues. Two of them mentioned that sometimes different types of interface appear, sometimes directly going to the particular chapter or the content and they have to find it and fix it for students. One participant said it's time consuming and the other participant identified students facing some issues depending on what browser they were using. Importantly, from the detailed feedback of 3 participants who rated as neutral, there are intricacies in browsing the contents in the linked materials (particularly eBooks) due to the heterogeneity of the data structure in provider websites.

Fig. 4
figure 4

How easy or difficult to browse the contents in the interface, feedback sorted by faculty (n = 18)

Full size image

In summary, we identify a need to improve the system’s usability, particularly with regard to the reading lists set-up and linking process.

4.2 Academics’ experiences with the use of notes feature

Notes is an important pedagogical feature available in the RL systems which allows academics to guide the students’ reading. This feature undoubtedly helps to make RL systems an important learning resource by adding pedagogical value to the lists. Here, we investigated the academics’ experiences and perceptions of the use of notes feature in the WRL.

First, we enquired about their awareness. We asked academics ‘were you aware of the feature that supports leaving notes for students in reference to a linked item?’, and received responses that 11 of 19 were aware about it (see Fig. 5). Out of 11 participants who were aware, only 9 of them actually engaged with this feature. Because 2 participants had not used this feature though they were aware that the feature was available. The reasons given by them include: ‘I haven't used it at the level of the individual resources..I use notes in the topic section’, ‘I don't know how it would work. I'm not sure if I add a note my student will receive’. Overall, it seems that the majority of participants (10 of 19) were not engaged with this feature due to the lack of awareness of the availability and the use of the feature.

Fig. 5
figure 5

Academics awareness of the notes feature, feedback sorted by faculty (n = 19)

Full size image

For the 9 participants who had experiences of the use of notes feature, we received the following further responses:

  • Ease of interaction: when asked about the ease of use of the notes feature, all academic respondents were positive (with 6 strongly).

  • How participants used the notes feature: Responses given included: to direct students to a particular chapter or the page (s) to read (7), to explain what covering in class (4), to explain the students a bit about the particular reading in terms of why it is important and how the reading applies (3), to put a few little reminders and messages (2), to explain about the assignments (2), classified list items (1).

  • Type of notes they used and why they used in that way: Responses given grouped as follows;

  1. (i)

    Signposting (5)–to direct students to read a particular part of a chapter or book or whatever it is.

  2. (ii)

    Guidance and for rationale (3)–to provide students with the reason for the reading, how the student can use the reading, how it will help their learning, and how it applies to what they're learning in class.

  3. (iii)

    Explanatory (3)–to explain about the particular reading.

  4. (iv)

    Miscellaneous (3)–to encourage, send messages and reminders to students.

Participants overall experience of using notes: Responses given included: it's nice, helpful and supports the teaching (5), not sure that the given notes are helpful to students or not (4), gives an opportunity to annotate the resource list (1).

4.3 Academics’ perceptions and suggestions to improve the WRL

In this section, we examine the varying perceptions, expectations and suggestions of the academics in relation to the WRL.

Sixteen of 19 respondents gave positive affirmation to the question ‘Do you think a reading list system is well-suited to your subject area/ discipline?’, only 1 gave a negative response and 2 remained neutral (see Fig. 6).

Fig. 6
figure 6

RLs system is well-suited for my discipline, feedback sorted by faculty (n = 19)

Full size image

When we asked them to provide more details about their rating on the above question, we saw that irrespective of their positive feelings many shared some interesting views and suggestions as in Table 4.

Table 4 RLs system is well-suited for my subject area/ discipline, detailed feedback
Full size table

Many academics appreciate the way that the WRL supports managing their teaching resources (see Table 4). However, they highlighted some important points with regard to the reading lists such as reading lists should be tailored into each week of learning (3), quite difficult to use as different ways of doing things (2), better UI interaction for linked/scanned items (2), training on how to use it and promote it among students (2), doubt about students usage (2) properly integrate with Moodle (1).

We then further examined the academics suggestions to improve the WRL beyond the support of teaching. We grouped their responses as follows:

  • Simplified process to add resources (4): Responses given included: ‘much more simple and straightforward system for identifying and uploading the readings’, ‘having a kind of space pool for us to save those resources we might need in the future’, ‘it requires us first to bookmark and then to go in and edit the link better to have simpler way of achieving that’, ‘if I could have the following or past years reading list available then it would easy to update’.

  • Make it more easy and intuitive (4): Responses given included: ‘it needs to be a lot more intuitive’, ‘update the system’, ‘make the setup template clearer to use’, ‘simplify, make it easy to use and make it intuitive’.

  • Improved UI experience (4): ‘update the interfaces, ‘enhance familiarity’, ‘i don't find that browser extension interface quite as useful as it might be’, ‘preview of the first page of the document or an expanded view’.

  • Proper structuring and formatting (3): Responses given included: ‘sometimes it's difficult to see where your sections are so having those sections in a more clearly differentiated way is better’,‘list the resources using a APA referencing style because it would be useful to model APA referencing style to the students’, ‘it would be good also to have like an option to collapse or to hide a part of a section rather than have all of them expanded’.

  • Automated/self guidance (3): ‘frequently asked questions kind of pieces of information’, ‘quick five minute video reminding us of how to use’, ‘clearer prompts, asking what kind of source do you want to put in a book or journal or website, in which week?’.

  • Training and support (2): ‘some scope for a bit of extra education there’, ‘it would be good to have a contact person’.

  • Streamline the digitization process (2): Responses given included: ‘I have to request the digitization for an item every year though if it's been digitized previously’, ‘I cannot look at the material that is scanned in full page view’.

Finally, respondents volunteered some views on the missing features within the WRL and some of them repeated points they made in earlier comments. We grouped their responses into seven main categories as follows:

  • More simplistic process for setting up a list (6): Here the academics request a more clearer, simplified and straightforward template for reading lists set-up.

  • Attractive and informative GUI (4): Some of the academics expressed that they don't feel like they engaged with the reading materials when reading on screen. They wanted more attractive interfaces with helpful browsing information. They prefer lists which can break down into subsections and icons, the thumbnails for the readings.

  • Synchronization into Moodle (4): Since academics engaged with both Moodle and WRL, they requested better integration among these two systems.

  • Formatting, editing and viewing list items (4): Here they wanted to format the texts they entered with respect to the linked items. For example, in a paragraph, changing the font, font size, style and the referencing style. Further, they indicate the requirement of a better view for scanned materials.

  • Export bookmarks (1): One academic requests the option of exporting all bookmarks at once rather than doing it one by one.

  • Usage dashboard/widgets (1): Simple way for knowing which resources are being used and how much the students are engaged with it.

In summary, respondents’ perception and the experience on the use of the WRL reveal that the majority acknowledged the usefulness of the WRL for their teaching.

However, their detailed responses strongly suggested that improvements are necessary, including a streamlined workflow for the WRL functionalities, attractive interfaces with clearer prompts and better synchronization into other teaching support systems.

5 Results and analysis of the lecturers’ notes in WRL

This section presents the results of our log analysis of notes. Here, we analyzed specific notes given by the academics to the students for each linked item in their RLs.

Figure 7 shows that the total number of items in the WRL increased gradually each year. However, the lecturer notes to the linked items have not increased proportionality to the total number of items. We observe that over time for books/chapters, academics includes proportionally more notes to the students than the other two categories.

Fig. 7
figure 7

Lecturer notes to the students sorted by the type of items linked (2016–2020). Note: Dark color indicates items with notes and light color indicates items without notes

Full size image

Though book/chapters contributed largely, when we explored the lecturer notes to the book/book chapters as a proportion to the all linked book/chapters items, it had fallen from 41% in 2016 (2227 items with notes out of 5452 items) to 34% by 2020 (3185 items with notes out of 9322 items). Taken together with the other two categories, articles/journals also fell from 12% in 2016 (573 items with notes out of 4947 items) to 8% by 2020 (842 items with notes out of 9979 items), while other items with notes increased from 22% (494 items with notes out of 2295 items) to 25% (1028 items with notes out of 4095 items).

Further, in this section, we directly examined the lecturer notes (for all categories) using content analysis methods. As stated by Holsti [30], content analysis is a research tool used to determine the presence of certain words, themes, or concepts within some given qualitative data (i.e. lecturer notes). Using content analysis, we quantified and analyzed the presence of certain texts (within lecturer notes), which act as pedagogical supportive words, themes, or concepts. Accordingly, Table 5 presents the categorization of lecturer notes with sample texts.

Table 5 Content analysis of the lecturer notes for books/chapters, articles/journals and other items
Full size table

5.1 Notes analysis of books/chapters

Here we present the analysis results for books/chapters. Figure 8 depicts the total number of linked book/chapters in 2020 and the inclusion of lecturer notes. Out of 9322 linked items, only 3185 (34%) items had lecturer notes.

Fig. 8
figure 8

Lecturers’ notes to the students with regard to books/chapters in 2020, group by faculty

Full size image

We observed field-based differences in the inclusion of the lecturer notes (see Fig. 8). One group of faculties (FASS; 1128 [37%], FEDU; 509 [22%], WMS; 837 [49%] and FSEN; 498 [69%]) contributed far more notes than the other group (FCMS; 44 [37%], FHSHP; 63 [25%], FLAW; 80 [27%] and FMIS; 16 [10%]). However, we note that this difference was mainly due to the variations in linked items in RLs by each faculty. Most prominent here is FSEN with 69% notes for the linked items. In contrast, FMIS is only with 10% notes for the linked items.

Then, we analyzed the total number of lecturer notes included across all eight faculties for each academic semester in 2020 (see Fig. 9). While the overall number shows high in semesters A and B, for other semesters the numbers are low. This is because most students attend semesters A and B, with fewer in summer schools. Other semesters are rarely used, mostly for postgraduate studies. Similar to the pattern we identified in Fig. 8, here for semesters, inclusion of lecturer notes were higher in the four faculties FEDU, FASS, FSEN, WMS (greater than 100 notes per semester, e.g., semester A, B 2020), compared to the other four faculties FMIS, FCMS, FHSHP and FLAW (less than 40 per semester).

Fig. 9
figure 9

Lecturers’ notes to the students with regard to books/chapters in 2020, sorted by faculty, group by semester

Full size image

As shown in Fig. 10, 35% (1102) of the lecturer notes did not provide any pedagogical supportive guidance to the students (i.e. ‘no signpost’ and ‘descriptive but no pedagogical support’). Only 5% (147) of the total lecturer notes contains pedagogically beneficial guidance to the students (i.e. descriptive guidance). Previously, in Fig. 8, we noted that FSEN included higher lecturer notes (498 notes) but according to content analysis, all those notes did not bring any pedagogical benefits to the students (only 1 note provides descriptive guidance but that also without pedagogical support). Similarly, irrespective of the higher numbers in WMS (837 notes) and FASS (1128 notes), the content analysis revealed that only 32 in WMS and 63 in FASS contain pedagogically beneficial guidance to the students (i.e. descriptive guidance, see Fig. 10).

Fig. 10
figure 10

Lecturer notes to the students with regard to books/chapters in 2020, sorted by faculty, group by note type

Full size image

Four possible interpretations may be drawn from these observations. First, academics lack awareness of the use of this feature. Second, academics may not believe that this was a good option for them to guide students. Third, lack of user-friendliness of the Note feature. Fourth, discipline-specific requirements (i.e. engineering, science, management, etc.)

Next, we examined the lecturer notes against the paper levels (see Fig. 11). Paper levels refer to the different levels at which courses are taught and are usually associated with years of study (see Table 2). First-year (100 level or level 1) courses are more general while fourth year (400 level or level 4) courses are more advanced. 500 level or level 5 (+) courses are graduate and postgraduate level courses. Among all levels, academics used to include more notes to the level 3 courses (see Fig. 11).

Fig. 11
figure 11

Lecturer notes to the students with regard to books/chapters in 2020, sorted by faculty, group by paper level

Full size image

This may be since the level 3 courses rely less on formal teaching and assessment and require greater student participation both in timetabled classes and through seminars and workshops. On the other hand, level 4 courses show the lowest. This is because most courses are offered and students attend levels 1, 2 and 3, with fewer in level 4.

5.2 Notes analysis of articles/journals

This section presents the analysis results for articles/journals. Out of 9979 linked items, only 842 (8%) items had lecturer notes in 2020 (see Fig. 12).

Fig. 12
figure 12

Lecturers’ notes to the students with regard to articles/journals in 2020, group by faculty

Full size image

Overall, among all faculties, lecturer notes numbers are low. In FSEN and FCMS, there are a few linked items and none of them include any notes. Though FASS, FEDU, WMS and FLAW showed a higher linked items compared to the other faculties, as a percentage, items with lecturer notes are low (FASS; 14%, FEDU; 11%, FLAW; 6% and WMS; 4%). From this log data it is difficult to identify the exact reasons for having a low number of lecturer notes.

However, one possible reason is that as the contents of the linked articles/journals are small and specific by nature (compared to books) and therefore, academics might not have seen any requirement for providing signposting or descriptive notes about the item.

When we analyzed the total number of lecturer notes for each academic semester, the overall numbers were high in semesters A and B while for other semesters the numbers were low (see Fig. 13). We saw a similar pattern with books/chapters as well (see Fig. 9). The only difference is that semester B showed higher lecturer notes for books/chapters whereas semester A showed higher lecturer notes for articles/journals. In semester B, the higher number of lecturer notes for books/chapters suggests that the course content may have relied more heavily on textbooks or specific book chapters. In contrast, semester A had a higher number of notes for articles/journals, indicating that the course materials may have included more academic papers or journal articles.

Fig. 13
figure 13

Lecturers’ notes to the students with regard to articles/journals in 2020, sorted by faculty, group by semester

Full size image

However, these higher numbers in the first two semesters are not surprising as many students attend for these semesters (many courses offered), with fewer in other semesters (mostly for postgraduate studies). We observe that at semester A, FEDU contributed far more lecturer notes than the other faculties (208; 46%). However, by semester B had fallen to about half the proportion (75; 27%). The FASS continued with the same proportion (~ 140 per semester; 31%) whereas for FMIS, FCMS, FHSHP and FLAW it was less than 50 per semester.

As depicted in Fig. 14, we see many lecturer notes for articles/journals contained signposts (668; 79%). However, only 4% (35) of the lecturer notes provide pedagogical supportive guidance to the students (i.e. descriptive guidance). 16% (139) did not provide any pedagogical supportive guidance (i.e. no signpost). In Fig. 12, we identified that the FEDU and the FASS included a higher number of lecturer notes (363 and 293 notes) but content analysis showed us that only 7 in FEDU and 19 in FASS contains pedagogically beneficial guidance (i.e. descriptive guidance). Possible interpretation we may draw from this observation is that since the contents of the articles/journals are limited by nature, academics may have thought that a detailed explanation was not needed for this.

Fig. 14
figure 14

Lecturer notes to the students with regard to articles/journals in 2020, sorted by faculty, group by note type

Full size image

When examining the lecturer notes against the paper levels (see Fig. 15), we see academics used to include more notes to the level 5 courses (i.e. graduate courses). For books/chapters, this was for level 3 courses (see Fig. 11). This may be as the level 5 courses are more reliant on research work and require greater access to articles/journals. Similar to the books/chapters, low figures in level 4 courses are not surprising. Because most courses are offered and students attend for the first three levels, with fewer in level 4.

Fig. 15
figure 15

Lecturer notes to the students with regard to articles/journals in 2020, sorted by faculty, group by paper level

Full size image

5.3 Notes analysis of other items

This section presents the analysis results for other items. Other items contained linked resources such as web URLs, audios and videos. Out of a total of 4095 linked items, one-quarter of items (1028) had lecturer notes in 2020 (see Fig. 16).

Fig. 16
figure 16

Lecturers’ notes to the students with regard to other items in 2020, group by faculty

Full size image

Similar to other two linked item categories (books and articles), for other items also we see one group of faculties (FASS; 343 [39%], FEDU; 237 [28%], WMS; 209 [20%] and FLAW; 92 [14%]) contributed far more notes than the other group (FCMS; 72 [75%], FHSHP; 29 [32%], FSEN; 3 [5%] and FMIS; 13 [19%]). This difference was mainly due to the variations in total linked items by each faculty (higher the linked numbers, higher the lecturer notes). However, as a percentage to the total linked items, FCMS and FHSHP showed higher figures. Most prominent here is FCMS with 75% notes for the linked items, which we have not seen with FCMS in the other two categories (see Figs. 8 and 12). In contrast, FSEN is only with 5% notes for the linked items. We observed a poor figure with FSEN in articles/journals as well (0%). However, FSEN showed their highest figures (69%) in the books/chapters category (see Fig. 8).

In alignment with the previous two categories (see Figs. 9 and 13), here also we observe a similar pattern in lecturer notes for items in each academic semester (see Fig. 17). Same reason echoes here as well. The FASS contributed more lecturer notes than the other faculties in Semester A (117; 39%). However, by semester B had risen to about double the numbers (219; 41%) while FEDU and WMS continued with the same proportion (~ 100 per semester).

Fig. 17
figure 17

Lecturers’ notes to the students with regard to other items in 2020, sorted by faculty, group by semester

Full size image

Similar to the articles/journals, for other items also we see many lecturer notes which included the signposts guidance (616; 60%). 4% (38) of the lecturer notes provide pedagogical supportive guidance to the students (i.e. descriptive guidance). 36% (374) of the notes have not contained any pedagogical supportive guidance, i.e. no signpost (see Fig. 18). Though the FASS, FEDU and WMS contributed for many lecturer notes (343, 237, 209) only very few contain pedagogically beneficial guidance (i.e. descriptive guidance). In previous discussions also we saw a similar pattern with regard to the other two categories (see Figs. 10 and 14).

Fig. 18
figure 18

Lecturers’ notes to the students with regard to other items in 2020, sorted by faculty, group by note type

Full size image

Finally, we examined the lecturer notes for other items against the paper levels (see Fig. 19). Even though academics had included more notes to the level 5 courses (i.e. graduate courses), overall we cannot see any significant variation between paper levels except Level 4 and Level 7.

Fig. 19
figure 19

Lecturers’ notes to the students with regard to other items in 2020, sorted by faculty, group by paper level

Full size image

In summary, as discussed above, the difference in the types of materials used in each semester could be a contributing factor to the variation in the number of notes. It is possible that when students are studying materials from books or chapters, they may find the content more comprehensive and structured, leading to many instances where additional notes are needed by their lecturer. On the other hand, academic articles or journal papers may require less guidance due to their specific focus, deeper analysis, or detailed information. Therefore, it is essential to consider the context of the papers and the nature of the materials provided to draw accurate conclusions about the reasons behind the discrepancy in the number of notes between semesters and paper levels. Additional factors, such as the teaching approach, assignment type, assessment method, and student preferences, may also influence the academics note-giving patterns observed. Further investigation and analysis of the paper content and academics feedback could provide more insights into the specific reasons for the observed differences.

6 Discussion

We discuss insights from our studies reported in this article, particularly in relation to related literature.

6.1 Lack of usability of RLs features

We found that the 81% respondents negatively commented on the reading lists set-up feature (see Fig. 1). The main reasons given by them were the complexity and lack of intuitiveness of the process. We therefore identified a need to improve the system’s usability in the reading lists set-up process. Similar observations were reported by Cameron & Siddall [24], Cross [5], Zhu [17] and Krol [13] in their studies. In addition, our participants agree with those in earlier studies in observing that time constraints were a limiting factor in setting up a list [5, 13, 17, 20, 22, 24]. Sixty-two percentage of the participants noted that it is not that easy to keep remembering the process because they do not use the RLs system regularly (see Fig. 2). Academics’ usually only engage with the list creation once or twice a year, sometimes after a couple of years, and the process of setting up a list may be easily forgotten.

Cameron & Siddall [24] observed that in their study the work required to set-up and maintain a reading list differed significantly depending on the academic discipline. We did not observe such a variation, which may be due to the small number of participants from each discipline. Instead, 84% of our respondents confirmed that the RLs system is well-suited to their discipline (see Fig. 6).

Our participants wished for more flexibility in structuring and formatting their lists and list items. They preferred lists which can break down into weeks, topics and subsections. They wanted to format the texts they entered with respect to the linked items (e.g. changing the font, font size, style and the referencing style). We did not see this aspect discussed in any other studies. However, some studies [1,2,3, 26] found that the students preferred well-structured and annotated lists.

We found that respondents used different means for linking resources: bookmarking, adding resources feature, or requesting support from the library for digitization and adding of information (see Fig. 3). We noted that several respondents had not been aware of the bookmarking feature. When analyzing the difference between faculties, we found that the bookmarking and add resource feature were common across FASS, FSDU, WMS and FCMS. One of the challenges identified by academics was missing metadata when bookmarking eBooks. FSEN and FLAW showed clear preference for sending their resources to the library to add on behalf. Krol [13] and Kumara et al. [31] reported on the type of materials linked in the RLs but did not discuss the features used for linking resources. Further exploration of different feature preferences between faculties is recommended. We noted participants’ concerns with clarity and the ease of use in both bookmarking and add resources for resource linking.

Based on the above discussion, we note issues in initial setting up and linking resources as potential barriers for academics’ uptake of the RLs. Therefore, we identify a need to improve the system’s usability of those features.

In addition, we had previously identified inconsistencies when linking to part of eBooks via the online systems offered by publishers [32]. Therefore, a standardization at the publisher or RLs system level would greatly ease the burden to the creators of RLs.

6.2 Underused notes feature in RLs

Adolphus [20], Taylor [18] and Secker [27] suggest that the note feature helps to make RLs systems an important learning resource by adding pedagogical value to the lists. However, none of the studies explored the log data of lecturer notes and did not examine the academics’ experiences of use of this feature. Our study is the first to explore these contexts. We observed that while the total number of linked items (books/chapters, articles/journals, etc.) in the WRL increased gradually each year, the lecturer notes attached to linked items have not increased proportionality. We found that the lecturer notes (any type of a note, see Table 5 for note types) for linked book/chapter items had fallen from 41% in 2016 to 34% by 2020 and articles/journals fell from 12 to 8% by 2020 (see Fig. 7). There might be several reasons for this. One possible reason might be that though over time reading lists numbers increased, academics did not engage with many features offered by the system. The reading lists numbers increased over time due to the increment of the offered papers and the continued support given by the university library [32]. As reading lists numbers increased, the number of linked items also increased. However, as we mentioned earlier, academics have not used many pedagogical supportive features offered by the system. The note is one of such features. In our interviews with academics we identified that more than half of the participants (10 of 19, 53%) have not engaged with the note feature due to the lack of awareness. The academics who did use this feature also have doubts about whether the given notes are helpful to students or not. Therefore, though the linked items increased over time lecturer notes for the linked items have not increased.

Our content analysis of the lecturer notes (see Table 5 for categorizations of lecturer notes) found that most lecturers did not provide any pedagogical supportive guidance to the students (see Figs. 10, 14 and 18). For books/chapters, only 5% of the total lecturer notes contains pedagogically beneficial guidance to the students whereas articles/journals and other items were included in the close-to-equal measure (4%). When looking at the reasons for this, in our interviews with academics, we found that they used notes in different ways. Mainly they used it to direct students to a particular chapter or the page(s) to read. Other purposes include: to explain what is covered in class and to explain the students a bit about the particular reading in terms of why it is important and how the reading applies. We identified that many of them did not use this feature to provide any descriptive guidance to students. Hence, the opportunity of utilizing this feature as a pedagogical supportive tool has so far been missed.

Finally, from the findings of the log data and the interviews, we note that this feature has been under-utilized to bridge the gap between academics and the students and helps to clarify expectations. Therefore, it does not currently offer comprehensive pedagogical support. Apart from the pedagogical significance, we believed that the streamlining and better utilization of the note features would help students to manage their time, and their money as well.

7 Conclusion

This article provides insights into the academics’ perceptions and experiences with the University's RLs system and the use of notes feature. From our interview responses and log analysis, we have drawn three conclusive points that shed light on the challenges and opportunities for improving the system's usability and pedagogical impact:

First, academics reported difficulties during the initial setting up of RLs, indicating a lack of user-friendliness in the set-up process. This finding emphasizes the importance of streamlining and simplifying the list creation process to enhance the overall user experience. By designing a more intuitive and straightforward workflow, we could empower academics to create reading lists more efficiently, allowing them to focus on the core aspects of their teaching.

Second, many academics heavily depend on library support for linking resources, revealing a lack of confidence and awareness in completing this task themselves. To address this issue, we recognize the need to improve the system's usability not only in list creation but also in resource linking. By providing clear guidance and support, we can instill confidence in academics to independently manage their resources within the RLs system, reducing their reliance on external assistance.

Third, our interviews and log analysis studies revealed that the notes feature has been under-utilized by academics. The lack of awareness of its availability and potential applications stands as a significant barrier to its adoption. Recognizing the untapped potential of the notes feature, we advocate for a more prominent and integrated presentation of this functionality within the system. By enhancing its visibility and highlighting its pedagogical benefits, we envision greater utilization and recognition of the notes feature as a valuable tool in supporting teaching activities.

This research provides actionable insights for enhancing the University's RLs system, focusing on user-friendliness and feature utilization. By addressing the identified challenges and incorporating valuable feedback from academics, we strive to create a more efficient and supportive platform for both educators and students. As we continue to refine and implement the improvements, we anticipate a more positive and transformative impact on teaching and learning activities at the University.

We are currently actively working on designing a more user-friendly workflow for RLs systems. Our goal is to develop an interface that is not only more accessible and intuitive but also empowers academics to harness the full potential of the system independently. We believe that this redesign will foster a more seamless and engaging experience, leading to increased adoption and utilization of the RLs system across the academic community.