1 Introduction

Reading lists (RLs) have long been part of academia [1, 2] and may provide pedagogical ‘scaffolding’ in which academics offer support to students through signposting and rich annotation on required readings [3]. They thus have a critical role to play in transforming students into autonomous learners [4]. Educators have noted the opportunity for managing and tracking reading materials in digital libraries [5, 6, 24] and for integrating digital libraries in academic learning environments [7, 8, 23, 25]. In the last decade, Online Reading List Systems are often integrated into an academic library’s offerings [9, 10], and academics are supported by academic liaison librarians in creating RLs. Copyright Licensing New Zealand [11] requires all universities in New Zealand to provide software solutions to enable electronic reporting on copyrighted material. To meet these reporting obligations with CLNZ, all eight New Zealand universities adopted RL systems in 2015.

Previous work has identified the need for a more detailed examination of RL systems, particularly in relation to enhancing academic engagement [12,13,14]. In this article, we explore RLs creation across the University of Waikato (UOW) and seek answers to three specific research questions:

RQ 1::

How many RLs were created each year within each faculty?

RQ 2::

What are academics and librarians’ experiences with RLs?

RQ 3::

What materials are included in RLs?

The remainder of the article is organized as follows: The following section gives an overview of related work on RL systems. We then explain our study method, the results of our study and data analysis. In the discussion, we compare our study insights with those of related work. The final section presents insights and recommendations from our study.

2 Literature review

The use of RLs in tertiary teaching across individual universities [1, 14, 15] as well as within parts of a university [9, 12, 16,17,18] has been well reported. A number of studies include a longitudinal approach. Krol [9] reported that the University of West London saw an increase in RLs from 4% in 2013/2014 to 100% of courses by 2018/2019. Few other studies have reported 100% saturation of RLs uptake by academics across a teaching division let alone an entire university. Taylor [18] found that at the University of Worcester 95% of modules had RLs in 2018/2019, after first introducing RLs in 2014/2015. More typically, Beasley’s [16] study at the University of Auckland found that RLs creation varied among faculties due to their disciplinary needs. The willingness of academics to engage in RLs creation seemed to vary across the different studies. Cross [13] at Nottingham Trent University highlighted that staff time constraints were a key barrier to the uptake of the RLs at their institute. Beasley [16] found that familiarity with the system, staff time constraints, and perceived usefulness of the system were also hindrances at the University of Auckland. Krol [9] discussed resistance and lack of interest by academics. Despite RLs being created for all courses with the help of library staff, the academics’ engagement with the RLs creation remained low due to a cited lack of time [9].

Most studies identified significant hurdles for academics to overcome in order to usefully engage with RL systems. While Zhu [14] found that the academics valued the facility of the sharing of copyright material via the RLs, 40% were dissatisfied with the overall RLs’ functionality, stability and ease of use. Consultations with staff at the University of Manchester identified the need for improved functionality of the system as well as integration into the learning management software, better support for users, and marketing to their users of the potentials and capabilities of the system [15]. Neil and Masto [17] found that academics at the Dublin Business School wished for better integration of RLs with their learning management system and also identified time constraints as the main barrier for academics to use the RLs. Other factors highlighted as hindrances to RLs uptake were the discipline and lecturing experience of the academics. Taylor [18] agreed with her colleague Devine [19] in arguing that the RLs needs to go beyond being a repository of teaching materials but should become a teaching tool in its own right. However, in what way RL system and a learning management system would integrate has not been addressed.

Academics also reported concerns that the RLs may not provide enough cost benefits for them and their students. Brewerton’s [1] study at Loughborough University found that some academics were not convinced that their efforts in maintaining the RLs were appropriate in comparison with the perceived benefit to the students. Cameron & Siddall [12] even noted concerns voiced by academics about RLs effectively “spoon-feeding” students and observed a lack of effective communication between librarians and academics.

The content of RLs is reported sparingly [13]. The literature also touches on the types of content found in RLs [9, 12] and where it has, studies do not distinguish between digital and print material. Cameron and Siddall [12] found that books contributed on average 74% of RLs material at the University of Northampton. While 13% of these books were available as eBooks, only 3% were labeled as such. Krol [9] found that books and eBooks were included in 96% of all RLs at the University of West London. Librarians at Loughborough University gave anecdotal evidence that engineering RLs were typically short and comprised mostly books, whereas humanities lists were longer and used more articles [1]. We observe that many of the available publications are reports reflecting on an institute’s journey and did not use log analysis. Table 1 provides an overview of the discussed studies on academics’ use of the RL systems. While eight out of the nine studies focused on academic experience with RL systems, only two studies used a detailed log analysis. Of these two, Beasley [16] focused predominantly on a single semester. Krol [9] covered a four-year period (2016–2019) but limited the study to a single faculty (Computing and Engineering). The generalizability of their results is therefore limited. In addition, only four of the nine studies covered all faculties at universities. Finally, only two studies included librarians’ views, with Cross [13] exploring the technical challenges, and Beasley [16] limiting the focus to two faculties. Therefore, integration of the views of academic liaison librarians’ experiences with RLs remains open.

Table 1 Summary of the user experience on RL system studies
Full size table

3 Method

This section describes the study context, method, data collection, data preparation and preprocessing for analysis. Our study employs a mixed-method approach [20] including a transaction log analysis and two questionnaires.

3.1 Institutional context

The UOW has 13,076 students and 649 academic staff [21]. Our study was conducted across all eight faculties, including Art and Social Sciences (FASS), Education (FEDU), Science and Engineering (FSEN), Waikato Management School (WMS), Maori and Indigenous Studies (FMIS), Computing and Mathematical Sciences (FCMS), Health, Sport and Human Performance (FHSHP) and Law (FLAW) (see “Appendix A” for particularities of the each of faculty). In addition, six academic liaison librarians participated. Academic liaison librarians work with academics to provide individual assistance for teaching and research. They also provide individual assistance on reference services, system support services and copyright tracking. To make the services more decentralized, each faculty is assigned with two academic liaison librarians.

Waikato Reading Lists (WRL) are typically created for each course instance, being assigned to different semesters and years, such as Summer Schools S and T, Semesters A and B, whole year Y courses, and Semester C (all other periods). Most students attend Semesters A and B, with fewer in Summer Schools S and T. Y and C are rarely used, mostly for postgraduate studies.

3.2 Study method

The first phase of our mixed-method study consisted of a transaction log analysis [22] of WRL creation over five years (covering January 2016–December 2020). The second phase of the study used online questionnaires for academics and librarians, which were designed based on the findings of the first phase of the research. The transaction logs recorded actual behaviors of academics’ in RLs creation at the University of Waikato. We used these logs to identify trends and patterns in user behavior. Some of the insights we derived through these trends needed to be confirmed and clarified through insight from users. Our questionnaire aimed to explore the users' reasons for the trends we had been able to identify from the log analysis. Our questionnaire was designed to gather data on academics’ and librarians’ experiences and opinions of use of reading lists based on our insights from the log analysis as well as unanswered questions about reading list use. Following ethics approval, the online questionnaire was emailed to selected academics and the academic liaison librarians. We here describe the data collection process for the two phases of our study.

3.2.1 Phase 1: log analysis

The raw data from the WRL transaction logs (including information about the lists creation and content) were automatically processed and collated into tables of summary data. Any RLs that were deleted from the system do not appear in the logs. Table 2 describes the preparation of the transaction log data for analysis.

Table 2 Preprocessing steps for analysis
Full size table

3.2.2 Phase 2: questionnaires

The study population for the questionnaire consisted of the UOW academics and the librarians who were involved with WRL as list creators in at least one case; 305 academics were identified. The questionnaires comprised close- and open-ended questions and were constructed under three main themes (see “Appendix B”):

  • Experience of the WRL setup

  • Experience in linking an eBook or other teaching materials in WRL

  • Experience of the ongoing use of the WRL

The electronic questionnaire was emailed to the 305 academics in June 2020; the survey ran for two weeks. A total of 73 replied to the questionnaire within the stipulated time period. This sample (73 respondents of 305) represents a response rate of about 24%. For an online survey of this kind of domain, conventionally, a response rate of 20–30% is considered a good result and is similar to other surveys in this field. In addition, 10 academic liaison librarians were identified based on their departmental responsibilities, who also received a customized questionnaire (see “Appendix C”). A total of 6 replied to the questionnaire within the given two-week period.

4 Results and analysis of transaction log data

This section presents the results of our log analysis. RLs may be newly created or rolled over (i.e., re-published year on year). RLs owners are then either new owners or owners of rolled-over RLs.

4.1 Reading lists over time

Figure 1 shows the total number of RLs created across all eight faculties. The overall number shows an upward trend (except for the sudden decline in 2020). We believe the decline in 2020 was predominantly due to the move to online teaching during COVID lockdown in New Zealand (taking full effect in B Semester 2020).

Fig. 1
figure 1

RLs active in each semester grouped by years

Full size image

To further explore this behavior, we analyzed the RLs in each faculty. We found that RL’ numbers were higher in the four faculties FEDU, FASS, FSEN, WMS (~ 100 RL per year, see Fig. 2), compared to the other four faculties FMIS, FCMS, FHSHP and FLAW (less than 30 RLs).

Fig. 2
figure 2

RLs created by each faculty grouped by years

Full size image

The number of RLs evaluated in relation to the number of courses offered within each faculty appears in Fig. 3. Fluctuating numbers of RLs may therefore be explained by changing numbers of courses offered.

Fig. 3
figure 3

UOW papers offered in 2016–2020 with and without RLs, sorted by faculties. Note: At the UOW, term ‘Papers’ refers to the subject, course or a module

Full size image

A comparison across faculties shows the following percentage of courses with RLs: FCMS 3%, FASS 30%, FLAW 31%, FMIS 37%, FSEN 47%, WMS 55% FEDU 63%, FHSHP 63%. We particularly observe that FCMS has a very low RLs percentage (e.g., 2% in 2020). While the faculty offered a considerable number of courses (170 in 2020), only a few staff (3 of 50+ in 2020) were engaged with the creation of 4 RLs (see Fig. 4). The reason may be that FCMS does not use many copyrighted materials and the WRL thus does not cater to the disciplinary needs of FCMS.

Fig. 4
figure 4

WRL created overtime by academics

Full size image

Overall, RL numbers at the UOW rose in small increments over the five-year period, despite variation in the contribution of each faculty.

We identified two causes for high RL numbers in faculties: the overall number of courses offered and the number of academics engaged with WRL. Faculties that have many RLs often had many academics involved in creating RLs (e.g., FEDU see Fig. 4). For example, FASS added a considerable number of new RLs each year, involving many academics (60–75).

Untypically, only a few staff (between 2 and 4) were involved in creating the large number of RLs in FSEN, which turned out to have been created by librarians. After an initial push in which 150 RLs were created, most RLs were being rolled over each year.

4.2 Items in reading lists by type

Figure 5 shows that the total number of items in the WRL increased gradually each year. We observe that at the beginning (2016–2017) book/book chapters contributed far more items than the other two categories (43% in 2016 and 44% in 2017). However, by 2018/2019 articles/journals had risen to about the same proportion, and by 2020 they had become the largest proportion of linked resources (43% of RLs content in 2020).

We further explored the proportion of online and physical items included in the RLs (see Fig. 5). We observe that the growth in RLs resources was mainly due to the increased inclusion of online resources over the five years. Taken together, online resources in RLs increased from 6791 items (54% of total RLs content) to 16,072 items (69% of all RLs content), while physical resources increased from 5903 (46%) items to only 7324 (31%).

When comparing the items linked in RLs from different faculties (see Fig. 6) in 2020, we observe field-based differences in the RLs composition. One group of faculties mostly linked articles/journals (FLAW 55%, FHSHP 55%, FEDU 52%, WMS 46%); another group had mostly books/chapters (FSEN 86%, FMIS 55%, FASS 51%). Most prominent here is FSEN with 86% books/chapters. Finally, FCMS was the only faculty to mainly link other items (55%).

Fig. 5
figure 5

Total number of items in the WRL (2016–2020)

Full size image
Fig. 6
figure 6

Number of items in faculty RLs 2020

Full size image

When comparing the RLs linking of paper-based items (physical) and the online/digital material for each faculty in 2020, the majority of faculties is predominantly including online/digital material (FHSHP 77%, WMS 76%, FCMS 72%, FLAW 71%, FEDU 71%, FASS 62%, FSEN 58%), with FMIS being the only faculty with only 48% of online/digital material.

When analyzing the format of items included in RLs, we find that all faculties used online/digital articles/journals more often than the physical version (FSEN 97%, FASS 97%, FEDU 93%, FCMS 63%, WMS 98%, FMIS 77%, FLAW 74%, FHSHP 99%). Similarly, other items were predominantly digital (72% to 97%). The picture is more nuanced for books/chapters: most faculties use predominantly physical books/chapters (FLAW 81%, FMIS 74%, FASS 66%, FEDU 65%, WMS 61% and FHSHP 59%), and only two faculties use predominantly digital books /chapters (FSEN 52%, FCMS 70%).

5 Results and analysis of questionnaires

The questionnaires investigated the experiences of academics and academic liaison librarians, respectively, with WRL under three sections: the WRL setup, linking an eBook or other teaching materials, and ongoing use of the WRL. We received responses from 73 participants, representing 7 of 8 faculties and thus almost a third of the total number of academics who were invited to participate (see Fig. 7). We did not receive an equal number of participants from each faculty. The faculties were represented as follows: FASS (24), WMS (18), FEDU (15), FLAW (5), FCMS (4), FMIS (4), FHSHP (2) and FSEN (0). In addition, one academic participant did not indicate their faculty. Six of 10 librarians responded to the questionnaire.

Fig. 7
figure 7

Respondents (academics) sorted by faculty

Full size image

5.1 Academics’ and librarians’ experience with the WRL setup

5.1.1 Use and structure of reading lists

Figure 8 shows that 36 of 73 (47%) academic respondents sought the help of others when setting up the WRLs, and about one-third of all respondents (24 of 73, 33%) had someone else setting up the RLs for them (more than one response was permitted).

Fig. 8
figure 8

WRL set up by the academics (n = 73)

Full size image

Figure 9 summarizes the kind of help the respondents sought when setting up the RLs, which was gathered through an open-ended question. Eight responses were identified as irrelevant to the question asked. Out of all the valid responses, help-seeking for the initial setup of the RLs was the most common factor for most of the faculties. We observe that the three highest factors (initial setup, uploading materials, and linking materials) were the basic functionalities of the WRL.

Fig. 9
figure 9

Help sought by the academics (n = 53). Note: Respondents are permitted to express more than one reason

Full size image

All six academic liaison librarians acknowledged that they set up RLs on behalf of academics (see Fig. 10). When comparing the help provided by the librarians to academics, we see tasks such as copyright guidance, linking of material, troubleshooting and guidance for linking online material such as eBooks and chapters.

Fig. 10
figure 10

Reasons and help provided to set up the WRL by the librarians (n = 6)

Full size image

Academics were asked how many of their courses used rolled-over RLs (i.e., RLs that were used in multiple years), see Fig. 11. The majority (63 of 68) had used rolled-over RLs (on average 2.63 times) and only 5 had never rolled-over a RL. This continued use of RLs is higher in faculties such as FASS, WMS, and FEDU; we note that the same faculties also had higher numbers of overall RLs (see Fig. 3). This may indicate that academics who created higher numbers of RLs tended to engage with them in the following years as well.

Fig. 11
figure 11

Number of courses taught by the academics which had RLs more than once (n = 70)

Full size image

We enquired about the types of materials that academics included in RLs (see Fig. 12). The majority of the respondents (62 of 73) indicated academic journals/articles as the most commonly included teaching material in the WRL, followed by books/eBooks (56) and web pages (28). We found that the top three resources were common across all seven analyzed faculties.

Fig. 12
figure 12

Teaching materials included in the RLs by the academics (n = 73)

Full size image

However, the most-often mentioned RL materials were different across faculties, with academic journals/articles being dominant for academics from FASS, WMS, FEDU, and FMIS, and scans books/eBooks being prominent for academics for FLAW and FCMS.

Surprisingly, web pages were the most often mentioned resource for FHSHP, but the respondent number was very low. The inclusion of such materials as web pages shows that WRL contains material in addition to the legally required copyrighted publications.

5.1.2 Ease of interaction with reading list setup

We sought feedback on the academics’ experience of setting up WRLs and received a mixed response (see Fig. 13). Twenty-eight of 72 participants agreed with the statement that WRLs are clear and easy to set up, 22 were neutral, and 22 disagreed. We did not identify any particular patterns across the faculties. Fifty-five participants provided further reasons for their feedback. Twelve of 55 participants commented that they found the interface intuitive and easy to follow (as reasons for their positive rating). Forty-three of 55 respondents gave a variety of negative feedback such as hard to understand, complicated and not intuitive/user-friendly (27), wanting more guidance on the process (15), difficult to link materials (7), time-consuming (6), and increasing the workload (1).

Fig. 13
figure 13

Setup of the RL is clear and easy to interact with—academics rating (n = 72)

Full size image

Answering the same question, 3 of 6 librarians were neutral, 1 disagreed, and 2 (strongly) agreed. Even though librarians have more experience and training on the setting up of the WRLs, their overall feedback was not very positive and in line with the academics’ views: not intuitive (2), okay after some use/training (2), minimal integration into other university systems (1).

Final feedback on the setup of RLs was given by 38 participants, including the following points: needing further guidance/struggling with the interface (12), appreciating the library support (9), not worth spending time / unnecessary burden (5), needs better interaction with Moodle and course outlines (2), easy to use and helpful (2). In summary, we identify a need to improve the system’s usability.

5.2 Academics’ and librarians’ experience with digital material in the WRL

Forty-three of 72 participants stated that they had successfully linked digital materials with their RLs. For the 29 participants who had not successfully linked digital material, we received the following further responses:

  • Participants had issues with using the RL system: Nine participants did not know how to link digital resources with the RLs though they were aware that the functionality was available. Other reasons provided further indicate the need for improving the linking process, such as confusion about the linking process (9), preference for other methods [of providing material] (8), and difficulty of the linking process (4). Six academics expressed a lack of awareness for the functionality of linking eBooks (6).

  • Participants chose not to link digital material for students in their RLs: Reasons given included: not using digital material in the course (5), preference for print material instead (2). Some access issues were mentioned (e.g., material not provided by the library). One participant said they circumvented the RL system, by a link[ing] directly to an eBook itself, not [via] the RL software (1). Another participant reported that a librarian had linked eBooks and other teaching materials for them. Reasons unrelated to the RL system were also cited (e.g., do not remember).

These findings highlight the need for additional training and support by the academic library, as well as the need for an easier interface for linking digital materials. The six librarian respondents were all able to successfully link digital teaching materials with the WRL.

When asked about the ease of linking digital material with the WRL, a large group of academic respondents (18 of 65) remained neutral (see Fig. 14); 24 were (strongly) positive and 23 gave (strongly) negative responses. In addition, four of 6 librarians were neutral, and one agreed, and one strongly agreed to the ease of use of RLs. Thirty-two of the 65 respondents made additional comments regarding the ease of linking material. While 5 respondents found it easy to link material, 27 of 32 respondents gave a variety of negative feedback such as difficult to understand (10), functions not memorable (5), and specifics such as that it was hard to decide which links to use (3). Four of the 6 librarians provided further feedback about challenges, such as inconsistencies with linking to resources at online publishers (3), and the difficulty of explaining the process (1).

Fig. 14
figure 14

Linking an eBook with WRL is simple and clear—academics rating (n = 65)

Full size image

The feedback from academics and librarians indicates that there are intricacies in linking digital material (particularly eBooks) due to the heterogeneity of the data structure in provider websites.

5.3 Academics’ and librarians’ experience with using WRL

Twenty-seven of 71 respondents gave positive affirmation to the sentence ‘The WRL made my job of teaching easier’, 20 gave a negative response and 24 remained neutral (see Fig. 15).

Fig. 15
figure 15

WRL made my job of teaching easier—academics rating (n = 71)

Full size image

From the detailed feedback of 52 participants, we see that 14 felt that WRL added to their workload without any perceived benefits for their teaching. Additional critical comments were the lack of integration with other university systems such as Moodle or Course Outlines (7), doubts about actual usage by the students (5), difficulties in setup and use (4), time-consuming (2). Positive feedback included the advantage of having everything in one place (7) and reduced workload (3).

The advantage of having everything in one place of course requires the academics to include not only the required copyrighted material but all other reading material in the RLs (or to omit using non-copyrighted material).

When further examining the usefulness of WRL to academics beyond the support of teaching, 22 out of 71 respondents were neutral, and 39 were positive and only 10 negatives (see Fig. 16). Thirty-six of the 71 provided further feedback, many of whom repeated points they made in earlier comments. For example, 7 of 36 found the system unnecessary, 5 doubted the students would use the system, 3 wanted better integration into university systems, 3 did not find it easy to use. On the other hand, 5 liked to have all their reading material in one place, 3 felt it benefited the students, 3 liked that it helped to adhere to copyright rules, and 2 felt it was easy to roll over existing RLs.

Fig. 16
figure 16

Having the WRL is useful—academics’ rating (n = 71)

Full size image

When asked about the satisfaction with the existing integration with Moodle (Fig. 17), 24 of 68 academics were neutral, 32 were positive and only 12 expressed a negative opinion. In contrast to the academics’ impressions, the academic liaison librarians’ responses on the same question were more negative: 3 gave negative feedback, 1 was neutral, and 2 were positive. In their detailed feedback, 12 respondents (9 academics, 3 librarians) stressed that the system needs better integration, 5 asked for the RLs to be more visible within Moodle (to encourage students to use RLs), 4 suggested to auto-populate RLs information. They suggested that the streamlining of integration may include, for instance, the inclusion of the readings with the relevant week/module/an individual item or topic.

Fig. 17
figure 17

Satisfaction of integration of WRL with Moodle—academics rating (n = 68)

Full size image

Finally, respondents volunteered some suggestions to improve the WRL, which we grouped as follows: better integration of the RLs with other teaching systems (11), easy steps to complete the functions (9), help and user guidelines on how to use the RLs (6), better UX features and user-friendliness (6), see Fig. 18. In addition, the librarians made the following suggestions on system functionality: adding a dashboard that measures students’ engagement (1), adding a function that facilitates hiding sections in the lists (1). Two further librarian comments suggested enforcing the use of RLs by academics (1) and abolish the system in exchange for a system that is based on Moodle (1).

Fig. 18
figure 18

Suggestion on how to improve the WRL (academics, n = 35)

Full size image

In summary, respondents’ overall feedback on the ongoing use of the WRL reveals that the majority acknowledged the usefulness of the WRL. However, their detailed responses strongly suggested that improvements are necessary, including a streamlined workflow for the WRL functionalities and better integration of the WRL with other university systems used in teaching.

When comparing the feedback from academics from different faculties, we do not find any discipline-specific feedback about the ease of use of the WRL.

6 Discussion

We discuss insights from our studies reported in this article, particularly in relation to related literature.

6.1 RL numbers over time

We observed that similar to other studies [9, 15, 16], the initial years of WRLs saw low numbers of RLs, with only a few academics being involved. In later years, the number of RLs grew at UOW, similar to other studies. In the engagement of academics with RLs creation, we identify two trends at UOW: RLs creation were at very high levels in four faculties (FEDU, FASS, FSEN, and WMS) and at low levels in the other four faculties (FMIS, FCMS, FHSHP and FLAW). While some of these differences may be explained by differing faculty size, there also seem to be disciplinary differences. Other studies confirm similar variations between faculties [16, 17], but the specific pattern observed in our study had not been observed.

Related work reported on a number of interventions that helped with increasing RLs numbers, e.g., library staff creating the lists, a dedicated team to create the lists, improving communication between library and academics, continuous training [9, 15]. At the UOW, the situation was found to be somewhat different: while more RLs were created over time, their overall increase was lower compared to that found in other studies. Even though similar interventions were in place at Waikato (e.g., a team of librarians creating RLs), the overall academic engagement remained low.

6.2 RL content types

From the questionnaire, we found that 85% of academics reported the inclusion of journals/articles, and 77% reported using books/eBooks in WRL. In addition, 38% of academics reported including other materials, such as web pages [12]; survey at the University of West London revealed books /eBooks being the most popular format for RLs inclusion (96%), with chapters/articles or audio-visual material being less often included (60%). We observe that overall, very little data are available and are difficult to compare due to conceptual differences.

From the log data, we found that 43% of all materials included in WRL in 2020 were journals/articles, 40% were books/chapters, and 17.5% were other materials. This seems to mirror the preferences stated by academics during the survey.

When analyzing the difference between faculties, we found that most linked journals/articles and books/chapters were included in the close-to-equal measure. The only exceptions were Science/Engineering with a clear preference for books/chapters (86%), and Computing with a preference for other items (55%). Even though the WRL content was driven by the need to acknowledge copyrighted material, these differences between faculties may highlight area-specific preferences. While Brewerton and Krol [1, 9] similarly commented on the differences between disciplines, their observations were merely anecdotal. Our study is the first to observe these differences in a log study.

6.3 Satisfaction with RL system

We found that at the UOW very few academics set up RLs independently, with 47% seeking help and 33% fully depending on someone else to set up the RLs (see Fig. 8). The main reasons given were the complexity in setting up RLs, and time constraints, which reflect the observations in other studies [9, 12, 14]. Many academics felt that the RL system was unnecessary, doubted the actual usage by the students, and disliked the lack of integration with other university systems. One of the challenges identified by the academics and librarians points to the inconsistencies between the online systems offered by publishers in terms of linking to eBooks, and to chapters/parts of eBooks. A standardization at the publisher or RL system level would greatly ease the burden to the creators of RLs.

Fourteen percentage of respondents did not find the WRL useful, and the feedback of the 31% of the respondents who remained neutral revealed that many were not satisfied with the RLs functionality. This is only a slightly better result than that reported by Zhu [14] where the majority of academics were dissatisfied with their RL system, as they did not find it stable and easy to use. Krol [9] similarly reported that only 50% of their respondents felt comfortable using the RL system, due to time pressure, lack of training, and lack of confidence. Our study results thus agree with earlier studies [9, 12] in observing that time constraints were a limiting factor in RL engagement by academics. We conclude that similar to many other RL systems, the WRL system did not offer academics sufficient incentives (i.e., teaching-related benefits) to want to invest their already-limited time.

From these reasons, we hypothesize that a lack of relevance of the RL system to the academics’ work as teachers may have been a barrier. It remains unclear from our study data if this lack of relevance was caused by an “asymmetrical disconnection” [23] between librarians (who set up and maintain the RL system) and academics (who use RLs in their courses). This may have led to a potential misalignment in the system purpose and academic needs.

Krol [9] found that 84% of their respondents agreed that RLs can be useful as a pedagogical tool. The original concept of academic RLs was to provide pedagogical ‘scaffolding’ by academics to students via signposting and annotations [3]. However, the WRL system does not currently offer comprehensive pedagogical support. We note that the opportunity of establishing a RL system as a pedagogical tool has so far been missed. We believe the reason is that the system was acquired predominantly to address the legal requirement of copyright reporting, and any potential pedagogical benefits to teachers or students were not considered.

6.4 Implication for digital libraries

Requirements of tracking the reading materials [5, 6, 24] and the integration of digital libraries in academic learning environments [7, 8, 23, 25] were highlighted in previous studies. From the results of our study, we conclude that for a digital library to provide reading list functionality, linkage to both internal documents and documents provided by external publishers is required. In addition, linking to external sources, such as webpages, needs to be supported. A seamless access to the different content types would be of importance. Features that could support pedagogical commentary would need to be provided by the digital library system to go beyond lists of content for each course.

7 Summary and conclusion

This article provides insights into the introduction of a RL system at the UOW over five years, and the experiences of academics and librarians with the University's RL system. From our log analysis and questionnaire responses, we draw the following five conclusions:

First, the faculties showed great variation in RLs uptake ranging from 3 to 63%, with a mean of 41, which we believe may be due to differing disciplinary requirements. In order to better support the various faculty teaching strategies, we recommend that the library takes a flexible approach and develops discipline-specific initiatives to increase the number of RLs.

Second, the results from our log analysis study confirmed the differences in RLs content across disciplines which was previously only reported through anecdotal evidence.

Third, we found the RLs setup process to be complex and time-consuming and only a few academics managed to set up the RLs independently. We recommend streamlining the RLs creation process.

Fourth, we observed that many academics struggle to link eBook and other teaching materials with the RLs and do not perceive the linking process to be user-friendly. We identified a need to improve the usability of the resource-linking process.

Finally, while acknowledging the University’s requirement to implement RLs for recording copyrighted materials, we see a significant need for pedagogical support in WRL to better integrate into academic teaching. We believe that the uptake and perceived usefulness of WRL could be improved through enhancing the pedagogical benefits and integration of RLs into teaching activities and support systems.

This study focused on exploring the experiences of those academics who were involved with creating RLs. As an extension of our work, it may be interesting to explore the feedback from those academics that did not engage with RLs. We are currently carrying out a companion study that explores the students’ perceptions and experience of RLs. Insights from this second study will help guide our future research into pedagogical features needed for RL systems.