Introduction

In keeping with its land grant mission, a university academic library partnered with several Open Educational Resource (OER) advocacy efforts on both the national and state levels to promote the creation and use of OER at the university. OER are instructional materials considered free to the end user and licensed to facilitate affordable or no cost distribution, replication, and modification (Almeida 2017; Hilton III et al. 2013; Jhangiani et al. 2018). Through its membership and leadership in this advocacy network, the library was able to offer innovative OER advocacy materials and programs, and to leverage external grant funding for the development and support of OER on campus.

While the program had some initial success in inspiring faculty to create and use OER in their courses, the effort proved difficult to sustain. Wiley (2007) defines sustainability as a program’s “ongoing ability to meet its goals” (p. 5). Issues of sustainability in the OER arena include both production and sharing of resources (Wiley 2007, p. 5). Costs associated with the production and sharing of OER include people, workflow development, infrastructure, incentives, education and advocacy (Wiley 2007, p. 5). These costs are proportional to the complexity of the OER, with increased development and technical resources and skills required for development of “interactive applications, games and simulations” (de Carvalho et al. 2016). As is common with non-income generating projects whose support is provided by “targeted, external funding” (Wiley 2007, p. 5), we faced questions concerning specific strategies necessary to best position the initiative to be self-sustaining once grant funding was depleted (de Carvalho et al. 2016; Wiley 2007). To address these challenges, we applied principles and practices from the Performance Improvement/Human Performance Technology (HPT) Model (Van Tiem et al. 2012) to better position the OER initiative for long-term sustainability. Although this project was not originally envisioned as a case study for publication, we sought and received institutional review board approval to analyze archival data from the project once we realized our experience may be of interest to a wider audience.

In sharing our experiences with this project, we seek to accomplish two goals. First, we offer insight on ways to support and sustain OER. Studies indicate a number of institutions have OER related policies in place, but these policies do not appear to consistently impact the institution as a whole (Andrade et al. 2011). Faculty and students express interest in innovative practices and affordability related to OER adoptions (Baker and Ippoliti 2018; Waller et al. 2018); however, this does not seem to “translate, to the same extent, into the existence of organization wide implementations” (Andrade et al. 2011, p. 8). The findings of the performance improvement process described here led to the development of strategic priorities that will help us stabilize infrastructure, increase instructor participation in the OER initiative, improve the quality of OER materials produced and increase the reach of our OER program.

Second, we provide a case study in how the performance improvement process can be applied in a higher education venue. In many instructional design programs offered by universities, courses in HPT are offered as electives but are not required as part of core study (Klein and Fox 2004). Demonstration of a practical application of HPT in a setting familiar to university students, as we present here, can provide valuable professional development opportunities through which students, researchers, and practitioners across multiple disciplines can improve familiarity with the process of HPT, inspiring them to incorporate its associated research and theories into their own practice. This paper describes how the Performance Improvement/HPT model (Van Tiem et al. 2012) was applied to improve an Open Educational Resources (OER) initiative at an academic library, as an example of how HPT might be applied to improve scalability and sustainability in other OER programs. This paper focuses specifically on three phases of the process: organizational analysis, environmental analysis, and gap analysis.

Performance Improvement Process

The OER initiative at this university began with the passionate interest of two librarians. In Spring of 2014, a librarian solicited student input regarding textbook use and costs. Based on this student input, the library identified the exploration and support of OER as a priority during the library’s strategic planning process. Open Education Week 2015 was used to publicize the library’s interest in and support of OER, and in the Fall of 2015, the library hired a Ph.D. student whose research interests included OER to assist with the library’s OER efforts. In 2015, a private donor with a passion for helping make college more affordable for students provided the library with one-time funding for a pilot open textbook initiative. Completed textbooks were housed as static PDFs in the library ePress for use by faculty members as primary textbooks in associated courses.

Due to a death, a graduation, and an unexpected move, the library found itself rather suddenly needing to replace the entire team responsible for the OER initiative. Having experienced mixed levels of success using one-time grant funding to begin the project, the library added a full-time OER librarian to facilitate the transition of the effort from a pilot project to a strategic plan. The OER librarian, needing to understand and improve the existing structure very quickly without much guidance, identified the application of Human Performance Technology (HPT) as an appropriate systematic way to analyze and address the sustainability problem.

Human performance technology (HPT), also known as Performance Improvement Technology, is a “process that bridges the gap between what is and what should be in human performance systems” (Wilmoth et al. 2002, p. 16), improving human “productivity and competence” (p. 16) through a systematic approach including performance analysis, cause analysis, intervention selection, intervention implementation and evaluation. Opportunities are realized and performance issues addressed through both “instructional and non-instructional interventions” (Klein and Fox 2004). The OER program was a human performance system (Wilmoth et al. 2002) where HPT could serve as a “unifying process that helps accomplish successful change [and] create[s] resilience and sustainability” (Dessinger et al. 2012, p. 10). In the following sections we will present the process and findings from the ongoing performance improvement project, which includes systematic performance analysis, needs assessment, intervention design, and evaluation. Completion of an organizational analysis allowed us to identify areas where the university and library missions align with the goals to be achieved through the integration of OER. Through an environmental analysis, we identified aspects of the organizational environment and key elements of the university workflow which might impact the integration of OER. The current state was observed, articulated, and compared to the desired state to facilitate a gap analysis. A cause analysis helped determine which interventions would be most valuable. Interventions selected, designed, implemented and evaluated will be included in the library’s long-term plan for intentional support of OER. The organizational, environmental, and gap analyses will be presented here.

Researcher Reflexivity

In case study research, trustworthiness is strengthened through the researchers’ articulation of how the researcher impacted phenomena under study as well as the impact of the research process on the researcher (Probst and Berenson 2014). Without shifting attention away from the phenomenon of study (Probst and Berenson 2014), we wish to acknowledge the positionality of the first author in this research process in which the researcher was central to data collection and analysis, “investigative strategy, and the end product” (Merriam and Tisdell 2015, p. 37).

The first author on this paper served as the academic librarian who spearheaded the project. As such, the first author was “an active part of the setting, relationships, and interpretations” (Probst and Berenson 2014, p. 814). The study was carried out to meet the professional purposes of the first author as OER Librarian, the practical purpose of informing performance improvement in the academic library, as well as research purposes described above (Watt 2007). Although the perspective of the first author could “unknowingly influence” the work, we selected and analyzed data based on its relevance to our line of inquiry (Yin 2018). To prepare our research report, we created a set of questions helping to remind us “of the data to be collected and why” (Yin 2018, p. 99). We created a list of sources which might contain useful evidence, and followed the crosswalk method described by Yin (2018) through which case study researchers determine the relevance of potential evidence in relation to the “questions of interest” (p. 99).

Organizational Analysis

Organizational analysis involves an examination of the values, mission and vision of the institution as a whole, to ensure that the specific project under study aligns with these larger goals. To determine this alignment, we examined communications formally released by the university, familiarized ourselves with the recent history of the institution, and read through the minutes from the previous two years of meetings of a faculty committee known as Faculty Council. To determine alignment from the process perspective, we relied on a review of the literature surrounding open educational resources.

The Institution

The university, recently designated as very high research, is a land grant system with an enrollment of more than 35,000 graduate and undergraduate students. A description of the institution and its mission and vision statements were easily located on the university web page. Minutes from Faculty Council meetings were also published on the institution’s website. An examination of these documents revealed that the university values community, excellence, diversity, integrity, service, intellectual freedom, and stewardship. Goals to which the work of the institution is oriented include academic excellence, student success, and community engagement. As a land grant institution, a defining aspect of its role is the extension of accessible educational opportunities into the community, along with a commitment to research, high-quality teaching, service and outreach.

Minutes from the Faculty Council meetings indicated the faculty were themselves oriented to the goals of academic excellence, student success, and community engagement. Documented conversations, activities and commitments reflected the institutional mission and vision, showing concern and commitment to teaching, scholarship and research. In the previous semester, faculty had indicated a particular concern with the high cost of textbooks and the impact textbook affordability might have on student success.

Open Educational Resources

A review of the literature revealed that facilitation of student success is a fundamental goal motivating the use of OER. Students whose classrooms use OER achieve learning gains similar to or better than those whose classrooms use traditional, purchased textbooks (Hilton et al. 2019). OER may offer the opportunity to reduce student expenses (Baker 2019), thereby decreasing barriers to access to higher education for underprivileged populations (Baker et al. 2017). The creation and licensing of educational resources that can be freely shared or adapted is intended to maximize “access, equity, distribution, participation, innovation, and sustainability” (Stacey and Pearson 2017, p. 7), suggesting a “relationship between openness and social justice” (Crissinger 2015, p. 2). OER are generally intended for initial digital distribution, facilitating a continually iterative process through which creators and users collaborate in strengthening current resources, cultivating innovative practices, and encouraging new ideas (Jhangiani and Biswas-Diener 2017). The digital environment of OER is potentially “exponentially larger and more diverse than that of a traditional university classroom” (Almeida 2017, p. 13), and lends itself to innovative teaching practices and creative pedagogy (Nascimbeni et al. 2018).

Alignment of Values

Having articulated the mission, vision, and core values stated by the university as well as the values and goals associated with OER as described in the literature, we determined alignment of values did, in fact, exist. The role of the land grant institution in extending educational opportunities through community engagement aligns with the goal of broad access to education as held by OER practitioners. Commitment to student success is central to both the mission of the institution and the force motivating the creation, adaptation and use of OER. Use of OER is represented in the literature as providing the opportunity for the diverse and innovative educational experiences included in the institution’s stated core values.

Environmental Analysis

Through an environmental analysis we identified aspects of the organizational environment and key elements of the university and library workflow which might impact the integration of OER. First, we examined the institution’s organizational chart and tenets guiding government of the university to determine at which points policy might be determined and implemented. Then we explored the role of elected student government in reflecting, acting upon, and influencing the concerns of the student body. Finally, we explored the role of the academic library in the institution.

Formation of Institutional Policy

The institution’s organizational chart reflects the Board of Regents as the group ultimately responsible for approval of policy. The University President reports directly to the Board of Regents. The Senior Vice President and General Counsel, Foundation President, Alumni Association President and Provost report to the President. Through and adjacent to the Provost are a series of Presidents, Vice Presidents and Deans concerned with various aspects of the student educational experience. Administrative recommendations concerning institutional policies and procedures are to be communicated through the President.

Although not positioned on the published organizational chart, the faculty handbook describes faculty as essential to the formulation and recommendation of university policy. The welfare of the academic community is established in Faculty Council governing documents as the direct concern of the faculty, with the Faculty Council tasked with the initiation and review of university procedures and long-range plans. The Faculty Council meets monthly, and individual faculty members are asked to communicate recommendations concerning institutional policies and procedures directly to Faculty Council.

Elected Student Government

The student body is represented by both a Student Government Association (SGA) and a Graduate and Professional Student Governing Association (GPSGA). Executive board members for SGA are elected by a vote of the entire student body. Executive board members for GPSGA are elected by representatives and liaisons of associated graduate student academic departments and organizations. The SGA meets weekly in the fall and spring semesters, and GPSGA meets twice monthly in the fall and spring semesters. The presidents of both associations attend monthly scheduled meetings with the Vice President of Student Affairs. Recommendations initiated by students concerning institutional policies and procedures are to be communicated by student organization representatives through the SGA and GPSGA presidents.

Role of the Library

The American Library Association defines librarianship as the study of information including but not limited to the access, organization, and implementation of programs and services (Core Competencies 2009). With the development of digital networked technologies, the roles of librarians and librarianship, in general, have shifted to a multidisciplinary perspective promoting global thinking, creativity, and innovation as part of the core foundations (Ammons-Stephens et al. 2009) Librarianship as a profession provides users with access to a diversity of information materials in different formats and the opportunity to create new knowledge within different programs and initiatives (Core Competencies 2008).

The role of the library at this institution is to enhance student potential and support faculty in their teaching and research pursuits. The library organizational chart includes the Dean of Libraries, three associate deans, librarians, staff, and student workers. The librarians act as academic liaisons and functional specialists who support faculty and students in their subject-specific research. Areas of functional specialty include scholarly services, student instruction, assessment, and open educational resources. The deans, associate deans, and librarians all hold faculty rank. In the case which bounds this study, the Open Educational Resources Librarian is in charge of creating and implementing OER initiatives across campus.

University and Library Workflow

The Faculty Council is positioned as central to the initiation, development and implementation of institutional policy and procedures. Approval for policy ultimately rests with the Board of Regents, but the minutes of the Faculty Council meetings indicate policies suggested by Faculty Council are usually approved. The presidents of the two student government organizations are positioned to share student body concerns through monthly meetings with institution administration. The dean and associate deans of the library meet on a monthly basis with institution-wide administration. Librarians are in direct contact with faculty as academic subject liaisons, and are given a degree of autonomy in carrying out initiatives as functional specialists.

Our environmental analysis found that governing representatives of student, faculty, and administrative populations were in regular communication with both those they represented and with each other. In an interview with the President of the Graduate and Professional Student Government Association, we found that the Assistant Vice President of Student Affairs held quarterly meetings (including breakfast) with the executive boards of the undergraduate and graduate student elected governing bodies. Various councils across campus publish both their meeting schedules and meeting minutes online, enabling us to see which representatives attended which meetings and to what degree the meetings met goals set forth in published agendas. An interview with the library’s associate dean confirmed that each department’s associate deans met weekly with their college deans, and that the deans of each college met weekly with the university vice-presidents and president. Representatives from departments, organizations, and colleges throughout the university attend monthly Faculty Council meetings.

Online organizational documents describe the Faculty Council as positioned to “initiate and review University policies, procedures, and long range plans”, indicating the Faculty Council held considerable influence regarding policies and procedures. This influence is evident in the minutes which record actions such as the faculty council’s discussion of topics such as “what things need to be considered before renaming a building”. We determined this consistent communication among institutional stakeholders, as well as the library’s direct contact with administration, faculty and students, provided the foundation for the development of an effective workflow.

Gap Analysis

After organizational and environmental analysis confirmed an overall supportive environment, we turned our attention to identifying the gap between the actual and desired performance of the OER initiative. Van Tiem et al. (2012) describe gap analysis as a process of clearly articulating the desired and actual state of performance, identifying the gaps, and then prioritizing the gaps so they can be addressed systematically. For this project, the desired state was determined according to a framework offered by Cohen et al. (2018) describing the characteristics of the ideal, or mature, OER program. This framework includes a conceptual structure through which OER programs may be brought to scale. Table 1 represents phases of OER initiatives as described by Cohen et al. (2018). We determined these phases to be an effective framework with which to frame our gap analysis, with the desired state including completion of all four phases.

Table 1 Phases of OER initiatives (Cohen et al. 2018)
Full size table

Data regarding the actual state of the library OER initiative were gathered through documents, artifacts, and interviews. Documents included minutes of faculty council meetings, a survey sent by a faculty council committee regarding faculty awareness of student textbook use, a financial worksheet associated with the pilot project, an archived conference presentation, the library research guides, and a book chapter authored by personnel previously involved with the library OER initiative. Artifacts included posters and fliers from OpenEd Week left by the previous librarian, as well as a copy of the librarian’s obituary describing his commitment to open education and student success. Interviews were held with the current supervisor of the library OER initiative, the former graduate assistant, faculty, and librarians who had been employed prior to the new OER librarian. Use of “multiple methods of data collection” (Merriam and Tisdell 2015, p. 244) enabled us to triangulate the data by comparing what was said in interviews, what we read in documents and what we observed in artifacts. We also triangulated through the use of multiple data sources, interviewing and engaging in conversation with the same people several different times to compare and cross-check our data for credibility (Merriam and Tisdell 2015).

Van Tiem et al. (2012) recommend viewing gaps in performance “as performance improvement opportunities” (p. 155). To analyze the actual performance of the library’s OER initiative we adopted an attitude of “appreciative inquiry” (Van Tiem et al. 2012, p. 155) through which HPT can present performance gaps as opportunities to improve, and celebrate areas in which the actual performance state “is equal to or excels the desired performance state” (p. 155). We applied the question “What is the current state of [the library OER initiative]” (Van Tiem et al. 2012, p. 158) to each subset of the OER program framework (Cohen et al. 2018).

For instance, we asked the question “What is the current state of pilot programs in the library OER initiative?” (Cohen et al. 2018). Documents helping answer that question included texts completed as a result of the pilot program, a financial worksheet detailing past and future payments to authors involved in the pilot program, a research guide on the library website inviting participation to the (now closed) pilot initiative, and a published book chapter authored by those involved in the pilot program. Interview comments such as “We definitely need to develop an MOU for every project from now on”, “We may not ever get all of the textbooks, but we have learned a lot” and “Do we need to keep these funds set aside to pay these authors? Are they going to finish?” supported our identification of the authoring project as a pilot, or preliminary, study.

To answer the question “What is the current state of development of allies in the library OER initiative?” (Cohen et al. 2018) we applied data indicating communication between the Faculty Council Student Affairs and Learning Resources Committee regarding a survey sent to faculty exploring faculty awareness of student textbook purchasing practices (see Appendix Fig. 1). We found peer reviews published on the Open Textbook Network website, which had been written by faculty after attending a library sponsored workshop, and were able to speak with a faculty member who had adopted an open textbook following the library sponsored workshop. The faculty member described a positive experience using the textbook, describing it as “very relatable for students and offered many great tools”. In an interview with the campus bookstore administrator, who had also attended the library-sponsored workshop, we found that the bookstore was responsive to student requests for print and bound editions of OER used in class. Others involved with the library OER initiative were uncertain about the extent of the bookstore’s involvement, asking questions such as “what are the details of our agreement with the bookstore?” We determined that the Faculty Council shared resources, faculty and bookstore staff attendance at the library sponsored workshop, subsequent engagement with the OER community through the Open Textbook Network (OTN) and the bookstore’s willingness to print OER on demand established the present state of the library OER initiative as developing allies (Cohen et al. 2018).

When asking “What is the state of the library OER initiative in connecting to other Initiatives” we found the library involvement with OTN workshops and peer review creation indicative of connection with other initiatives. Funding for the pilot program itself served as data answering the question “What is the state of the library OER initiative in exploratory funding?” The financial worksheet detailing payments associated with the pilot program as well as the research guide inviting participation in the pilot program both described the funding as the result of a one-time private donation. Concerns voiced in interviews such as those quoted above regarding the need to set aside funds from the donation support our finding that the funding was exploratory rather than ongoing.

Using the “six-cell gap analysis” (Van Tiem et al. 2012, p. 157) we organized our findings to identify positive gaps, neutral gaps, and negative gaps in the current state of the library OER initiative as compared to the ideal state recommended by Cohen et al. (2018) (See Appendix Table 2). The process also articulated the potential future state of these gaps “if trends continue” (Van Tiem et al. 2012, p. 157).

Based on the analysis described above, we identified neutral gaps for the library’s OER initiative in the exploring phase of the OER program framework (Cohen et al. 2018). We identified negative gaps for the library’s OER initiative in the evolving and mature phases of the OER program framework. Van Tiem et al. (2012) include “establishment of reasonable goals” (p. 156) in articulating a desired state as essential to successful HPT application. The OER librarian wanted to develop a mature OER program (Cohen et al. 2018) which would first require resolution of gaps in performance with the evolving phase of the OER framework. We determined that resolution of the gap in performance between the actual state of the library OER initiative and the evolving phase of the OER program framework (Cohen et al. 2018) would involve goals that were “both feasible and sustainable” (Van Tiem et al. 2012, p. 156).

We applied various forms of the question “What should be the. ..” to the subsets of the evolving phase of the OER program framework (Cohen et al. 2018) to identify areas for performance improvement opportunities (Van Tiem et al. 2012). For example, we asked “What should be the administrator support of the OER initiative?”, and “What should consistently be taking place in the OER initiative?” These subsets included development of the program, increased capacity, administrator support, and legislator support (Cohen et al. 2018). The gaps were prioritized during brainstorming sessions with library administration, and the decision was made to address increased capacity, administrator support, and development of the program (Van Tiem et al. 2012).

The Work in Progress

After completing the organizational, environmental, and gap analyses described in detail above, we completed the HPT process with a cause analysis followed by an intervention design. The implementation of this intervention is still in progress. The cause analysis revealed that although steps had been taken to initiate and explore the use of open educational resources on campus, a structure had not been established through which the program could evolve and mature. The library had identified the exploration and support of OER as a priority during its strategic planning process, but a framework through which this could be accomplished was not articulated. The lack of documentation of OER efforts hindered accountability for projects underway as well as consistency of effort despite changes in library personnel. An intervention was needed to clearly document the policies, procedures, and accomplishments of the OER initiative. In addition, limitations of the e-press platform, which supported documents only in pdf format, were identified as a cause of low adoption and slow progress.

The desired performance of the library’s evolving and mature OER initiative (Cohen et al. 2018) includes infrastructure that facilitates increased participation as well as improvements in the quality and reach of the program. Having determined a lack of documented strategic planning as a defining cause of the gap in performance, we identified interventions through which this gap might be eliminated. Suggested interventions included staffing and infrastructure choices specific to the library’s OER initiative, long-term strategic planning, and documentation of planning, implementation, and evaluation associated with the library’s OER initiative. The ongoing intervention addresses all of these concerns. Staffing issues are being addressed through hiring graduate assistants and student workers to assist the OER librarian, and better coordination of efforts with other library personnel, such as the instructional designer. To improve the infrastructure dedicated to the OER initiative, the library purchased an institutional instance of PressbooksEdu, a platform broadly used by the OER community, and will use it to host faculty created OER, curate OER adapted and used by the institution, and share other resources considered useful for campus stakeholders. The weakness in strategic planning is being addressed through the development of mission and vision statements specific to the OER initiative. These statements will guide the development of strategic priorities through which infrastructure can be stabilized, participation increased, and the quality and reach of the OER program expanded. Finally, documentation is being addressed through the formation of a campus-wide working group to develop a formal process for documenting the work of the OER team and its partners across the campus.

Discussion and Lessons Learned

Implementation of the HPT process as described in this paper took place in the context of an academic library OER initiative. The initiative had experienced initial success but faced long-term challenges to sustainability. These sustainability challenges were related in particular to issues of infrastructure, participation, quality and reach.

Through the HPT process we were able to establish the OER program as aligned with the mission and values of the institution. This alignment provides some assurance that the OER initiative as a whole contributes to the university’s mission and is therefore worthy of further development (Andrade et al. 2011). Having articulated this alignment, we were able to secure official administrative, faculty, and student support for the library’s OER initiative. The Provost is assisting with the formation of an OER working group and will issue a specific call guiding and supporting its efforts. The Faculty Council passed a recommendation supporting the library in providing education regarding OER and requesting consideration of the creation of OER in promotion and tenure. Additionally, the GPSGA passed a resolution supporting use of OER. Graduate student support is particularly influential, as this constituency experiences OER from both student and instructor perspectives.

HPT has developed increased credibility “in the eyes of those charged with improving organizational efficiency” (Wilmoth et al. 2002, p. 16). Because of our attention to detail in working through each step of the HPT process, we were able to specifically identify causes contributing to the gap between the current and desired states of the library’s OER program. Having identified specific causes, we were able to design achievable, measurable interventions. The rigor and clarity of the HPT process provided the support administration needed to justify allocation of resources needed for implementation and evaluation of suggested interventions. As a result of the HPT findings, the library committed to long-term support of OER initiatives through the hiring of additional staff, re-allocation of responsibilities of current library faculty and staff, and procurement of an institution-specific instance of a digital platform through which OER can be created, adapted, and retained.

We found the HPT process to be valuable, as it helped identify specific aspects of the project which would require change to facilitate a sustainable program. Perhaps our key discovery related to the importance of having a team whose official assignment includes OER-related work. This team can work together to build partnerships including multiple stakeholders and develop a formal mission and vision statement by which the program could be guided. Once the mission and vision of the program have been identified, a strategic plan can be created identifying program goals, plans for achieving those goals, and metrics through which that success will be measured.

Additionally, having completed the HPT process helps the OER librarian avoid repeating past behaviors that hampered sustainability. The research university is anecdotally prone to “siloing,” with day-to-day departmental, teaching, and research concerns creating conditions that favor individuals working in isolation. We felt the results of the HPT study spoke resoundingly of the restrictions such isolated work can place on program growth. As a result, the OER librarian is intentional about building partnerships. OER education is being included as part of seminars and programs presented by the campus-wide center for teaching excellence.

Conclusion

This paper has described how the Performance Improvement/HPT model (Van Tiem et al. 2012) was applied to improve an OER initiative at an academic library and to provide an example of how HPT might be applied to improve scalability and sustainability in other OER programs. We provided a case study in how the performance improvement process can be applied in a higher education venue, demonstrating the practical application of HPT in a setting familiar to university students, through which students, researchers, and practitioners across multiple disciplines can improve familiarity with the process. It is hoped that others might find this account useful in incorporating associated research and theories into their own practice.