Case Study 3: Exploring Open Educational Resources for Informal Learning

Abstract

This chapter explores the potential of informal learning within a Personal Learning Environment (PLE), as well as the identified informal learning cultures that have evolved from the use of Open Educational Resources (OER). A variety of research instruments and strategies have been employed to promote the use of PLEs in this case study and capture a rich variety of feedback from Communities of Practice. In particular, there is a focus on the active use of a PLE and its integration with OER available from the OpenLearn project of the Open University. Additionally, this chapter describes the discovered necessary guidance conditions, emergent contrasting learning contexts and evolving different scenarios in use within the selected Communities of Practice. This research has led to the identification of valuable lessons as well as the documentation of challenges that are faced by those using PLEs in the context of informal learning scenarios.

Description of Case Study

This case study focuses on the analysis of the informal learning opportunities presented by the Responsive Open Learning Environment (ROLE) project. Essentially, this research contains a series of different informal learning scenarios to examine, each of which will be assessed separately. The premise of informal learning (The Organisation for Economic Co-operation and Development (OECD) 2010) in this chapter relates to learning that has been gained through experience and not necessarily from an organised standpoint i.e. the opposite of formal education where pathways are often prescriptive and delivered from an instructor.

Additionally the information presented in this chapter relates primarily to the use of, as well as outputs of the OpenLearn project¹—an Open Educational Resources (OER) repository of the Open University (OU). OpenLearn offers in excess of 12,000 h of self-study materials in a variety of formats. These include study materials repurposed as OER from original OU courses, i.e. initially designed for formal delivery, as well as new bespoke OER created by both OpenLearn academics and other non-OU educators; in other words, further facilitating OER available for informal delivery. As the majority of the study materials presented on the OpenLearn platform are made available using a Creative Commons licence,² it has been possible not only to reuse selected information but also to author new OER materials for the ROLE project, which can be freely shared to a worldwide audience.

This chapter sets out to examine how an existing platform designed for global OER delivery, i.e. OpenLearn, can be enhanced with the introduction of Personal Learning Environment (PLE) technologies. It also considers a selection of social aspects of informal learning because this often plays a key role within a community of learners (Lane 2008a). These groups may have similar backgrounds or goals, so that a PLE can be used to support them throughout their informal learning process. With this in mind, it is also important to remember that the wider OU staff community is allied to the OpenLearn repository. Thus access to OpenLearn has enabled further investigations to take place in active Communities of Practice that contain a cross section of Higher Education (HE) staff: academic, academic-related, technician and librarian colleagues.

A number of research instruments were deployed to gather data and information from these groups. This included using a variety of dissemination opportunities ranging from seminars relating to ROLE and the use of PLEs to the development of an interactive eBook describing Self Regulated Learning (SRL), plus presentations and conference publications. Table 1 outlines all the events and activities that were monitored, as well as their location and date.

Table 1 Overview of the events, activities and artifacts that constitute this case study’s findings

In this chapter, a selection of these events will be described in full detail. Evaluation activities will also be outlined and the subsequent analysis of the research instruments’ information presented in Chap. 3. Since many of the events and activities have overlapping learning contexts and objectives, they are illustrated initially in more general terms in the next section. Representative events will be described in more detail in subsequent sections.

Learning Context and Objectives

The first research instrument used in this case study is the seminar/workshop type of events. In terms of a learning context, these events comprised of an introductory presentation about the ROLE project, followed by the setting of a framework for the subsequent workshop element of the seminar, i.e. the use of widgets inside a PLE for learning purposes. All participants had access to a laptop or tablet computer in order to engage with the provided online learning materials.

The workshop element of the events then allowed participants to be introduced to a hands-on and guided use of a PLE. Participants used one of the two bespoke ROLE online courses, generally sharing the experiences in groups of two or more learners. Initially the presenter gave a short summary of workshop objectives but thereafter the individual groups explored the ROLE online course(s) for themselves, by following the self-study structured learning materials. The presenter(s) remained nearby in the room and circulated amongst the participants, answering questions as required. In the final section of the ROLE online courses, there was an opportunity to evaluate the experience and all participants were, therefore, invited to complete a short questionnaire.

The second type of informal learning activity involved presenting and, therefore, disseminating information directly about the ROLE project at a variety of international conferences. This activity took the form of a presentation followed by a question and answer session. Again, this gave the opportunity for the presenter to illustrate particular aspects of the ROLE project, highlighting individual and relevant PLE elements, as well as enabling them to indicate the existence of the bespoke ROLE online learning courses in the form of self-study interactive OER materials. Attendees were encouraged to visit both ROLE online courses post-event, in order to learn more about the project and to complete the short questionnaire related to their learning experience of using the bespoke OER materials (Mikroyannidis and Connolly 2013b).

Dissemination about the ROLE online courses also took the form of using posters. These were used as a visual medium to promote and signpost relevant information related to both the PLE aspect and also the availability of the structured self-study OER materials. Posters were displayed at a number of international conferences and proved to be popular talking points for those attending the events and, in effect, an unanticipated informal learning opportunity for enquirers. The posters gave a visual representation of the ROLE project and provided an opportunity for individuals to enquire and learn more about PLEs from the ROLE representative(s) who presented the poster. Further promotional materials in the form of bookmarks were also available, enabling visitors to “take away” some tangible and memorable signposts with further details about different aspects of PLEs.

A third learning context for this case study is a group described as “artifacts” that have provided bespoke opportunities for informal learning. This group includes the creation of both ROLE online courses. As indicated previously, these practical courses were often embedded into events that were focused on raising awareness about the PLEs. They were also available online for anyone to study at their own time. A further artifact that provided an innovative informal learning platform has been the adaptation of the ROLE online courses into an eBook. Initially, the content of the first ROLE online course was taken and adapted for presentation as an eBook. As a pioneering development itself the ROLE online course had been through a number of iterations and changes, ultimately being published with somewhat different emphasis in terms of content. Likewise, the ROLE eBook followed a similar pattern of development. Whilst the learning objectives for the eBook remained relatively similar to those in the original ROLE online course, i.e. introductory, the actual learning content of the published ROLE eBook was somewhat different in form. This was because the SRL online course materials were merged with the original ROLE online course content. The resulting interactive eBook thus contained the selected contents from both courses.

In summary, it can be said that the underlying curricula presented in each of the previously described themed informal learning events or activities has been, broadly speaking, very similar: i.e. an introduction to the ROLE project’s purpose and, in particular, a focus on the use of OER within a PLE to support informal learning. On occasion, specific topics were presented, as required, to the different Communities of Practice too. In addition, information relating to any necessary guidance for the associated workshop events was given where relevant. Furthermore, the specific development of the two ROLE online self-study courses also enabled either attending participants, or those directed to the courses at a later date, to study the materials at their own pace and, potentially, at a time or place of their choice. Finally, the repurposing of the online course contents as an eBook has enabled the dissemination of information about PLEs and by using associated OER materials has facilitated it to be widely distributed as well as used in many further, and unanticipated, informal learning contexts.

Setup and Organisation of Learning Activities

As previously described, the range of informal learning events offered by the OU has been divided into three broad dissemination categories: seminar and/or workshop events, conference presentations and thirdly the development of bespoke learning activities that have been tailored to meet the identified learning needs and objectives of particular Communities of Practice. Consequently, the associated setup and organisation of the related learning activities was also often custom-made to meet the needs of an event’s anticipated or identified audience. It is reasonable to state, however, that most of the embedded learning activities were developed from an original basic master set and then adapted and/or repurposed for each of the individual contexts thus actively implementing one of the fundamental themes of OER that of reuse (Hilton et al. 2010). The underlying premise of adaptation described here having been to identify an appropriate ROLE widget, then create a structured activity around it and subsequently produce a set of achievable self-study tasks related to the topic.

How that learning activity was embedded into the individual event or into the online environment varied according to a number of factors such as adhering to the adopted pedagogic model of the event; assessing the availability and suitability of technology for the learning activity, e.g. considering the appropriateness of the delivery platform, such as Moodle or the eBook. It was also necessary to reflect upon whether the learning activity could be evaluated at a later date using relevant and appropriate research instruments too. The mediation of the learning activity also had to be well thought out. The basic master set of learning activities had been originally developed for use as self-study materials, thus necessitating the additional “presence” of structured and carefully crafted feedback for the user to read, understand and so allowing learning to take place (Conole et al. 2011). Interestingly, in the many face-to-face events it was possible to offer supplementary feedback and answer participants’ queries instantly, as required, thus providing another unanticipated blended form of delivery. Mentors, as such, were not provided for any activities as such but, as previously described, there were several opportunities in the many seminars, workshops, conference presentations as well as alongside poster displays for presenters to give supplementary guidance and provide signposts to further details about the use of PLEs.

Learning Culture

The OER-based informal learning materials that were developed by the OU initially adhered to distance-learning pedagogic principles. This basic premise was later adapted to suit a blended model of delivery according to the circumstances of the individual events. The learning culture as such could, therefore, be described as promoting or adhering to one of self-study further described as attractive to those who have a high level of SRL (Zimmerman 1989). Additionally, in general terms of the types of people taking part in the events it can be reported that the majority of those participating, online as well as face-to-face, were staff from HE institutions, often the OU but not exclusively so. As previously described, participants were drawn from a number of different Communities of Practice, including academics, academic-related-staff, technicians as well as librarian colleagues.

PLE Intervention

The overall PLE intervention in this case study was essentially established from a basis of embedding structured learning activities that used selected ROLE widgets and then delivered them via a web interface. The chosen platform employed to display both online courses was the LabSpace³ area of OpenLearn, which uses Moodle for the delivery of materials and associated educational technology (McAndrew et al. 2009). The PLE in this illustration, thus, involved a Moodle environment as well as the use of the selected ROLE widgets that were embedded in the online courses, enabling participants to access a whole array of further OER materials in their informal learning context.

Table 2 lists the contents of the two ROLE online courses. The first course introduced the concepts and technologies of the project. It provided a combination of tools and services that enable learners to build their own PLE based on their needs and preferences. It also established a course template that was subsequently reused for the development of the second online course focusing on SRL. The template was based on an original OpenLearn study unit format that had been established in the LabSpace area of the website (Lane 2008b). Associated structured learning activities, using widgets, encouraged participants to explore and discover further OER materials or tools to enhance their knowledge.

Table 2 The ROLE and SRL online course tables of contents

The second online course, focusing on SRL, followed a similar pattern in terms of layout. It also has sections, learning outcomes, and embedded activities. Both courses were released under a Creative Commons licence, as indicated earlier, thus enabling their contents to be not only used or studied in situ but also to be “taken away” and, potentially, repurposed elsewhere within the terms of the licence. In other words, whilst the intended purpose of the two courses was to raise awareness about specific aspects of the ROLE project by introducing selected widgets in a PLE to access further OER resources, the design and deployment of the course materials as OER also meant that they could be reused by others too. Again this adheres to a fundamental premise of OER and that materials, tools and technology should be freely shared and accessible to a wide audience (Hilton et al. 2010).

OpenLearn as a PLE

The OpenLearn project has changed significantly since its launch in October 2006 (Lane and Law 2012). It was originally designed as an OER repository, using two websites: LearningSpace and LabSpace. It set out to offer a full range of HE academic subject materials ranging from the arts and history to science and nature, at all study levels from access to postgraduate. Commencing with 900 h of study materials in LearningSpace and 900 h in LabSpace, the now enhanced OpenLearn website currently offers in excess of 1,200 study hours. A number of changes have taken place since the launch, the most notable being the significant increase of available types and styles of study materials. In 2010, the OpenLearn brand also expanded in size and content to incorporate a significant number of both audio and video materials from the former Open2.net website—a platform that had been developed to support joint OU-BBC programmes designed to encourage public engagement with materials related to a variety of HE subjects. It has been reported that there have been 24 million unique visitors and approximately 320,000 registered OpenLearn visitors (Lane et al. 2013).

Through offering the original OER study materials, and then further developing the OpenLearn website to incorporate the Open2.net resources too, the OU has also endeavoured to add value to its Open Content by deploying leading edge Learning Management System (LMS) technologies for learner support. At the same time, using such an approach, it has also actively sought to encourage the creation of informal collaborative learning communities. Alongside these developments, the OU has also pursued the development of international research-based knowledge about modern pedagogies for HE (Sharples et al. 2012). These improvements have also presented the ROLE project with an excellent opportunity to gain access to a wide cross section of learners and educators who engage with OpenLearn OER materials.

The ROLE project has embraced OpenLearn, both in terms of building on its experience of creating OER study materials, as well as enabling access to some of its constituent informal learning communities. By using the LabSpace area of OpenLearn for the development of the two ROLE online courses it has also been possible to not only introduce the idea of a PLE but also offer direct access to selected ROLE widgets that permit end-users to create their own PLE. Whilst materials for the two courses were hosted on OpenLearn as self-study OER units they can, of course as previously indicated, be used in face-to-face settings too. Again, this enhances and improves the possibility of accessing a further variety of audiences that may have an informal learning context. OpenLearn also provided ROLE with the potential access to an unanticipated large global audience.

Evaluation Objectives and Instruments

This case study consisted of a number of events, activities and artifacts that enabled different approaches to be taken to explore the use of OER materials for informal learning. The overall aim of the research was to ascertain how PLEs could be implemented for different groups of educators in the HE sector. The first evaluation objective, therefore, was to determine which of the selected ROLE widgets were appropriate for the different audiences, as well as assess the impact of introducing the idea of PLEs to those who may not have been aware of these technologies. It was important to gauge whether people were receptive to the idea of PLEs in addition to trying to determine if individuals were prepared to adopt any of the widgets in their teaching delivery.

The basic research instrument deployed in the different events was the Perceived Usefulness and Ease of Use (PUEU) Survey, which has been based on the Technology Acceptance Model (TAM) (Venkatesh and Bala 2008; Venkatesh and Davis 2000). The questions in this survey were used in all the test beds of the ROLE project and were tailored thereafter for each event according to the composition of the surveyed groups. Necessary changes to the original survey were, in reality, minimal. The PUEU survey was available online,⁴ where all data and information was also collated. It is important to note that each of the ROLE test beds in fact used the PUEU survey thus allowing further analysis across the project to take place. As noted earlier, the PUEU survey was also included as an evaluation opportunity within both of the ROLE online courses. Again it contained the original questions slightly tailored for these informal Communities of Practice.

Where appropriate, additional research instruments were used. For instance, during the workshop elements of an event, the ROLE presenters were able to offer help to individuals or groups as required but in doing so could also observe, first-hand, any pertinent issues that arose. Whilst this was not a systematic collection of data, more so observational and unplanned, it did serve as an excellent opportunity to see how ROLE widgets, in particular, were received, understood and used. Observational notes were recorded. This view of participants’ engagement, or not, with a PLE also enabled the presenters to gauge the usability not only of the selected ROLE widgets but also participants’ interactivity with the two ROLE online courses. In some respects these observations could be described, therefore, as informal research instruments as previously indicated.

The presenters also could observe any perceived impact that the ROLE widgets had on an individual’s informal learning. This was, of course, more difficult to ascertain as an observer but was often reinforced when participants, on occasion, requested help repeatedly thus indicating that they were having difficulty in understanding some aspect of the course. Furthermore, observation could also be employed to ascertain the acceptance of the PLE by different HE groups. Again, whilst this was not a systematic collection of data or information, it did serve to supplement the recorded responses in the PUEU surveys.

Methodology, Evaluation and Participants

There are three themes for this case study (see Table 1). Firstly, the seminar and workshop group events. Secondly, the dissemination activities that involved presenting information directly about the ROLE project at a variety of international conferences. Finally, the third group contained a selection of mediating artifacts that provided opportunities for informal learning. Selected examples from each of these three groups will now be described in terms of research methodology.

Event 1: The eLC Seminar

This was an opportunity to introduce the ROLE project to the e-Learning Community (eLC) of the OU and took place in January 2012. The eLC has more than 300 members including those from both academic and related staff in the OU campus as well as potentially comprising of many Associate Lecturers of the OU who are based throughout the 4 Nations and 13 regions in the UK. The eLC offers a regular programme of workshops and seminars to OU staff, also available to invited visitors, and covering a wide range of innovative e-Learning-related educational technology projects.

The ROLE seminar presented an opportunity firstly to describe and then, in the workshop element, encourage the attending 20 eLC members to use the online ROLE courses. As previously mentioned, the self-study units include introductory text about the ROLE project and have various structured activities that enable the learner to use a selection of ROLE widgets. The ROLE widgets that the participants were invited to use are shown in Fig. 1 and are the following:

• Binocs: A widget used to search for OER in a number of Web 2.0 repositories, such as YouTube, SlideShare and Wikipedia.

• ObjectSpot: A widget employed for bibliographic searching in popular bibliographic indexes, such as DBLP and Google Scholar.

• EtherPad: A widget used for synchronous collaborative authoring of a document, where participants shared the OER they found in the other two widgets, as well as their experiences from using the widgets.

Fig. 1

A selection of ROLE widgets for finding and sharing OER

By using the online self-study ROLE courses in the workshop, the 20 participants were also able to gain further insights into how to build or adapt a PLE. This was achieved by completing the online courses, which included guided access to the ROLE Widget Store⁵ where further relevant, learning and teaching widgets are located. After the initial presentation defining an overview of the ROLE project, its aims and specifically outlining the objective of PLEs, eLC participants were invited to follow the structured activities in the ROLE online courses. During the structured activities, the ROLE presenters moved around the room offering help to participants as required and attempting to aid them with their engagement with the ROLE widgets on request. On the whole, this meant fielding a wide variety of questions, as well as offering constructive advice about how PLEs functioned and where to find out more information relating to this developing area. The ROLE presenters did not actively seek to intervene in the participants learning processes and only responded when requested to do so by individuals. In effect, their presence in the workshop part of the eLC event offered a blended learning opportunity to participants and similarly acted as an informal research instrument to observe participant interactions first-hand.

Event 2: “Build Your Own PLE” Workshop, JTEL Summer School

This event was held in Estoril, Portugal during May 2012. It was established originally by the EU-funded ProLearn project and subsequently supported by the European Association for Technology-Enhanced Learning (EATEL) along with other EU networks of excellence. PhD students in the area of Technology-Enhanced Learning (TEL) from across Europe spend the JTEL Summer School week learning about the latest trends in TEL, and exchanging ideas about their Ph.D. projects.

The “Build your own PLE” workshop was delivered during the JTEL event. Some 14 students attended the session and used a variety of ROLE widgets in order to find learning resources and start building their own PLE according to their research interests. Both the ROLE online courses were used in the workshop.

Other ROLE workshops were also held during the JTEL summer school and these included a coding session using the ROLE SDK (a development service that focused on communication and collaboration), a widget design session, as well as a session concentrating on personalised support for SRL. Thus the “Build your own PLE” event was one of a family of sessions focusing on the ROLE project. Once again, the “Build your own PLE” workshop enabled participants to use the widgets previously described in Event 1.

Event 3: “Build Your ROLE” Workshop, PLE Conference

The third annual PLE conference took place simultaneously in Aveiro, Portugal and Melbourne, Australia in July 2012. Researchers, educators and practitioners in TEL and PLE were brought together for a lively exchange of ideas, practices and visions. A number of ROLE partners, including the OU, delivered a half-day workshop entitled “Build your Responsive Open Learning Environment”.⁶ Participants were introduced to the ROLE tools and learning methodologies and were encouraged to use these tools in order to build their own Responsive Open Learning Environment. Additionally, they were able to design their desired tools, according to their learning scenarios and requirements, as well as submit the results to a subsequent ROLE widget competition.⁷ Once again, there was a focus on the two ROLE online courses within the workshop, along with an opportunity to complete the PUEU survey.

A paper was also presented at the PLE conference, concerning some of the emergent lessons learned from the OpenLearn test bed of the ROLE project. The presenter described the use of widget-based PLEs by informal learners who sought and discovered new OER materials as a result of using the ROLE widgets. The presentation took the form of a “speed-dating” style i.e. not a ubiquitous PowerPoint. A recording of this presentation and the slides were made available after the conference too (Mikroyannidis and Connolly 2012a).

Event 4: The Dev8eD Workshop 2012

Dev8eD is organised by the Developer Community Supporting Innovation (DevSCI), a community of developers in the learning provider sector. Thus the Dev8eD event, held in Birmingham, UK in May 2012, was designed for developers, educational technologists and users working throughout education, who wanted to further the development of tools, widgets, apps and other resources for education.

The ROLE workshop was attended by ten conference delegates. Participants were first introduced to the ROLE project through a brief presentation and then had the chance to use selected ROLE tools during organised group activities. The purpose of these activities was to enable participants to understand how a PLE can be used to support them in their everyday learning and research tasks. Additionally, participants had the opportunity to build their own PLE according to their own learning and research activities.

Event 5: International Workshop on Cloud Education Environments

A workshop was hosted by Galileo University in November 2012 in Antigua, Guatemala. It focused on the exchange of the relevant trends and research results, as well as the presentation of practical experiences gained while developing and testing cloud education environments, both from a teaching and a learning perspective. This workshop raised awareness about both the ROLE project and the function of PLEs in cloud-based environments. Once again, the two ROLE online courses were used initially to attract the new external stakeholders, as well as to underpin this workshop. The workshop was focused on a cloud education environment by examining how such informal OER materials and services can be distributed using a number of different publication channels.

Activity 1: The ROLE Poster and Other Publicity Materials

The Paris OER Declaration was formally adopted during the 2012 World OER Congress held at the UNESCO Headquarters in Paris in June 2012. Over 550 delegates including representatives of governments, educators, NGOs, and international universities attended the Congress, which was organised in full partnership with the Commonwealth of Learning (COL) and supported by the William and Flora Hewlett Foundation (USA).

A poster was displayed in the exhibition area, promoting the two ROLE online courses.⁸ The poster gave a visual focal point to which congress participants could attend and enquire about the ROLE project. It was an excellent opportunity to promote and disseminate information about ROLE to a wide range of international delegations. This event proved to be an excellent opportunity to promote both the project and PLE developments in a significant OER global gathering. In turn and, as recorded earlier, the poster also acted as a visual mediating artifact that enabled informal learning about the ROLE project to take place.

Activity 2: World Conference on E-Learning Paper

This is an international conference organised by the Association for the Advancement of Computing in Education (AACE) and co-sponsored by the International Journal on E-Learning. It was held in Montréal, Canada, during October 2012. The conference serves as a multidisciplinary forum for the exchange of information on research, development, and applications of all topics related to e-Learning in the corporate, government, healthcare, and HE sectors.

A paper was presented describing a number of the lessons learned as well as the best practices that were observed from the findings of the ROLE OU test bed in summer 2012 (Mikroyannidis and Connolly 2012b). A round table discussion, involving eight people, also took place hosted by the OU. It set out to explore the challenges associated with supporting SRL in HE.

Artifact 1: The ROLE SRL eBook

As previously mentioned, a fundamental aspect of OER is the ability to share and potentially, therefore, try to encourage the reuse of the developed materials (Hilton et al. 2010). It can also be argued that in doing this, it is possible to reach out, disseminate and make contact with new and, possibly, unanticipated audiences. The two ROLE online courses, for example, had been designed with this in mind: reuse and sharing potentially using multiple formats and platforms. The materials that were presented in the LabSpace area of OpenLearn were designed using structured content and XML. This enabled them to be transferable to other platforms as OpenLearn offers numerous export facilities, for example: Moodle backup, SCORM, and IMS package. OpenLearn also allows its structured authoring documents to be used as databases (Hirst 2012).

During 2012–2013, the OU has been exploring as well as taking advantage of new and innovative ways to widen participation in its courses and associated informal learning tools (Connolly 2013; Lane and Law 2012). Whilst there has been a focus on OpenLearn as a vehicle to achieve this, further platforms including Apple’s iBook Store and YouTube have also been used as opportunities for informal learning. The advent of the eBook has offered a new opportunity to harness not only existing structured content but also include levels of interactivity previously restricted to the LMS platform (Moodle). As a distance teaching institution, the OU has always endeavoured to extend the boundaries of publishing, as well as take advantage of educational technology to do so. Thus “rethinking” the publication of printed books as eBooks has offered the opportunity to not only alter models of production (i.e. from print to online to mobile) but also to “open out” and extend the fundamental and familiar idea of a book by creating new and exciting experiences for the readers.

With this in mind, the publication of the ROLE SRL eBook (Mikroyannidis et al. 2013) has taken place alongside a growing series of interactive concept publications produced by the OU.⁹ It has taken advantage of the HTML5 technology to produce an eBook that can include interactive ROLE widgets and other inline resources. As a consequence, the ROLE eBook provides an introduction to new learning technologies that empower the reader in terms of SRL and by providing access to information about as well as using PLEs. In effect, it is selected content from the previously published two ROLE online courses that makes this interactivity and raised awareness possible. A selection of learning tools has been included that will help an individual to build his/her own PLE and encourages him/her to become a self-regulated learner too. Readers have an opportunity to try these tools through a set of interactive learning activities included in this eBook.

The Evaluation’s Participants

As previously mentioned the primary research instrument for the majority of the described events and activities in this case study was the PUEU survey. Essentially, the survey was used to gain an understanding of how participants from different Communities of Practice have attended ROLE events and perceived the usefulness and ease of use for a number of selected and presented ROLE widgets in addition to capturing their experience of using the two ROLE online courses. As indicated earlier, a number of further observational notes have been collected. The latter will only be reported here to verify and support information collected from the PUEU survey.

The objectives for the overall evaluation, therefore, focused on participants’ understanding of the concept of PLEs, their use of specific widgets and capturing knowledge of their opinions and interactions with bespoke ROLE products such as the eBook. In this respect, the PUEU survey has proven to be quite useful as it embraced all of these objectives. The survey was easily accessible to all workshop participants and users of the ROLE online courses.

The numbers and profiles of the people who attended each of the OU-led events varied enormously according to location and timing. In summary, it can be reported that the groups ranged in size from 10 to 50. The majority of participants were staff from the HE sector although one group was PhD students. The attendees’ age range appeared to be between 21 and 50. In general terms most people who completed the survey had some experience of TEL although few had either practice or full understanding of the potential that a PLE could offer. It is reasonable to say that there was a fairly even spread in terms of gender across all events. Table 3 summarises the profiles of the participants in each event or activity.

Table 3 Brief profiles of those participating in selected events or activities

Evaluation Results

The evaluation results will not be described for each of the completed events and activities, but reported for the first group of events only. As previously mentioned, the main research instrument to be deployed for this case study was the PUEU survey. The majority of results of this survey has been recorded for the events and will, therefore, be presented here. A number of the observations made by the ROLE presenters will also be included. In addition, comments and quotes recorded by the participants via the EtherPad widget will also be presented where appropriate. Table 4 summarises the themed groupings of this case study.

Table 4 The groups used in this case study

The eLC Seminar

The respondents to the PUEU survey appeared to have an even split of knowledge amongst them in relation to TEL. Conversely, however, a significant 88 % of those participants felt that they had “some” rather than a “good” knowledge of PLEs. In other words it appeared that the group as a whole was relatively new to the idea of a PLE.

Observation of the interaction with the EtherPad widget revealed that some participants were hesitant to use this type of technology and required encouragement from their peers or more experienced colleagues (either those in the group or from either of the two ROLE representatives who were facilitating the workshop). The types of information recorded by the participants ranged from anxiety to amazement that such tools could enable individuals to learn collaboratively. There was also clear evidence, however, that a more experienced group member took advantage of the EtherPad widget to communicate with a colleague in another part of the room about some mutual work unrelated to the subject in hand (i.e. ROLE and PLEs). It should be noted, nonetheless, that the same person used the widget firstly to give advice to his colleague regarding more effective use of the tool before moving onto the separate non-workshop subject. What this also revealed, of course, is that despite encouragement to explore ROLE widgets it was not a compulsory activity and at least one participant chose to continue his own non-ROLE/PLE work as well as participating in the workshop.

It appeared that the EtherPad widget was used more constructively to exchange as well as record pertinent PLE-related information. For example, one participant detailed a blog address that they felt to be “an interesting take on PLEs. Just a tad off topic… (Note to self this is the wiki)”. Others noted comments relating to enquiries about how each ROLE widget functioned. These ranged from “how are keywords supposed to work” to “cannot get ObjectSpot to show on iPad2”. Indeed this idea of recording questions was taken somewhat further by one participant who remarked: “Why do we need two widgets for search?”

The final question in the survey asked participants to record their feelings about their use of PLEs. There was a 77 % agreement that PLEs would be slightly useful for participants work, followed by a slight disagreement that the same PLEs would help participants accomplish their work more effectively than their current use of learning technology. Again this was not surprising as most group members were established and experienced users of learning technology and had revealed that they only had limited use of PLEs. Half the group proffered a neutral response to the statement relating to “It would be easy for me to use a PLE” whilst the remainder recorded that there was a slight chance that that would be the case. There was a more even spread of responses to the statement “It would be clear to me how to assemble a PLE using widgets” ranging from slight disagreement (11 %) through to slight agreement (also 11 %). Most participants remained neutral on the subject.

Interestingly, the statement “I would find using a PLE frustrating” invited the most disagreement to be recorded with the majority (55 %) remaining neutral alongside 33 % saying they slightly disagreed and 11 % strongly disagreeing. Once again, the statement “I would find interacting with a PLE requires a lot of mental effort” statement invited a strong neutrality (55 %) yet 22 % of participants strongly disagreed with this premise whilst 11 % recorded that they slightly agreed that this would be the case for them.

The remaining part of the survey related to participants’ motivation to using a PLE in their learning process whereby 55 % remained neutral in their responses and 44 % slightly agreeing with this statement. The last statement of “I predict that I would frequently use a PLE if I had access to it” invited an even response (33 %) between slightly disagreeing through neutral to slightly agreeing.

The JTEL Summer School Workshops

A ROLE workshop took place in both the 2011 and 2012 JTEL Summer Schools. The participating research students were enthusiastic and willing to try out the widgets within the ROLE online courses. Each workshop had the same format of an introduction to the ROLE project, followed by the practical activity of using the widgets within the ROLE online courses. In general terms the research students’ overall opinion in both workshops was a positive one. They engaged with the structured activities, actively used all of the provided tools as well as recording their thoughts (and sometimes frustrations) in the EtherPad widget.

The EtherPad widget was used in many different ways in the 2011 workshop. Most of the research students used the tool although some were a little surprised by the real-time aspect of it: “…somebody is writing on the screen!!!! I am scared…” Others considered additional aspects to the experience in that it highlighted some potential gaps in their own skill set: “…I probably have to work on my search skills…” By contrast in the 2012 workshop, however, the EtherPad widget was used actively by only a few of the participants. Generally, it functioned as a means to record and exchange URLs of relevant resources such as the “Learn Portuguese language vocabulary” YouTube video¹⁰ that the participant described as: “This is a great video”. It was also used in identifying a Stephen Downes Slideshare presentation about “Personal Learning The Web 2.0 Way”¹¹

All participants in both workshops were aged between 20 and 40 years. There was also an equal 50 % male/female split. In answer to the question relating to the participants’ knowledge of TEL, in the first workshop there was a significantly higher response rate to the “some” option whilst in both workshops most stated that they had a “good” knowledge of TEL. In the second workshop with regard to the question related to PLEs, however, there was a greater spread of responses: 50 % recorded that they had a good knowledge whilst 25 % stated a “good” knowledge and the remaining 25 % claimed to be an expert in the field of PLEs. The free-text responses within the questionnaire provided some insight into the participants’ views of PLEs as well as the use of the ROLE widgets.

The question: “What did you think of the widgets of the workshop activities?” also invoked a variety of responses in both workshops, most acknowledging that the widgets were interesting. In addition others said: “I have found them very useful for my PhD and for my learning” along with a remark that (the widgets were): “Small apps which can expand your daily routine”. Others simply said that the widgets and interface: “look good”. It is also important to note a word of caution too though, summarised by one respondent who remarked: “Found them quite interesting. Collaborative text editor had lower quality in contrast to GDoc, surely I would not use it”.

In relation to the question: “Were you able to find suitable widgets for building your PLE during the second workshop activity?” there was almost unanimous response in the second workshop alluding to the intermittent internet issues that appeared to plague the entire session. One quote, perhaps, sums up the frustration that most participants felt dominated, their experiences: “Internet connection issues did not let us perform this activity”. Nonetheless, 90 % of the same participants responded positively to the next question: “Did you find the workshop activities useful for your research needs and goals?” which can be summarised by one remark: “I found it an interesting approach to be tested in the future”. One respondent did, however, offer a rather more circumspect response: “I do not know. I have to check those pages more when I come home”.

Opinions from both the workshops’ attendees about widget-based PLEs were, in general terms, evenly spread. One notable exception, however, was the response from workshop 2: “I would find interacting with a PLE requires a lot of my mental effort”. Nonetheless, the respondents overwhelmingly recorded that they were neutral in their feelings about this statement for the second workshop. Research students in the first workshop appeared to be more discerning in their learning and, as noted by the ROLE presenter, the majority of the students appeared to focus on the Binocs widget rather than the ObjectSpot widget. Again this seemed to colour their view of the overall experience of using the ROLE technology.

The Dev8eD 2012 Workshop

This workshop was attended by developers and learning technologists, predominantly male and in the age range of 20–50. Most participants recorded that they had a good knowledge of PLEs but some also declared little or no knowledge of this area. Once again the EtherPad was used during the event to record notes and information relating to the workshop that participants wanted to share with each other. In this event, however, the participants took a more strategic view of the EtherPad by using it to store personal observations such as:

I've used Etherpad before. The problem with these synchronous writing tools is the way the connection suddenly stumbles and your flow is disturbed—Just had this problem with Etherpad. It was static for a while so I assumed people were still having connection problems—then I clicked to type and got a huge update!

In addition others used the EtherPad to record their thoughts about the activities themselves as well as how the widgets worked or performed, for example:

Some searches return “60 results” apparently—all of them YouTube videos—I untick YouTube as an option and get 54 results—but there were way more than 6 YT videos in my previous list.

The term ‘reflection’ isn’t especially useful for search… Binocs’ first result may be the kind of ‘reflection’ I’m looking for but ObjectSpot results cover a range of different types of ‘reflection’.

At the end of the workshop, participants were encouraged to complete the PUEU survey. Again this gave them an opportunity to record their thoughts about both the ROLE widgets, the implementation of them in PLEs as well as the activities of the workshop itself. Responses for the question: “What did you think of the widgets of the workshop activities?” were mainly positive but with reservations about the mechanisms that were used to make the widgets function. Participants were constructive in their observations saying, for example:

Could be useful, though a few flaws here and there. Binocs had a odd way of searching and filtering. Etherpad is a great idea but it didn’t always sync correctly and would jump-start again when clicked. The Mash-Up Recommender is great but not all widgets were installable to iGoogle!

There was also positive affirmation that the workshop enabled the participants to be introduced to new widgets, for instance: “The widget can be useful to put different tools together” and: “Useful because I had never heard of or seen these widgets”. Recognition, however, was also given to the technical issues such as: “Some problems with Etherpad on the iGoogle page—would be better if it sized down. Also the Binocs broke”.

The question: “Were you able to find learning resources that relate to your topic(s) of interest during the first workshop activity?” also invited a variety of responses. Some participants: “Found some things but would need more time to explore—will do that soon” whilst others were circumspect: “yes, though would like to understand better why two separate search boxes. I’m guessing one is API driven, one is custom Google search? could they be combined?”. Remaining responses affirmed that the participants were, generally, happy using the ROLE widgets, for example: “Yes, I searched for 6lowpan and found some relevant videos and slides” and “I tried one topic relevant to my institution. The resources were good”.

There was also overall positive response to the question: “Were you able to find suitable widgets for building your PLE during the second workshop activity?” once again, however, the participants did not hesitate to record their actual experiences of installing widgets or not, for example: “The ones I did find looked useful but didn’t add to iGoogle (there was an error)”. Another participant also remarked: “I think so, but I’d need to think about how I’d integrate it with my other tools (like Evernote, Twitter, Google calendar)” demonstrating that as a developer or educational technologist that they were giving some thought to the application as well as implementation of the ROLE widgets.

There was an overwhelming positive response to the question: “Did you find the workshop activities useful for your research/teaching/learning needs and goals?” once again summed up by: “Yes, a useful overview/primer of what’s possible” as well as: “Yes, Very useful”. Similarly the final question of the survey offered participants the opportunity to add any remaining questions, comments or suggestions that they wished to record. Some insightful comments were made such as:

I still have lots to learn about this area but this was an informative session to get me started!” as well as: “I can’t help thinking that if I have the digital literacy skills and confidence to create a PLE then I don’t need a PLE. (a bit of a paradox!)

Lessons Learned

From earlier, interim, research about this case study (Mikroyannidis and Connolly 2013a) three main themes were identified:

• The usability of the learning tools, i.e. the widgets.

• A consideration of the types and styles of related learning activity formats (often embedded in the ROLE online courses).

• A reflection as well as action upon suitable methods that might encourage future participants to consider, engage and continue using PLEs for their own learning purposes.

In respect of the additional events, activities and artifacts described here it can be reported that a number of further lessons have been learned. These will be considered in the context of successes and failures. The underlying lesson learned was the importance of ensuring that all the technologies are stable, available and accessible at the time of engagement as this leads to successful deployment. Additionally, planning an event, activity or even an artifact should also include a level of adaptation or localisation for particular audiences e.g. PhD students have different requirements to the more experienced researchers.

The creation of the first ROLE online course alongside the development of the second ROLE online course focused on SRL were invaluable in the success of all the components described in this case study. Each course had been structured in a similar manner (using the same template) and thus contained learning outcomes, defined learning activity opportunities and clear signposts to relevant ROLE widgets. Thus the use of such structured content made its repurposing as an eBook much easier to carry out. Consequently, it was possible to build on the success of both ROLE online courses by raising awareness about PLEs and SRL through an additional informal based educational channel: the interactive eBook was developed. Similarly the eBook reused the previously developed course template by providing an introduction to PLEs and SRL as well as giving an opportunity to readers to try a selection of ROLE widgets through a set of embedded interactive learning activities.

Successes and Failures

The level of success for this case study can be measured in a number of different ways. Both qualitative and quantitative data has already been presented that outline both the positive and negative impacts that were observed by ROLE presenters or recorded by participants in the PUEU survey. Most participants were willing and able to take part in the ROLE workshops although some were hesitant to use this type of technology. Those people sometimes required encouragement from their peers or more experienced colleagues in order that they make progress. Most of the free-text responses recorded in the PUEU survey ranged across the possible spectrum of experiences from anxiety to amazement in terms of the potential use of such widgets to enable collaborative learning for example. It is fair to say, with this supporting evidence, that the case study has been successful in terms of raising awareness of PLEs to a cross section of HE staff and groups of research students.

It would be unfair to state that there were direct failures in this case study. Nevertheless, it should be noted that when technology was intermittent in terms of access this had a dire influence on the experiences gained by those attempting to complete the workshop activities. Frustrations with widgets not fully working or a simple breakdown in internet access had a very negative impact on all participants but in particular proved to be major stumbling blocks for those who were less confident or competent with the PLE or individual widgets.

Best Practices for PLE Adoption in Informal Learning

As a result of recording the successes and reflecting on some of the perceived failures in this case study, it is possible to list a set of best practices for the adoption of PLEs by informal learners:

• Accessible and easy-to-use tools: Best practice in this case study indicates quite clearly that a simple format for the ROLE tools is required that enables a range of learners to use them effectively and efficiently.

• Multi-format introductory and guidance learning course materials: Learners are in need of guided learning materials that will help them understand the value of the new technologies. Best practice in this respect included the development of the ROLE online courses and the ROLE eBook.

• Tailor tools to meet the needs of specific subject audiences: Best practice here should be to enable adjustment or even design for learners studying particular subjects or, alternatively, educators researching a wide range of topics to be implemented as required.

• Tools that harvest information from appropriate repositories/platforms: A set of generic search widgets were fully tested in this case study thus enabling use across a variety of learning contexts. As a consequence, some ROLE widgets (e.g. Binocs and ObjectSpot) have been successfully used and repurposed in other test beds of the ROLE project.

• Fostering a culture with a community willing to engage with new innovative technology: In order to maximise the adoption of PLEs, a suitable culture towards new technologies needs to be fostered. This case study demonstrates that the teacher or trainer may also need to adapt their own approach in order that they too are receptive to change.

Conclusions

The different components of this case study have enabled us to extend our understanding of the potential impact of the ROLE technologies within a wide variety of informal learning contexts and scenarios. This case study has numerous rich contexts in which there is potential for significant impacts of both PLEs and SRL. The challenges faced and lessons learned in each component of this case study have also been reported here. As with every new technology, some resistance was expected and initially faced in most cases by the participants be they educators or managers in HE institutions or likewise e-Learning practitioners elsewhere. Overwhelmingly, however, most of the ROLE tools were positively received by those who participated in the events, activities, or used the artifacts. Each event, activity or artifact that has been explored has also involved a cross section of representative individuals. This approach has enabled the ROLE project as a whole to collect experiences covering a large variety of learning contexts and requirements.