Keywords

1 Introduction

The National Research Council of Canada’s (NRC) Learning and Performance Support Systems (LPSS) program is developing a personal learning and performance support platform that crosses organizational boundaries to create a collaborative learning network. The LPSS platform provides individuals with access to their own learning resources and their learning records, allowing for job preparation and continued learning. LPSS is built around the Personal Learning Record (PLR) which contains the user’s learning records. The personalization of the LPSS is achieved via the Personal Learning Assistant (PLA).

This paper is based on the findings of a literature review on Personal Learning Environments (PLEs) including current learning needs analysis, and on the results of LPSS user feedback studies. Selected articles were reviewed by researchers in order to assess the state of the art functionality of existing PLEs, where the users personally customize a framework to enhance their learning. The articles on Personal Learning Assistants (PLAs) that involve the discovery, annotation and access to learning resources within PLEs were also reviewed in order to validate future work on LPSS. In the paper, we summarize the role of personalization in PLEs and provide a brief overview of the LPSS, including user feedback and evaluations related to personalization. Future work on LPSS platform and on the Personal Learning Assistant is also discussed in the conclusion.

2 User Learning Needs

Learning needs have changed over time due to the proliferation of instant communication and access to information [1]. Increased connectivity has led to a blurring of both work and personal lives, especially with younger employees, who turn to social media for professional and personal communication [2]. As a result of this change we need to rethink how we learn; from traditional models of learning to digital education through personalized learning paths [1] which include a wide variety of collaboration tools for communication, team collaboration, writing/editing and engaging/networking [2]. There is a need to connect and support people with a broad range of technologies, processes and content as well as a need for on demand learning that doesn’t disrupt workflow and productivity [3].

In traditional e-learning content, the learning units and the order in which they are taken is decided by the provider. Learning management systems are course-centric. Current trends in learning include: learners moving into different and potentially unrelated fields over their lifetime; the rise of informal learning; lifelong learning – learning as a continual process; and know-how being replaced with know-where as a result of exponentially growing knowledge [4]. Trends in learning and communication point towards the need for personal learning environments (PLE) where the learners choose the content, their learning path and work at their own pace [5, 6].

PLE is a learner-centric ideology rather than a software application [6]. A PLE system is: decentralized; places the individual at the center; grants user access to LMS systems as well as other sources of information; allows for direct connections to collaborative environments separate from LMS; and connects individuals with LMSs (which are centralized). Within the PLE the individuals’ data is theirs to manage and share as they see fit and the data goes with them as they change jobs or training programs [7].

The literature on personalization shows a need to move beyond traditional systems, as well as a shift towards lifelong learning that is not just personalized but also personal: this is why LPSS was designed. LPSS is a single point of access for development and training needs, and ultimately, career development and enhancement. The main objective of the NRC’s LPSS program is to design, deploy, refine and commercialize an online system for improving people’s learning and work performance [8]; however, LPSS goes beyond the limitations of traditional systems. LPSS emphasizes an individualized learning path with context-aware support. The platform places emphasis on personal learning through self-guided learning (including the use of simulations and simulation data), and will use social networks to create personal knowledge and learning through a variety of different communication systems [9].

3 Personalization

In the context of learning, personalization is an evolution of the concepts of differentiation and individualization. There is no single shared definition of the meaning of personalization; we define it as systems and methods that incorporate technology to differentiate resources and processes, based on each learners’ skills, interests, needs and learning profile in order to accelerate and deepen learning [10, 11]. The next step after personalization learning is personal learning, which implies a custom-built learning system for each user [12].

The need for personalization is not something new; in the paper Learning for Mastery, the author writes “the basic task in education is to find strategies which will take individual differences into consideration but in such a way as to promote the fullest development of the individual” [13]. Personalization is important because online learning reach a much wider variety of users than traditional environments and also because users work with the system individually [14]. Personalized learning systems can enhance learning effectiveness [14, 15] and help learners to feel more motivated [16].

We identify two main categories of research on personalization and learning. One category focuses on structured learning in a specific context, like the Protus system designed to help to learn specific programming languages which work with a predefined set of learning contents [17]. The other category focuses on Personal Learning Assistants (PLA) or other Personal Learning Environments (PLE), some examples are the TRAILER where the main objective is to help learners manage competences acquired through informal learning [18] and the LPSS project [8] shown in this paper. These two categories are not exclusive, as more structured and formal learning can also be provided through PLA or PLE; and both also have some common points, like the type of variables that are considered to personalize the learning experience (learning style, performance level, etc.). Personalization within PLEs can be achieved in different degrees and in different ways depending on the nature of the system, the data gathered or its purpose. The main concepts that are related to personalization in PLE are described below.

Recommendation systems are a main point of research in relation to personalization, and different variables are used to leverage the recommendations: learning styles, performance, learner’s activities, browsing behaviors, learner’s interests or social connections among others. Some research points to the value of considering context (computing, location, time, physical conditions, resources, user and social relations) to “better predict and anticipate the needs of users, and act more efficiently in response to their behavior” [19]. Recommender systems are a crucial part of personalizing learning as they aim to provide suitable resources to learners. The output of the recommenders is usually in form of a list of resources (content, user actions, people) and in some cases the learner can give feedback about the usefulness of the recommendations proposed in order to improve future recommendations [14].

Collaboration is used as a mechanism for personalizing the learning experience through different techniques. For example, a collaborative voting approach is used to find the average difficulty of specific course contents [20] or a combination of social network data and learner’s learning context is used in E-SoRS to provide the best connections to solve questions emerged while researching or studying [14]. Collaboration is strongly related to the concepts of social connections and community [19].

The importance of defining the user model is emphasized in the research on personalization. The model could be inferred from behavioral data, as found in eTeacher [21], or extracted from tests and surveys answered by the learner, like in Protus [17]. Learning styles are considered one of the most important factors influencing e-learning and personal academic competence [16]. To personalize learning, one needs to personalize learning objects, learning activities and learning methods, which are all interconnected with the learning style of the learner [22].

In PLEs there is a shift in control which comes from the move from institutionalized learning environments to learning spaces centered on the user. In this new space, users control the process of learning and create their own environment by searching, aggregating, creating and sharing contents [23]. Research indicates that learning styles can vary depending on the task or the learning content so the learner needs to be able to change this information at any time [17]. To personalize the experience, the learner should have control over the information and organization of the system.

Strongly related to the shift in user control, there is also the adaptation aspect, which refers to the ability of the system to learn from its own use. The user controls the system when he actively modifies it, but to empower the adaptation, the system needs to use behavioral data, activities, paths, explicit feedback or other data collected for the improvement of the system. For example, the system uses the feedback provided by the user in order to suggest future actions [21].

The use of affective information is also a part of the research on personalization [19]. Influence of emotions on learning and how they can be used to improve learning experience [24], and the importance of using strategies to motivate learners, like providing the opportunity for a second formative assessment [25] are relevant for this subject.

4 LPSS and Personalization

Since the launch of the pre-release in 2014, the National Research Council of Canada (NRC) is continually developing the LPSS platform, a system that shifts the learning perspective from classroom learning to personal learning. LPSS creates a personalized, dynamic and ongoing learning environment with access to enhanced learning resources on demand. This learning platform goes beyond personalized learning towards personal learning. Rather than offering a customized version of a generic offering, the goal of LPSS is to enable each learner to develop their own custom learning program from the ground up [26]. The LPSS pre-release version that was launched on October 21, 2014 (www.lpss.me):

provides individual learners with the tools and support necessary to access learning from any number of providers – not just educational institutions, but also their friends and mentors, their current and future employers, community and social programs, and much more. Built on current and evolving learning technology standards, it is designed to provide access to MOOCs, to traditional learning management systems, to stand-alone courses and software, and even to the world of the Internet of things [26].

Within the LPSS platform, a personal instance is created for each learner, designed to keep track of everything related to learning – exercises followed, tests taken, games and simulations attempted, web pages accessed, etc. LPSS stores all that in a single location within the Personal Learning record (PLR). Unlike a learning management system, LPSS plans to combine data from the user’s learning environment, the work environment and the social environment, thus enabling adaptive learning software to close the loop between learning and performance, and capture both formal and informal learning. PLR manages user’s learning and training records, credentials and badges over a lifetime, making it easier for employers to identify qualified candidates and for prospective employees to identify skills gaps.

The LPSS platform aims at enabling features such as an automated recognition of competencies and analysis of workflow and job skills, as well as contextual assistance and access to references and learning materials on demand from any device. In addition, LPSS aims to improve the personal learning experience by providing more insight into user’s learning patterns [27]. A snapshot of a personal LPSS dashboard is provided in Fig. 1.

Fig. 1.
figure 1

Personal LPSS dashboard

Full size image

In the process of designing the LPSS platform, the NRC team conducted an in-depth literature review on the personalization of adaptive learning systems [8]. This study found the analysis of user characteristics to be an essential part of an adaptive system development process and identified features that have been successfully employed in many adaptive systems. Individual user characteristics are crucial to consider in designing features that are relevant for the system’s adaptation, for advancing personal learning experiences beyond the one-size-fits-all approach and to ensure that user individual characteristics are taken into account to accomplish quality learning with proper performance support mechanisms in place.

Since LPSS is a networked personal learning platform, user studies involving collaborative learning spaces are relevant as well. For example, Höver et al. [28] looked at various ways to link, share and filter learning resources; provide visualization; archive resources and provide detailed levels of anchors for linking learning resources, including the need for referencing paragraphs, sentences, words, timestamps, and annotations. Other researchers [29] have also looked at linking course data to allow for different recommendations for users with different educational backgrounds. This is of particular relevance for the design of the LPSS personal recommender and a toolkit, both designed to provide enhanced access to resources, activities and credentials in a dynamic personal learning environment.

Since the launch of the LPSS pre-release, the research team has conducted various user studies to collect information on system use, perceived usefulness, perceived ease of use, and user satisfaction with the various tools and functionalities available within LPSS. The parts of these studies relevant to personalization are described in the next section of the paper.

5 LPSS User Feedback and Evaluations

In order to gather user feedback and evaluate the LPSS website, NRC researchers used three methods. Firstly, a feedback email address was provided to the users of the website to allow them to send their comments and suggestions. Secondly, surveys were conducted with the users. Finally, an internal review of the lpss.me website was conducted in the form of a usability study. This section provides details on the methods used to obtain user feedback, followed by a discussion of personalization feedback and recommendations from the users themselves.

5.1 Feedback Email

Since the launch of lpss.me, users have been able to communicate with NRC researchers via a feedback email provided via a link at end of each lpss.me webpage. Overall, 34 feedback emails were received since the launch of the system. These emails can be divided in three categories: (1) promotion, which included emails that promoted the user’s own technical skills or the skills of the company they work for; (2) improvements suggestions for the LPSS research team on what the system should do; and (3) bug reporting, where the users email LPSS researchers when a feature isn’t operating as it should. The second category of feedback emails (improvement suggestions) yielded several user comments related to the personalization of the LPSS system, and will be discussed in the results section.

5.2 Survey

An invitation to participate in a survey was sent to 232 people who registered into the LPSS platform. Between November 2014 and December 2015, 49 users responded to the survey. The majority of respondents were male, from between 40 to 69 years old, involved in online learning and highly educated. As a result, the findings from the survey cannot be seen as representative of the general population, but are sample viewpoints from power users within the online learning community.

A total of 24 questions were asked, ranging from demographics questions (e.g. age, gender, education, and familiarity with online learning, etc.) to more specific questions about the LPSS system. Questions that elicited responses concerning personalization of the system included the open ended text question: “What would you like a learning and performance management system (like LPSS) to do for you? How would you like to use it?”

5.3 Internal Review

An internal review of the website was conducted during the fall of 2015. This review was designed as a usability study of the LPSS platform conducted with NRC employees. It is the first step before conducting an external review of the platform.

When doing usability studies, the number of participants matter and the maximum benefit/cost ratio comes from testing four users [30]. Therefore, four persons (3 men and 1 woman) were recruited to participate in the internal review. They all had graduate degrees and were employees of the NRC involved in the LPSS program.

Participants were asked to complete eight tasks within the LPSS platform while thinking aloud in the presence of a researcher. The tasks were:

  1. 1.

    Log into the website, scan the information provided and report on what you think it is offering and how you can benefit from it.

  2. 2.

    Go to the dashboard and look at it. Report on what you think is the purpose of the different sections that you see.

  3. 3.

    Think about a topic/domain/skill you would like to improve and add it to your competency profile.

  4. 4.

    Without looking at the system, report where you think you could find relevant information about this topic.

  5. 5.

    Proceed to one credential you have in the learning record section and convert it to a pdf.

  6. 6.

    Log into the system, access the dashboard and explain what you see. How do you think it can be used? Would you like more/less or different information?

  7. 7.

    How would you filter the resources by type or keyword?

  8. 8.

    Now that you know more about the system, could you explain in your own words what you think it does? Would you use it? Why?

The internal review was conducted on regular laptops running the Morae 3.3.3 usability testing software. The screen display, mouse clicks and the webcam recordings (user’s face and voice) were recorded for later review and analysis.

Before the study the users were given a short introduction to the study and its methodology. They were then provided with a computer and asked to perform eight tasks using the think aloud protocol. For some of the tasks, test accounts were created to avoid spending time on the sign up process that was not in the scope of the test.

5.4 Results

Some useful feedback on the platform was collected using the above mentioned methods. Given the focus of this paper, we are reporting only the results that relate to the personalization of the LPSS site.

Feedback emails.

One user gave the following advice for improving the recommendation system: “It may be convenient to allow for the rating of resources in terms of relevance to the user. That may make the recommendations more intelligent. For example, I like the recommendations by the aggregation website http://www.elearninglearning.com/.”

Survey.

Respondents were asked what they would like LPSS to do for them. The answers related to personalization were:

  • Provide a way to network around competencies; learn from what others interested in similar competencies are studying.

  • Highlight things of interest based on personal profile and activities.

  • It could be interesting to browse competencies with the users profile associated, to optimize the network effect.

Internal review.

One issue reported was that the user expected his competencies level would be determined by a third party organization, instead of having the user manually adding his competencies.

Another reported issue was that users would appreciate an increase in the number of options to filter and narrow their searches.

A third reported issue was that when you create a credential in the Learning Record section, the system should automatically add your name, since the LPSS.me account is personal.

A fourth reported issue was that when exporting his credentials to a pdf, a user would have liked to have the option to tailor what is shown in the pdf, by selecting specific credentials.

Beyond the need for additional tailoring, the users revealed via email, survey and internal review responses the importance of social learning and the usefulness of rated resources. Users wanted to see what others with similar competencies and interests are learning in order to further their own learning. Also users wanted to see resources rated, both automatically and by other users who have used the resources, as well as easier filtering in the system.

6 Conclusion

This paper presented research and development underway at the National Research Council of Canada to develop personalized learning technology for learning and performance support. The paper provides an overview of the importance of personalization in online learning and in the context of LPSS, demonstrates that personalization can be applied in different ways within the learning platform.

The paper demonstrates the importance of understanding end-user needs, expectations and doubts when gathering personalization requirements, which can be accomplished through different methods like feedback emails, surveys and internal reviews. In the initial phase of LPSS development, considering the user feedback received, the main personalization findings are related to the social aspect of competencies (exploring what others in the same context are doing), better filtering capabilities and more options in relation to resources or user searches.

Future work on the LPSS platform will continue its focus on personalized learning by strengthening the social network aspects of the platform, further allowing learners to create and share information. External reviews will also be conducted to get user feedback from external users. User interactions will be captured in order to allow data analytics on them. Finally, researchers working on LPSS will continue exploring the challenges and opportunities for creating a truly personal learning environment, customized for and created by each user.