Introduction

Traditional lecture style is a common teaching approach in higher education classes; however, the traditional lecture style of teaching can often place students in a passive role, which typically involves students retaining isolated facts that can later be forgotten. Following Bligh (2000) definition, traditional lecture style is a “continuous exposition by the teacher”; in a traditional lecture instruction students’ activity is limited to taking notes and asking questions to the instructor. Over the last few years, instructors have been moving away from the traditional lecture style by implementing more active learning practices, like project-based learning and flipped classroom, and increasing the technology use as a way to extend and enhance students’ understanding.

Active and problem-based learning activities are founded upon a constructivist theory and traditional lecturing derived from direct instruction methods is founded upon behaviorist ideology. Active learning is a model of instruction that focuses the responsibility of learning on learners. It was popularized in the 1990s by its appearance on the Association for the Study of Higher Education (ASHE) report (Bonwell & Eison 1991). During the last years, active learning practices like project-based learning and flipped classroom have gained prominence worldwide, however sometimes we face a lack of consensus on what exactly active learning is. A working definition of active learning, derived from collecting opinions from 338 experts is the following: “Active learning engages students in the process of learning through activities and/or discussion in class, as opposed to passively listening to an expert. It emphasizes higher-order thinking and often involves group work.” (Freeman et al., 2014). Although this definition is quite generic, it perfectly portraits the rationale of active learning, without being restrictive. Active learning has been deployed in a number of education studies, however a framework describing the learning dynamics is typically not described; the development of a learning ecosystem framework to support active learning, will allow us to better understand and further develop teaching approaches enhancing students’ dynamics and needs.

A learning ecosystem has been described as a means for orchestrating a variety of learning approaches given by the varied characteristics of learning processes (Siemens, 2003). Learning ecosystems have seen as environments which are “consistent with (not antagonistic to) how learners learn.” (Giannakos et al., 2016), focusing on the learning process and take into account learners’ characteristics, needs and the potential dynamics and interactions with different actors (students, educators), as well as the learning environment and resources. Thus, the concept of learning ecosystem provides an ideal ground to orchestrate multiple tools and practices in the best possible way (Dillenbourg & Jermann, 2010).

Today, diverse technologies have been applied in a variety of active learning practices. However, isolated use of different technologies does not offer an overview of the different learning dynamics. Developing a learning ecosystem framework, which will allow us to describe “the complex of living organisms” as well as their interrelationships, will help us to better understand and further develop our teaching approaches. In this work we work towards this direction, by the following twofold contribution, first we present a conceptual framework of a learning ecosystem which can host active learning instruction and second we provide some first analytics-based evidence regarding its effectiveness and acceptability.

Building upon existing technologies and practices like video-assignments, clickers and micro-project approach, in the next section we propose a learning ecosystem for active learning. In the third section we present an empirical study following the proposed approach, were by collecting diverse-sourced data, we portray students’ experience throughout the course of the semester. The last section of the paper draws conclusions and discusses ideas for further research in the area. This study aims to provide a springboard for other scholars and practitioners to further examine the efficacy of this specific blended learning approach. Our conceptual framework is a flexible procedure that can be utilized and adapted to meet different needs.

Learning ecosystem for active learning

A traditional ecosystem has been described as “the complex of living organisms, their physical environment, and all their interrelationships in a particular unit of space” (Encyclopedia (2011)). By applying this simple and good working definition to learning; we can describe a learning ecosystem “as the complex of living organisms in a learning environment (e.g. students, educators, resources), and all their interrelationships in a particular unit of space (can be digital or physical)” (Giannakos et al., 2016). In a learning ecosystem it is important to consider the interrelationships of the main actors (students and educators) but also the role of the learning space (both digital and physical). The learning space is by analogy the physical environment in a traditional ecosystem, includes (organisms) information and digital resources like slides, lecture recordings, blog entries and forum discussions; but also physical materials like books, notes and handicrafts, to mention some. The space is where teaching or learning is happening and where such processes and interrelationships are conducted. The interrelationships exist (Shum & Ferguson, 2012; Bernard et al., 2009; Chang & Guetl, 2007) between the main actors (students and educators), the main actors with the resources (content), and the resources themselves (e.g. recommender systems) (Anderson, 2003), see Fig. 1. Those interrelationships shape the quality and value of students’ learning experience and our ability to develop teaching approaches enhancing students’ dynamics and needs is critical towards the emerging ubiquitous learning era.

Fig. 1
figure 1

Different types of interrelationships (interaction) in digital and physical learning spaces (adapted from, Anderson, 2003)

Full size image

Active learning as defined by Freeman et al. (2014) is broad and fits many classroom activities. Active learning acts as a superset for both peer-assisted, problem-based as well as flipped learning approaches. Prince (2004) clarifies the relationship between peer-assisted and problem-based learning, indicating that problem-based learning is, “always active and usually (but not necessarily) collaborative or cooperative.” The flipped classroom is also always active and usually (but also not necessarily) collaborative or cooperative, and might be supported with a project work. Hence by adapting and expanding an initial diagram from Bishop & Verleger (2013) we can portrait the complex relationship between these groups of theories, under the term active learning, with the following Venn-diagram (Fig. 2).

Fig. 2
figure 2

Diagram of learning theories and methods follow an active learning type of instruction (adapted from, Bishop & Verleger, 2013)

Full size image

Our approach emphasizes relationships and interactions related to the information flow as well as knowledge transfer and transformation. Similarly with a biological ecosystem, in a learning ecosystem, individuals can form groups spontaneously and can interact with each other or with learning utilities at the individual or group level. They can also perform specific behaviors in order to contribute to or perturb to the success of the learning ecosystem (Chang & Guetl, 2007). Changes in the learning ecosystem conditions (external influences) shape the “behavior” of the system and its components. To be successful and to be valuable for the system, each individual and group must adapt to the environmental conditions to find their niches. In order to fit them all together, proper learning infrastructures must also be available (Fig. 3).

Fig. 3
figure 3

The learning ecosystem to support active learning

Full size image

The development of the framework emphasizes in the generic view of the learning ecosystem, hence it is possible for an individual to apply it to any active learning situation, such as project based learning, or peer instruction. Another important aspect is to assert that the interrelationships and interactions with all the organisms, external influences as well as the infrastructures of the learning ecosystem are in principle dynamic (Fig. 3). This generic view helps to get a better picture about a specific learning situation, and allows educators and practitioners to achieve a more holistic approach for the development of more effective learning. A graphic representation of this definition is shown in Fig. 3.

Methods

An initial empirical validation of the proposed framework took place in a public university. The goal of this empirical validation is to provide the first analytics-based evidence regarding the effectiveness and acceptability of the proposed framework. The early results should not be seen as a rigorous evaluation of the proposed framework, but as reflections rising from a particular case study as well as empirical evidence for further development of the framework.

Context

The proposed framework was applied in an introductory computer science course, named web technology. Like any course, web technology was framed within predetermined external influences (conditions) like learning goals, methods, workload and content. The focus of this course is on the World Wide Web as a platform for interactive applications, content publishing and social services. By the end of the course students are expected to be able to design and develop web-pages and web-applications; using markup (e.g., HTML), design (e.g., CSS) and client-side (e.g., JavaScript) programming languages. Students have to deliver specific biweekly assignments, work with a self-selected group project and take written examination; these three components are also the evaluation criteria. The course materials, digital communication as well as the assignments and project-work are orchestrated from a Learning Management System (LMS).

Following the proposed framework (Fig. 3), we implemented video-assignments to scaffold the know/believe part, this helped us to assure that students have attained the fundamental knowledge. This is typical in many active learning approaches (e.g., flipped classroom), where students are involved with the learning materials in order to obtain the initial fundamental knowledge. This basic knowledge was made available using video lectures, in addition we employed an integrated assessment to the video lecture (see Fig. 4) and weekly exercises. These self-regulated, but also well-defined concepts allowed students to better understand the video and reflect about their learning (Giannakos et al., 2015; Kleftodimos & Evangelidis, 2014). Upon students’ completion of the video lecture and the respective assessments, instructors can access all the collected data, visualize students’ activity and progress, and ultimately identify students’ interaction with the materials. Such information allowed instructor to be well-prepared for the active engagement session by addressing all the misunderstandings and misconceptions of the students (Ruipérez-Valiente et al., 2013; Zhang et al., 2010).

Fig. 4
figure 4

An example of a video assignment (using video learning analytics system; Giannakos et al., 2015)

Full size image

Based again on the proposed framework, the active engagement activities are developed with a main focus on recalling the basic knowledge and then engaging students with active learning and critical thinking processes. The recall part is conducted using a mobile game/quiz at the beginning of the class. The instructor prepared a session with questions related to the basic knowledge, supported with different forms of audio visual materials (e.g., videos) (Wang 2015). The class was equipped with a projector, which was used to display the main screen of the game (see Fig. 5a), and each student used his/her own mobile phone to give the answer to the respective question (see Fig. 5b). At the end of the course, the instructor could download all the collected analytics of the game and explore students’ progress and understanding.

Fig. 5
figure 5

An example question by the clicker: (a) projected question; (b) students’ mobile – using Kahoot! (Wang, 2015)

Full size image

In order to engage students deeply in the process of learning, they worked with a group project throughout the semester and were asked to apply the obtained knowledge as well as to make progress presentation and get feedback. This gave them the opportunity to be involved in an active learning process. The aim was to engage students more deeply in the process of learning course material by encouraging critical thinking and fostering the development of self-directed learning. Active learning affords the opportunity for application and practice, and the asking of questions. During the team micro-projects, students had to find a client and mimic the professional software development process. In particular professional software development projects had the following sequence of phases: requirements, design, implementation and testing.

Sampling

The empirical evaluation was conducted in two identical classes (in terms of learning goals, teacher, teaching method and so on), those classes had 510 computer science students (20–29 years old, 105 females and 405 males) enrolled in the web technology course. The course lasted 12 weeks, and we applied the proposed framework (as described in section 2.2). In addition to students’ analytics obtained from the aforementioned systems, we employed a post survey. A total of 73 students’ (14.31 %) volunteered to participate on the survey (14 females, 59 males, with mean age 22.38 S.D. 2.50).

Measures

In order to be able to portray students’ experience during the course, we employed three different types of measures.

  1. a)

    Students’ video navigation (collected via a video learning analytics system (Giannakos et al., 2015), log-files),

  2. b)

    Students’ learning performance/score (collected via the quizzes (Wang, 2015), log-files) and,

  3. c)

    Students’ attitudes toward the course (collected via the post survey, see Table 1)

    Table 1 The measures and its definitions
    Full size table

In particular, video navigation was captured based on students’ interaction with the video player. Students learning performance was collected in pre-middle-post measures throughout the semester. The survey included multi questions factors of 1) ease, 2) control over the course, 3) intention to participate in similarly made course, and 4) usefulness of the overall teaching approach as well as the single question factors of 1) enjoyment, 2) excitement and 3) boredom. Table 1 lists the operational definitions and the number of items (questions) of each of the constructs (measures), as well as the source from which the multi question measures were adopted. We employed a 7-point Likert scale anchored from 1 (“completely disagree”) to 7 (“completely agree”).

Statistical analysis

As aforementioned, the collected data consists of three different types; therefore, an appropriate data analysis was used for each different set of data. Students’ video navigation was analyzed with aggregated time series visualizations, in order to identify students’ navigation throughout the video lecture and the importance of having a concrete assignment alongside with the video lecture. To do so we employed the socialskip.org video analytics system (Giannakos et al., 2015).

As for students’ learning performance, we captured students’ pre-mid-post assessments scores mapped them in a diagram and employed an Analysis of Variance (ANOVA), this will allow us not only to capture students’ performance toward the course, but also to identify any potential shift during the course. Last but not least we used descriptive statistics on students’ attitudes towards the course.

Research findings

By visualizing students’ video usage throughout the semester, we concluded that most of the viewings are happening in three instances. First during the first days of the video release, second during the assignment period where students might need to use some information/knowledge from the video and third before the exam. As we can see in Fig. 6 from a typical video usage visualization, the three main peeks have been identified in the release week, the assignment week and during the last week before the exam.

Fig. 6
figure 6

Students’ video usage throughout the semester

Full size image

With the visualization of the students’ activity using graphs, we reach the conclusion that the use associated with the video assessments provides an important motivation for students to watch the videos and be prepared for the class active learning activities. As we can see from Fig. 7, most of the students’ stopped the video after reaching into the segment where the information for the last question was found. On the other hand, the “attractive” (many views) video segments were identified at the video segments where the presenter was giving the solution of the respective problem.

Fig. 7
figure 7

Students’ activity graphs, identifying students’ dropout after finding the response of the last question of the assignment (1 = max, 0 = min; randomly picked graphs)

Full size image

Another important question is related with the intensity of the questions (number of questions per video) in the video assignment. In order to investigate any potential relation between questions’ intensity and students’ watching behavior; we used activity graphs from video assignments with different intensity. We qualitatively concluded that low questions intensity (very few questions, i.e., 2 or 3 in the video) decreases the number of students’ views and increases the early dropouts from the video. Very high intensity (too many questions, i.e., more than 10 in the video) also decreases the number of students’ views and increases the early dropouts from the video. Medium questions’ intensity (i.e. 3–10 questions) is found to keep students’ interested in the video and also not to frustrate them. Figure 8 gives an indicative example of how questions’ intensity is related with students’ watching behavior. We didn’t investigate questions’ quality, but it is very likely that questions quality and demands (questions need students’ to be able to assemble and synthesize instead of merely reproduce the content from video) will also play an important role in students viewing behavior.

Fig. 8
figure 8

Students’ activity graphs, identifying the importance of question’s intensity in a video-assignment

Full size image

To examine any potential shift in students’ performance during the course, we used Analysis of Variances (ANOVA) between the pre, mid and the post-scores. As we can see from Table 2 students’ scores were at very high levels in the pre, mid and the post assessment. Performing an ANOVA, the results showed no significant difference (Table 2). As a consequence, there was no shift in students’ score throughout the course.

Table 2 Students learning performance in pre, mid and post assessments
Full size table

Regarding students’ attitudes toward the approach, we used an attitudinal survey to assess them (Table 1). We proceed to test the reliability of each measure using Cronbach α coefficient. The result of the test revealed acceptable indices of reliability in all the factors (> 0.7). We also computed descriptive statistics for the factors under investigation. Students expressed high control (mean: 6.36/7; S.D.: 0.88) over the course, lower (though still very high) level on the intention to participate (mean: 5.91/7; S.D.: 1.06) in similarly developed course in the future. In addition, students’ rated very high the easiness for them to follow the processes (mean: 5.84/7; S.D.: 0.91) of the video course and the usefulness of this teaching approach (mean: 5.63/7; S.D.: 1.09). High levels of these constructs indicate positive views concerning usability, control, and usefulness of the proposed approach. In addition to the multi-questions attitudinal factors, we employed three single-question factors related with students’ feelings regarding the teaching approach. Students expressed high scores on the positive feelings of enjoyment (mean: 5.68/7; S.D.: 1.29) and lower (though still very high) level on the excitement (mean: 4.90/7; S.D.: 1.5) and low scores of boredom (mean: 2.63/7; S.D.: 1.37). Figure 9 depicts the mean scores of the attitudinal survey.

Fig. 9
figure 9

Student’s attitudes toward the approach

Full size image

Although all the factors employed in the post-test exhibit students’ intentions and beliefs, they possibly have interrelationships. Noticeable, participants’ intention to participate in the course is quite high, however it would be very important to explore factors that may be strongly related to participants’ intention to participate, we used the Pearson’s rank-correlation coefficient, statistically testing the null hypothesis which is equal to 0. Pearson’s test suggests that some of the factors are related, in some cases relatively strongly. More precisely, the most important positive relationships involve intention to participate on the one side and all the other positive attitudes on the other side. In addition, intention to participate is negatively related to boredom. We present the results of Pearson’s test in Table 3.

Table 3 Correlations Pearson’s correlation coefficient between factors (n = 73)
Full size table

To visualize the effects of other attitudes in students’ intention to participate in the course, we divided Easy, Control, USE, Enjoyment, Excitement and Boredom on high and low categories performing median split. We then created a diagram in order to provide a better understanding of the effect. In Fig. 10, we can observe that Boredom is the only one with a negative effect. Overall, Fig. 10 clearly exhibits the positive and significant (based on the Pearson’s correlation) influence of students’ attitudes on their intention to participate in similarly developed courses in the future.

Fig. 10
figure 10

The influence of students’ attitude in their intention to participate in the course

Full size image

Discussion and conclusions

Practical principles and heuristic models that enable actors within the learning ecosystem to understand and shape their learning future, is considered as one of the cornerstones of smart learning environments (Kwok, 2015). In this research, we presented a learning ecosystem framework and the first captured results of its application. The framework can be put into practice using basic e-learning tools and active learning practices; hence the framework can be used for those interested in incorporating project-based principles as well as flipped classroom or any other active learning approach in their teaching, since it is a flexible procedure that may be adapted to meet different needs.

By exploring the notion of learning ecosystem, we shown that is applicable to describe and model the main actors, diverse resources and sociotechnical dynamics within learning. Hence, the concept of ecosystem is an interesting approach which can be applied in learning and give an overview of the different roles, processed as well as learning dynamics.

The proposed framework was applied in an introductory course in web technologies. In the empirical study, we investigated students’ content navigation, learning performance and attitudes. We also indicated that the main quality of the “attractive” information video-segments is the rich and useful amount of transferred information and knowledge, and of course its association with students’ assessment. Last but not least, we presented students’ progress throughout the course using pre-mid-post assessment, and examined their attitudes regarding easiness, usability, usefulness, and acceptance of the course as well as the two positive and one negative emotions.

We want to emphasize that our findings are clearly preliminary with inevitable limitations. As for the internal validity of the empirical study, data are based on a self-reported method, log-files and assessments. Other in-depth methods such as semi-structured interviews could provide a complementary picture of the findings through data triangulation. As for external validity, the subjects were computer science majors, which may somewhat limit the generalizability of our results. Nevertheless, the insights drawn are not connected with the subjects’ background and can be applied on any population.

Another direction of research is to apply the framework proposed in this paper to the investigation of existing highly engaging learning practices, such as peer instruction and inquiry-based pedagogy. Our future research will concentrate on further refinement of the proposed framework by applying and evaluating it on classes of larger scale and different topics. Further research will also inform the development and evaluation of e-learning environments to better support active learning and livable and sociable learning spaces (Kwok, 2015). This study can provide a springboard for other scholars and practitioners to further examine the efficacy of this specific approach to learning, since it is an established flexible procedure that can be used and adapted to meet the needs of those interested in using the flipped classroom approach.