Towards a Physiological Computing Infrastructure for Researching Students’ Flow in Remote Learning

Abstract

With the advent of physiological computing systems, new avenues are emerging for the field of learning analytics related to the potential integration of physiological data. To this end, we developed a physiological computing infrastructure to collect physiological data, surveys, and browsing behavior data to capture students’ learning journey in remote learning. Specifically, our solution is based on the Raspberry Pi minicomputer and Polar H10 chest belt. In this work-in-progress paper, we present preliminary results and experiences we collected from a field study with medical students using our developed infrastructure. Our results do not only provide a new direction for more effectively capturing different types of data in remote learning by addressing the underlying challenges of remote setups, but also serve as a foundation for future work on developing a less obtrusive, (near) real-time measurement method based on the classification of cognitive-affective states such as flow or other learning-relevant constructs with the captured data using supervised machine learning.

Introduction

Advances in educational technology have enabled new opportunities to provide detailed insights into how learners engage with the provided learning environment and in learning processes. Specifically, learning analytics use static and dynamic educational information from digital learning environments, administrative systems, and social platforms for real-time modelling, prediction, and optimisation of learning processes, learning environments, and educational decision-making. Accordingly, learning analytics is expected to provide benefits for all stakeholders (e.g., students, teachers, designers, administrators) in the higher education arena (Ifenthaler 2015). In particular, students may benefit from learning analytics through personalized and adaptive support of their learning journey.

With the advent of physiological computing systems, new avenues are emerging for the field of learning analytics related to the potential integration of physiological data (e.g., heart rate, blood pressure, skin temperature, or respiration). Hereby, physiological computing systems refer to a class of information systems in which system interaction is achieved by monitoring, analyzing, and responding to sensed human physiological activity in real-time (Fairclough 2008). Specifically, with respect to responding to the user needs in real-time, research has highlighted the advantages of increasingly reliable physiological measurement (Bastarache-Roberge et al. 2015), as well as of designing and developing physiological computing systems (Adam et al. 2017). In addition, such systems span a range of application domains (Loewe and Nadj 2020) such as (1) health (e.g., Bailey et al. 2006), (2) aviation (e.g., Wilson and Russell 2007), (3) transportation (e.g., Cao et al. 2016), and (4) learning (e.g., Shen et al. 2009).

To this end, we developed a physiological computing infrastructure in order to capture students’ learning journey in remote learning.¹ The infrastructure developed can also serve as a basis for future work aimed at classifying cognitive-affective states, such as flow. Hereby, flow refers to “the holistic sensation that people feel when they act with total involvement” (Csikszentmihalyi 2000). For students, being in flow while learning is a desirable state because it can increase concentration and perceived control. As a result, being in flow can in turn lead to increased individuals’ well-being, motivation, and performance (Fullagar and Kelloway 2009). Recent research in the disciplines of psychology, computer science, and information systems focused on the unambiguous classification of flow states based on physiological data using supervised machine learning methods (Rissler et al. 2020). Because of its potential to identify complex patterns in data, machine learning methods are particularly suited to identifying flow states (Shearer 2016). Specifically, this involves learning a function that can differentiate between “high” and “low” flow states of an individual based on given pairs of physiological features as input and self-report scales as output. Various methods are available as classification procedures. For example, promising results have been obtained in the classification of cognitive-affective states based on physiological data using Random Forest (Wen et al. 2014). Especially cardiovascular features such as heart rate variability (HRV) have been identified as measures of interest in the context of flow (Rissler et al. 2020). Technological advances in recent years enable researchers to measure HRV data with the required field accuracy using cost-efficient wearable devices (Züger et al. 2018). While such wearables are easy to apply, they also provide additional challenges to the targeted physiological computing infrastructure. They often need a dedicated device to store the recorded data. This storage must be secure, reliable and, above all, in the context of remote learning, independent of the participant’s existing IT infrastructure with remote support capabilities and yet easy to use.

In this paper, we present preliminary results from a field study in remote learning using the physiological computing infrastructure with medical students from the EDU Institute of Higher Education. Our proposed infrastructure builds on the Raspberry Pi mini computer and Polar H10 chest belt, which we call SEN-Pi (shortened from SENsor Raspberry Pi). Hereby, the students’ physiological data in form of HRV, survey data, and their browsing behavior for a limited set of white-listed domains (which were predefined by the EDU Institute of Higher Education) were collected during their remote learning sessions.

Our main contribution is two-fold: on the one hand, our results offer a new direction for more effectively capturing different types of data (e.g., HRV data, survey data, and browsing behavior data) in remote learning by addressing the underlying challenges of remote setups. On the other hand, our SEN-Pi infrastructure enables the necessary data acquisition for the application of supervised machine learning in remote setups for the defined classification goals. For instance, as in our case, HRV data and/or browsing behavior data could serve as model input, while self-reports of flow and/or other relevant learning constructs could serve as model output. Herein, we see the potential to develop a less obtrusive (near) real-time measurement method for the defined classification goals, based solely on HRV and/or browsing behavior data, and integrate this kind of assessment as an additional feature in our SEN-Pi infrastructure.

The remainder of the article is organized as follows. First, we describe the theoretical foundations of flow and its connection to human physiology. Second, we provide an overview of related works and identify research gaps on this basis. Third, we describe the physiological computing infrastructure in terms of requirements and solution proposal. Fourth, in the Sect. 5, we document the field study design, procedure, and data collection. Fifth, we present results from the analysis of the collected data. Sixth, in the Sect. 7, we highlight our contributions to the identified research gaps, as well as discuss the lessons learned from our field study and the limitations of our approach. Finally, we conclude our article.

Theoretical Foundations of Flow

Following Mihalyi Csikszentmihalyi’s flow theory, nine characteristics are used to describe flow (Csikszentmihalyi 2000; Moneta 2012): (1) challenge-skill balance, (2) clear goals, (3) unambiguous feedback, (4) focused concentration, (5) merging of action and awareness, (6) loss of self-consciousness, (7) perceived control, (8) distortion of time, and (9) autotelic experience.

These nine characteristics can be grouped into antecedents (1–3), subdimensions (4–8), and consequences (9) of flow (Nakamura and Csikszentmihalyi 2009). A great deal of research has documented the positive relationship between flow and performance in various contexts such as playing games (Engeser and Rheinberg 2008), exercising sports (Stein et al. 1995), or making music (Wrigley and Emmerson 2011). This positive relationship is derived from the flow subdimensions, which describe a highly functional state (Engeser and Rheinberg 2008). For instance, loss of self-consciousness and high levels of concentration are believed to help in streamlining attentional processes during flow (Engeser and Rheinberg 2008; Schüler and Brunner 2009). Moreover, the autotelic experience resulting from flow, commonly conceptualized as enjoyment, is thought to be accountable for the rewarding and intrinsically motivating nature of flow perceived after a human has left the state of flow (Csikszentmihalyi 2000). This enjoyment originates, for example, from the achievement of mastering a challenging task at the limit of one’s personal skill (Csikszentmihalyi 2000).

For these beneficial reasons, scholars are broadening the understanding of how flow can be fostered in daily life, and we likewise see promise in examining and facilitating students’ flow in remote learning. This is consistent with flow theory and backed by a large body of empirical research that flow can be experienced in any task where active engagement is needed and the following conditions (also called flow antecedents) are met (Moneta 2012; Nakamura and Csikszentmihalyi 2009; Rissler et al. 2020): (1) a balance between the perceived challenge of the task and personal skill, (2) the setting of clear goals, and (3) unambiguous feedback.

Unfortunately, measuring flow is still considered a major challenge: most research relies on self-reports to assess individuals’ flow (Moneta 2012). However, these self-reports interrupt the flow of the study participants, justifying the need for less obtrusive (near) real-time measurement methods that can assess flow during task execution (Rissler et al. 2018).

With this in mind, scholars have already begun to explore the relationship between individuals’ flow and their underlying physiology (Knierim et al. 2018). This has involved concentrating on the two autonomic nervous system (ANS) branches, i.e., the sympathetic nervous system (SNS; “fight or flight”) and the parasympathetic nervous system (PSNS; “rest and digest”), to determine physiological signals that shed light on their activation degree while individuals are experiencing flow (Berntson et al. 1997; Knierim et al. 2018; Peifer 2012). In this regard, cardiovascular features offer promising sources of information since the heart is innervated by both ANS branches (Berntson et al. 2007; Peifer 2012). In particular, HRV measurements are usually used to evaluate the efficiency of cardiovascular changes (Berntson et al. 2007; Task Force of the European Society of Cardiology the North American Society of Pacing and Electrophysiology 1996). HRV data are thus of diagnostic interest as integral descriptors of activity in both branches of the ANS. With our physiological computing infrastructure, we aim to tap this wealth of knowledge for the field of learning analytics.

Related Work

We relied on a recent literature review by Rissler et al. (2020) as a foundation to identify relevant studies for our article’s focus and to analyze the current state of knowledge of flow physiology research with supervised machine learning.

Table 1 Overview of the literature review by Rissler et al. (2020)

Including their own work, Rissler et al. (2020) provide an overview of six relevant studies as presented in Table 1. However, most of the studies reviewed were conducted in a controlled laboratory environment (Berta et al. 2013; Chanel et al. 2011; Chatterjee et al. 2016; Müller and Fritz 2015; Rissler et al. 2020) and only one study was performed in the field (Rissler et al. 2020). Thus, there exists a lack of physiological flow research in real-world field settings (Gap 1; Rissler et al. 2020). Moreover, three of these studies explore the game context (Berta et al. 2013; Chanel et al. 2011; Chatterjee et al. 2016), whereas the rest examine tasks typically found in a work context (Müller and Fritz 2015; Rissler et al. 2020). Thus, studies investigating the physiology of flow during remote learning are currently absent in literature (Gap 2) although being in flow can lead to increased individuals’ well-being, motivation, and performance (Fullagar and Kelloway 2009).

Table 2 Overview of devices used by the studies found in the literature review by Rissler et al. (2020)

We further examined the aforementioned studies to determine what type of devices were used to record participants’ physiology, particularly with regard to HRV data, and present our findings in Table 2. We used publicly available information from the manufacturers’ websites when we could not evaluate the devices ourselves based on the information in the respective article.

Berta et al. (2013) relied on an Elemaya Visual Energy Tester device with a finger-mounted photoplethysmography (PPG) sensor to record participants’ heart rate. Chanel et al. (2011), in turn, used a Biosemi Active 2 system with an attached PPG sensor. Chatterjee et al. (2016) relied on a finger-worn Contec Pulse Oxymeter to measure heart rate activity, also using PPG technology. The Empatica E3 used by Müller and Fritz (2015) is a wrist-band that also applies PPG to capture the wearer’s heart rate. Rissler et al. (2020) were the only research team to rely on electrocardiography (ECG) devices to capture participants’ heart rates, using a Plux Biosignalplux in their laboratory study and a Polar H10 chest belt in their field study. Such course of action seems important since PPG sensors are easy to wear, but their reading accuracy is often suboptimal (Khandoker et al. 2011), highlighting the need for more flow research using reliable physiological readings like ECG (Gap 3).

Furthermore, studies conducted in a laboratory were able to use a dedicated laboratory computer to connect to their recording devices (Berta et al. 2013; Chanel et al. 2011; Chatterjee et al. 2016; Müller and Fritz 2015; Rissler et al. 2020). The field study by Rissler et al. (2020), in turn, used a self-developed Java application to connect to the recording device, which was used by participants on uniform company laptops. Thus, none of the identified systems are platform-independent or can be widely deployed at participants’ homes or workplaces (Gap 4).

To access the recorded data, the device employed by Berta et al. (2013) requires a proprietary software. Similarly, the Biosemi Active 2 system used by Chanel et al. (2011) needs a proprietary hardware connector and proprietary software. The Pulse Oxymeter employed by Chatterjee et al. (2016) does not, to the best of our knowledge, have any recording or transmission capabilities, but only shows the current reading on a display. The Empatica E3 used by Müller and Fritz (2015) supports a proprietary software connection with a cloud upload to a cloud storage of the device’s manufacturer. The devices used by Rissler et al. (2020) allow full data access and transmission via a common Bluetooth connection. In summary, all studies conducted in a laboratory setting did not intend to give their participants access to the recorded data. Only in the field study by Rissler et al. (2020), participants received a report after the study was completed. Thus, we see the need for a physiological computing infrastructure that grants researchers full access to the data without the need for proprietary middleware (Gap 5), while allowing participants access to their recorded data, preferably in (near) real time (Gap 6).

For a price estimate, we also used publicly available information, for instance, from the manufacturers’ websites. We only considered the hardware costs for the sensor solution used by each study, disregarding any costs for software development, licensing, or additional infrastructure like laboratory laptops. For the Elemaya Visual Energy Tester used by Berta et al. (2013) we could not find any sales information. The same is true for the Biosemi Active 2 system employed by Chanel et al. (2011), but the price of 540 Euro for the official carry case represents a good lower bound, as we assume that the device itself costs significantly more than its carry case. Chatterjee et al. (2016) relied on a common Pulse Oxymeter manufactured by Contec for 20 Euro. Müller and Fritz (2015) used the Empatica E3, which has since been superseded by the newer E4 model, priced at 1690 Euro. Rissler et al. (2020) employed a Plux Biosignalplux system for 995 Euro in their laboratory study and the same Polar H10 chest belt in their field study we used as HRV sensor in our study at 80 Euro a piece. The additional hardware required to complete our SEN-Pi system is billed at another 80 Euro, but eliminates the need for special infrastructure considerations such as target platforms or laboratory computers.

The Physiological Computing Infrastructure SEN-Pi

Requirements

As described in the introduction, field studies conducted in a remote setup face numerous challenges with respect to the underlying infrastructure. In consultation with our cooperation partner—the EDU Institute of Higher Education—we have identified the following requirements. (1) Secure Storage All physiological data collected must be encrypted and stored securely. Ideally, this should be done without the use of cloud services, as personal data is collected. (2) Reliability The data must be stored reliably. This includes the writing operation itself, but also the resilience to crashes and connectivity problems with the Bluetooth connection to the wearable. Users must be able to take short breaks from learning. If they leave the home office while wearing the wearable, the infrastructure must automatically disconnect and reconnect without user input. (3) Ease of Use The use of the infrastructure by participants should be easy to use and not require any technical knowledge. All processes should be automated as far as possible and require limited user input. (4) Platform Independence We target platform independence, as we cannot assume a common denominator for the IT platform of all users. They could potentially use very different hardware setups with different operating systems (OS) installed. This is especially important with regard to the Bluetooth connection to the wearable. (5) Remote Support We do not have physical access to the users as they live all over the country. Therefore, our solution must have remote support capabilities that require limited input from the user. (6) Cost Efficiency We should target cost-efficient wearables, because the infrastructure should be scalable to support future field studies with larger sample sizes.

Solution Proposal

We did not find a commercial off-the-shelf solution that met all of the above requirements. Therefore, we decided to develop and deploy our own infrastructure (i.e. the SEN-Pi) based on a combination of a Raspberry Pi mini-computer and a Polar H10 chest belt wearable.

We compared several wearables available on the market before selecting the Polar H10 chest belt. Our pre-tests showed that photoplethysmography (PPG) sensors, which are often used in smartwatches, do not meet our defined requirements. While such sensors are easy to wear, they have reliability issues due to movement artifacts and shortcomings in sensor accuracy, which have also been documented in research findings (Khandoker et al. 2011). This essentially excluded all smartwatches and most fitness bands available on the market at that time. Other factors for device exclusion included the use of proprietary connection protocols, cloud only data upload or no access to raw data. In the end, we chose the Polar H10 chest belt because it is a cost-efficient wearable that offers accurate sensor readings and platform independence through the use of the Bluetooth connectivity standard.

Another hurdle was how to collect the recorded HRV data, as we aimed for secure storage. We decided to keep all the data offline by configuring and deploying a mini computer based on the Raspberry Pi platform. The Raspberry Pi is a small, affordable single-board computer with an ARM CPU (Gay 2014). By using a Linux OS, we were able to easily deploy our own software. It also offers secure remote support capabilities via SSH and VNC, two technologies that allow full control of the computer via encrypted connections over the Internet (Richardson et al. 1998). This is important so that we can provide assistance and apply updates (if necessary) during the field study without having physical access to the device. The Raspberry Pi also offers a GPIO interface to which a variety of sensors could be connected in the future, such as Galvanic Skin Response (GSR) or environmental sensors (e.g., temperature and CO2 sensors), providing additional insight into the learning environment of participants.

In addition to HRV data, we also wanted to collect survey data. Hereby, we implemented a logic for the well-established Experience Sampling Method (ESM) (Larson and Csikszentmihalyi 2014). In this method, participants are surveyed several times throughout the day in their natural environment for the defined study period. Specifically, the ESM logic was implemented in a browser extension for Google Chrome. This solution has the advantage that it is independent of the OS and can be easily deployed via the browser’s extension store. Thus, whenever participants launch the browser during a field study, the ESM browser extension is active in the background. In addition, the browser extension is also able to collect browsing behavior data for predefined URLs from a whitelist. Specifically, when navigating to a new web page, the URL of the new web page is recorded only if it is included in the whitelist. In turn, URLs not listed in the whitelist are not saved in any capacity whatsoever. An icon in the browser bar indicates to the participant whenever this recording functionality is active. Hereby, the browsing behavior and survey data are stored in a secure online database via REST API.²

Fig. 1

Schematic visualization of SEN-Pi’s software stack including ESM browser extension

Figure 1 shows the software stack that makes up the SEN-Pi infrastructure including the ESM browser extension. We used the Linux derivative “Raspbian” as the OS. An SQL database is used to securely store all data recorded by our Python application. This application controls the connection handling with the wearable and runs as a background service whenever the device is turned on. This recording application automatically opens and closes the Bluetooth connection when the paired wearable enters or leaves the transmission range. The recording application parses incoming data packets and stores the HRV values in the database. In turn, the ESM browser extension is decoupled from the rest of the infrastructure and runs independent from the users’ computer system.

Fig. 2

Each point of the Poincare chart represents a pair of subsequent interbeat intervals (IBI). An IBI measures the time in milliseconds between two heartbeats. The dots are typically clustered along the diagonal from lower left to upper right. The orange cluster in the lower left simulates what a recording might look like during periods of high activity, consisting of smaller IBI values or a high heart rate. The blue cluster in the upper right is an example of a recording at rest with large IBI values or a low heart rate. If the Poincare chart shows many dots in the red circled areas in the upper left and lower right quadrants, this indicates sudden changes in heart rate. Such situations could point to a faulty sensor or that an individual is suffering from a medical condition. (Color figure online)

A second application accesses the database to present the recorded data in a web dashboard. This dashboard is served by a Python web server running on the SEN-Pi. Participants can connect to a WiFi hotspot opened by the SEN-Pi to access the web dashboard using their individual login credentials. This gives participants both (1) personal insight into their own physiology during the field study and (2) exemplify what data is stored from them. Given the medical background of the participants, the web dashboard presents charts used in a medical context, such as a Poincare chart (Carrasco et al. 2001) (see Fig. 2) or a visualization of heart rate in beats per minute over time (see Fig. 3). In addition, the web dashboard also allows participants to download their personal data as MS Excel files to perform their own analyses.

Fig. 3

The heart rate in beats per minute (BPM) over time is shown. A boxplot of the BPM values of the presented recording session (blue) and all recording sessions stored on the SEN-Pi (orange) is also illustrated. Finally, the mean heart rate for this recording session and its comparison with the overall mean and healthy ranges defined by research findings for the gender and age group are shown (Nunan et al. 2010). Gender and age can be configured by the user under “Profile Settings”. (Color figure online)

Research Method

Study Design

Eight medical students from EDU volunteered for the field study. Two of the participants were male and six were female. Because they were attending an online institute of higher education, the participants did not share a common office or learning space. Therefore, the field study was conducted in their homes across the country.

Hereby, the field study consisted of three phases: First, the preparation phase was conducted in the first week (5 working days) to test the SEN-Pi infrastructure in the field and resolve any potential issues that may arise. Second, the testing phase was conducted in the second week (5 working days) to collect the actual data. Third, the extension phase offered participants the opportunity to continue their participation for an additional third week (5 working days) to collect more data.

As introduced in the Sect. 4.2, we implemented a logic for the well-established Experience Sampling Method (ESM) to also collect survey data (Larson and Csikszentmihalyi 2014). In particular, participants were surveyed several times throughout the day while studying or participating in online medical classes for the defined field study period. The surveys asked questions about the participants’ current flow experience. In addition, we collected data from a learning behavior questionnaire that was asked once during the study period, HRV data using a Polar H10 chest belt, and participants’ browsing behavior on predefined websites (see Sect. 5.3 for details). HRV data and browsing behavior were collected continuously throughout the study period.

Procedure

Written informed consent was obtained from each participant before the start of the study. Each participant received a package consisting of the Polar H10 chest belt and the Raspberry Pi mini computer in the week before the study began. Each package included a card labeled with a unique pseudonym³ and password combination. This pseudonym represented their unique participant ID throughout the study. The password allowed participants to access their personal web dashboard (see Fig. 3) in order to offer (1) personal insight into their own physiology during the field study and (2) exemplify what data is stored from them.

During the installation, the ESM browser extension asked the participants to login with their participant ID. This allowed us to match the flow survey responses stored in an online database with the recorded HRV data on the SEN-Pi after the field study was completed. On the first day of the study, a kick-off meeting was held to install the ESM browser extension and explain how the Polar H10 chest belt and SEN-Pi infrastructure work. A video tutorial was also created where all installation steps were explained and illustrated once again. Participants could access the video tutorial at any time during the field study. If a participant had an issue during the field study, they could connect the SEN-Pi to the Internet so that an experimenter could securely access the SEN-Pi remotely to investigate the issue and offer assistance.

For each study day, participants were instructed to follow the subsequent procedure: First, the participant supplies power to the SEN-Pi device via the provided USB charger. They put on the chest belt and open their browser. At the beginning of the study (i.e., on the first day), participants are asked to answer the one-time questionnaire about their learning behavior. In addition, every day, when the browser is started, the installed ESM browser extension opens a page to record a physiological baseline for the respective day. In particular, participants are asked to breathe deeply and relax while watching a video of the Great Barrier Reef with relaxing background music for 300s (Piferi et al. 2000). The ESM browser extension remembers when this step is completed and disables the reminder until the next day. The timestamp when the video is started or paused are stored in an online database.

After the daily baseline recording is completed, participants can begin their normal learning day. They can use the online resources provided by the EDU Institute of Higher Education for remote learning. Hereby, participants’ browsing behavior from predefined websites is stored. Moreover, during the day, randomly scheduled notifications with a visual and audio cue by the ESM browser extension interrupt their activities and prompt participants to answer the flow survey. These interruptions are scheduled at random intervals between 40 and 90 min. The notification shows two buttons, one to open the flow survey, the other to dismiss the notification. If there is no interaction for one minute, the notification is automatically dismissed. At the end of their day, participants close their browser, remove the Polar H10 chest belt, and disconnect the SEN-Pi from the power source.

Data Collection

During the field study, we collected different types of data:

• 1. Learning Behavior Survey. The learning behavior survey consisted of three constructs: (1) metacognitive awareness (Schraw and Dennison 1994) measured on a binary scale, (2) class-related emotions (Pekrun et al. 2002), and (3) technology affinity (Mills et al. 2014)—both assessed on a 7-point Likert scale. This questionnaire was only asked once during the study period.

• 2. Flow Survey. Flow is measured via the Short Flow State Scale (SFSS) (Sa and Marsh 1996). Participants are reminded to only consider their experience of the last five minutes before answering a total of nine questions measured on a 7-point Likert scale. This survey is presented according to the ESM study design (for details see Sect. 5.2).

• 3. HRV Data. The participants’ physiology is recorded by measuring their HRV data with a Polar H10 chest belt at a sampling rate of 130 Hz.⁴ This data is recorded continuously while wearing the wearable.

• 4. Browsing Behaviour Data. The browsing behaviour is only recorded for a limited set of whitelisted domains which were predefined by the EDU Institute of Higher Education. This whitelist includes websites from the EDU Institute of Higher Education, but also online textbook libraries (e.g. the Oxford Medical Library) or commercial tools used for learning. An icon in the browser bar indicates the participant whenever this recording functionality is active. In addition, participants were informed before the field study which websites would be tracked.

The complete whitelist is shown in Table 3. Whenever a site belonging to a page on the whitelist is visited, the URL and timestamp was recorded.

Table 3 Domain whitelist used for tracking browsing behavior

Results

Of the eight participants, three collected HRV data, one lost the SEN-Pi equipment,⁵ and four decided not to collect HRV data during the study period. Participant #1 recorded a total of 77 h 35 m of HRV data throughout the field study. Participant #2 recorded 8 h 50 min, and Participant #3 39 h 06 min of HRV data.

The ESM browser extension collected 45 completed flow surveys from these three participants. The mean flow was 5.36 with a standard deviation of 1.74. However, the collected number of flow surveys was too small to train a machine learning classifier. Therefore, we resorted to an existing machine learning classifier trained with data from a previous field study we had conducted with nine software developers from a software vendor. During its previous testing this classifier achieved an accuracy of 70.6% and F1 values of 73.7% and 66.7% for high and low flow, respectively. The training and testing process of the machine learning classifier is described in Rissler et al. (2020). According to this classifier, participant #1 spent 48 h 50 min (63.7% of the time) in flow. Participant #2 was 5 h 55 min in flow (67.0% of the time). Participant #3 spent 27 h (69.1% of the time) in flow. Nevertheless, caution should be taken when interpreting the results of the classifier because we could not train and test a machine learning classifier based on the data collected in the current study. Rather, the application of the existing classifier illustrates the potential for applying machine learning in future field studies for remote learning.

Fig. 4

Amount of browsing events across the day

The learning behaviour survey was answered by all seven participants who returned their equipment. In particular, metacognitive awareness (Schraw and Dennison 1994) achieved a mean of 0.70 with a standard deviation of 0.20. Class-related emotions (Pekrun et al. 2002) had a mean of 3.25 and a standard deviation of 0.63. Finally, technology affinity (Mills et al. 2014) achieved a mean of 3.94 and a standard deviation of 0.80.

In addition, we recorded over 6.200 browsing behaviour data from all seven participants. Figure 4 shows a histogram of activities across the day. Participants preferred to work later in the day, with most of the activities occurring at 3 pm and between 9 pm and 2 am. Each activity lasted approximately 50 min on average.

Finally, we analyzed the domains that participants visited (see Table 4). The first domain, the EDU Institute of Higher Education website, accounted for 86.2% of the recorded activity. The second domain, the online classroom, made up 4.5%. The third domain, a knowledge resource for medical terms, accounted for 4.3% of the recorded activity. The fourth domain, a drawing game, was visited 2.7% while the fifth domain, another learning tool, was visited 1.7%. The sixth domain, an online library for medical texts, accounted for 0.6% of the recorded activity.

Table 4 Browsing activity per domain group

Discussion

Recent technological advances enabled scholars to measure HRV data with the required field accuracy using cost-efficient wearable devices (Züger et al. 2018) opening up new avenues for the field of learning analytics. While such wearables are easy to apply, they also pose additional challenges to the targeted field infrastructure often requiring a dedicated device to store the recorded data. This storage, however, must be secure, reliable, and especially for remote setups, independent of the participant’s existing IT infrastructure with remote support capabilities and yet easy to use. To gain additional insight into these challenges, we developed a physiological computing infrastructure called SEN-Pi, based on a Raspberry Pi mini-computer and a Polar H10 chest belt, and field-tested it with medical students from the EDU Institute of Higher Education to capture their learning journey during remote learning.

In the following section, we discuss the contributions of our article, the lessons learned from our field study with suggestions for future research, and the limitations of our approach.

Contributions

First, our study is among the first to collect different types of data via a physiological computing infrastructure in remote setups, contributing to the goal of using physiological field data to enable machine learning classifications of flow or related constructs in the near future (see Gap 1, Related Work). Laboratory experiments remain of undisputed importance and are especially well suited for achieving high internal validity because they keep extraneous variables in the form of noise, lighting, and temperature stable (Mitchell and Jolley 2010). However, the conditions generated in laboratory experiments, as emphasized by researchers (Fairclough 2008; Loewe and Nadj 2020), correspond only to a certain extent to the conditions of a real world environment, which justifies the need for our SEN-Pi infrastructure.

Second, being in flow can lead to increased individuals’ well-being, motivation, and performance (Fullagar and Kelloway 2009) and therefore seems to be a desirable state for students during remote learning. For this reason, our work lays the groundwork to the goal of developing interventions that could support students’ flow during remote learning, for instance, by monitoring flow in (near) real-time using individuals’ HRV and/or browsing behavior data (see Gap 2, Related Work).

Third, our SEN-Pi infrastructure employs accurate ECG-based sensors to record physiological data. In contrast, most previous studies relied on PPG-based sensors (see Gap 3, Related Work), which have undesirable characteristics such as sensor artefacts caused by excessive motion. ECG devices provide more accurate readings and are less sensitive to motion artifacts (Khandoker et al. 2011). Therefore, the inclusion of ECG-based sensors in our infrastructure allows us to measure accurate data that will form the basis for the development of future classifiers using machine learning.

Fourth, SEN-Pi provides an infrastructure for field experiments that is platform-independent and can be widely deployed at participants’ homes or workplaces (see Gap 4, Related Work). Because it is a complete system, it does not impose requirements on an existing IT infrastructure, but remains cost-efficient. Thus, our physiological computing infrastructure supports future research in large-scale field experiments, which seems important since larger sample sizes control for the risk of reporting a false-negative finding (Biau et al. 2008). In addition, large data sets open up the possibility of using deep learning algorithms in the future, which has led to improvements in other complex task including ECG and EEG analysis in recent years (Rim et al. 2020).

Fifth, our physiological computing infrastructure provides research teams with full access to their collected data without the need for proprietary middleware (see Gap 5, Related Work). Other systems that have proprietary access may incur additional cost and/or effort to access the raw data using a subscription model. Also, automatic uploading of data to proprietary cloud storage does not always comply with legal regulations, such as the European Union’s General Data Protection Regulation. Our solution circumvents these issues while still providing full access to all stored data.

Sixth, the SEN-Pi infrastructure makes the collected data transparent to study participants (see Gap 6, Related Work). In particular, individuals can view their personal data in (near) real-time on a web dashboard. Thus, participants not only retain control over their personal data, but can also gain new insights and perform their own analyses with their data.

Lessons Learned and Suggestions for Future Research

The logic of when and how a notification should appear was an important aspect of our ESM browser extension. We decided to rely on the notification module offered by Google Chrome.⁶ This module uses the notification center available in the OS to send a notification. However, during the first two days of the preparation phase, we noticed that some participants did not submit a completed flow survey. Among the participants, several different OS were used in different versions. While the majority were using MS Windows 10, some participants were using outdated versions. With these outdated versions, notifications were sent correctly, but the notification center attenuated the notifications by default, so the user could not see them unless they directly accessed the center or changed the default value in the settings.

For this reason, we deployed an update for the ESM browser extension on the third day of the preparation phase. This update solved the problem, as it no longer relied on the notification center available in the OS, but used ordinary browser alerts to interrupt participants. This technical problem confirmed the need for an infrastructure that is independent of participants’ IT systems and the importance of conducting a preparation phase to detect such technical problems in advance.

Since EDU medical students typically attend their online courses in a block format and an exam had to be taken after each online block, the participants’ schedules were already busy. Therefore, in order to conduct the study in this context at all, it was important to offer a balance between mandatory online appointments for the field study and flexible appointments as needed. In consultation with the EDU Institute of Higher Education, we offered four sessions: two online briefing sessions to promote the study, one kick-off session to install the software with the students, and one debriefing session to explain the results of the field study. In addition, if a participant experienced problems during the field study, they could connect the SEN-Pi to the Internet so that an experimenter could securely access the SEN-Pi remotely and offer assistance. The experimenter support was offered during regular working hours.

In the future, it would be interesting (if organizationally possible) to also offer a mandatory exchange session after the end of each study week, in order to get an immediate impression of all participants from the respective week.

Next, we want to discuss the incentive structure of the field study conducted. As an incentive to participate in the study, students were able to view their own physiological data via a personal web dashboard as well as download the data as MS Excel files for further analysis. For each pseudonym, an analysis of the collected data was also performed by the experimenters after the end of the study and stored in a secure database. Only the owners of the respective pseudonym could access the corresponding analysis. In the two briefing sessions, the students reacted very positively when these incentives were presented and declared their willingness to voluntarily participate in the study under these conditions. We explain this willingness by the fact that the topic “human physiology” is an inherently medical one and that a basic affinity of medical students was served by the study. Second, the prospect of not only receiving a one-time, personalized analysis at the end of the study, but also being provided with a personalized web dashboard with established visualizations from the medical context (e.g., the Poincare chart) in near real-time over the entire study weeks seemed to offer another explanation.

However, user behavior only partially coincided with students’ voluntary willingness to participate in the study, as ultimately only three of seven participants actually recorded HRV data. For future field studies in such setups we therefore suggest to consider additional incentives such as study credits or financial compensation.

Finally, we would like to discuss the importance of data privacy and data protection in our solution. In our requirements elicitation process, we found that HRV data required special attention in this regard. Therefore, we opted for a secure hardware storage solution in the form of the Raspberry Pi. The data was only stored locally in encrypted form. Unauthorized access via the Internet was thus made impossible. As mentioned earlier, all recorded data was made transparent to each participant in near real-time via a locally served, personalized web dashboard. All recorded data could be downloaded and viewed by participants, who retained control of their data at all times. Moreover, by using randomly assigned pseudonyms, no one, including the experimenters, was able to match the recorded data to a participant’s identity.

Nevertheless, software-based storage solutions such as cloud storage could be considered for the future. In doing so, important issues such as the location of the server, data access by the host or other third parties, and participant data sovereignty need to be discussed and evaluated. In terms of data privacy and data protection, we therefore see the current hardware storage solution offered by the SEN-Pi as an advantage.

Limitations

We are aware that our approach comes with limitations. In particular, with 8 participants, we expected to collect at least 160 completed flow surveys (5 working days * 8 participants * 4 completed surveys per day), with up to 480 in an absolute best-case scenario (10 working days * 8 participants * 6 completed surveys per day). Our assumptions were consistent with the amount of data required to train a machine learning classifier according to our previous research outcomes (Rissler et al. 2020). Although we did not meet our goals for data collection, we see value in communicating our findings for conducting remote field studies since current flow research has mostly neglected requirements for field infrastructures in remote setups, especially for physiological computing (see Sect. 3).

Next, our current web dashboard was also designed as an incentive for medical students to participate in our field study. Although the charts displayed were selected for their ease of reading and explanatory screens were provided, they represent more specialized medical insights. For students in other disciplines, the dashboard could benefit from additional or different (simplified) metrics that are more appropriate for the targeted user group. Similarly, incentives for users to participate would also need to be adjusted in other contexts, such as the workplace.

Finally, we did not ask our participants to indicate their age, as it would have been possible to identify each participant by age given the limited sample size. In general, participants’ ages fall within the 20–30 age range typical for college students. However, we are confident that our infrastructure is suitable for very mixed age groups undertaking remote learning, as long as individuals are comfortable using personal computers and the Internet.

Conclusion

Overall, we were able to demonstrate the feasibility of our SEN-Pi physiological computing infrastructure. The usage patterns derived from the analysis of browsing behavior offered the EDU Institute of Higher Education feedback on how they could fine-tune their curriculum or evaluate which external services students actually use. The SEN-Pi fulfilled the requirements as defined in Sect. 4.1. In particular, it provided a reliable, cost-effective, easy-to-use, platform independent and secure solution capable of providing remote support. Once the technical issue was resolved during the first week of the preparation phase, the device functioned flawlessly for the remaining two weeks. In summary, our results do not only provide a new direction for more effectively capturing different types of data (e.g., HRV data, survey data, and browsing behavior data) in remote learning by addressing the underlying challenges of remote setups, but also serve as a foundation for future work on developing a less obtrusive, (near) real-time measurement method based on the classification of cognitive-affective states such as flow or other relevant learning constructs with the captured data using supervised machine learning. On this basis, one could expand the hardware to include additional sensors (e.g., GSR, temperature, CO₂ sensor) or incorporate more sophisticated software, such as a pre-trained machine learning classifier. Although we did not meet our goals for complete data collection, we gained valuable insight into technical and organizational challenges in a remote field setup. By incorporating our experiences, we are confident that we can successfully conduct future remote field studies with a potentially even larger sample size.