1 Introduction

In its 2009 report, The European Agency for Safety and Health at Work (OHSA) named stress to be the second most frequent work-related health problem, affecting 22 % of workers from the EU27 [4]. Today’s society imposes multitude of different responsibilities and demands constant multitasking, which leads to unprecedented surges in stress levels [7]. In order to cope, many people resort to popular momentary relaxation and reflection techniques [14]. Still, overwhelmed and unable to see dangerous trends in their stress levels, people often start reflecting on their stress patterns and possible stress coping strategies too late, when one of the stress-related diseases affects them. Especially stress responses experienced over longer periods of time, that build up, can lead to health related problems [6].

There exist individual patterns in person’s behavior that lead to increases in the level of stress or to better relaxation [11, 13]. A behavioral pattern can manifest itself in different contexts and the intensity of the stressful response can depend on the context, which makes pattern identification difficult. On the other hand, thanks to the digitalization of the world and recent advances in sensor technologies it becomes possible to facilitate the discovery of behavioral patterns [3].

In this paper, we look at peoples’ behavior in relation to their physiological responses and visualize the obtained information using a tool called LifelogExplorer. We combine arousal information, tightly linked to stress, with contextual information in form of artifacts produced by users’ activity in the digital world such as calendar activities, e-mails, (logs of) phone calls. The tool allows users to explore correlations between their activities and their emotional responses and reflect on their behavior. By realizing such relations they may identify possible problems, attempt a change in their behavior, and observe its effects. The first version of LifelogExplorer was presented in [8] and we further improved it based on user feedback from multiple studies.

In our previous case studies [8, 9] we evaluated our approach with LifelogExplorer and the arousal data obtained with the DTI-2 sensor wristband [19]. The focus of those studies was on the evaluation of the quality of measurements and the understandability of the visualizations generated by LifelogExplorer. Since we aimed at an objective evaluation and thus avoided interfering with the users’ perception of the experienced stress during the studies, the users were only given a possibility to use LifelogExplorer and explore their data at the end of the studies. The obtained results showed that long-term measurements of arousal allows to reveal people information about their behavioral patterns perceived as meaningful and useful.

The question we address in this paper is whether the users are actually interested in seeing their behavioral patterns on a regular basis, able to learn from them and, if needed, to act upon this knowledge to undertake a behavior change intervention. We carried out a case study with teachers of vocational school, organized with the help of Human Capital Care (HCC)—an occupational health organization. We show the analysis users’ activities on exploring their data with LifelogExplorer during the four weeks of the study.

The rest of the paper is organised as follows: In Sect. 2, we describe our information sources, their interpretation for generating the LifelogExplorer views and the general functionality of the tool. In Sect. 3, we present the case study and its results. Finally, in Sect. 5, we draw conclusions from our work.

2 Approach

The area of behavior change in context of wellbeing management focuses on changing user’s unhealthy behavior into a more healthy one. Most of the strategies employed are derived from stage-based model [15], setting theory [10], and cognitive dissonance theory [5]. Although focused on various aspects, a common requirement for all of them is the need of first making the user aware of the need for behavior changes. In the context of stress management, it means that the user needs to become aware of the main sources of stress, related behavioral patterns and be able to monitor the results of interventions aimed at behavior changes.

Monitoring stress and arousal became an active research field due to recent advances in wearable sensor technologies. Unobtrusive and continuous measurements of skin conductance and/or heart rate variability supplemented with information about physical activities, can be used to estimate the stress level [18].

Several approaches for the continuous monitoring of stress and arousal were proposed in the last years. In the Feel project, the authors created a system for automated annotation and monitoring of phone calls and stress responses of the users  [1] using a mobile phone and a wearable biosensor. The presented visualization of the data is restricted to calendar views: day, week, month and a list view for events, limiting the ability of the user to spot non-time-related trends and reflect on them. AffectAura [12] is another system aimed at enabling self-reflection on emotional states over an extended period of time. The authors employ multiple sensors in order to capture the precise context of emotional responses, which limits the applicability of the approach to office-workers. Also here, the visualization of data is driven by the time dimension only. The AffectiveHealth project focuses on live presentation of the physiological signals with the main focus on providing real-time, short-term feedback to the user; the measurements are visualized in the form of a dynamic spiral to encourage personal reflection on body reactions throughout the day [16]. The recall and the pattern discovery are out of the scope in that approach. An extensive survey of recent work done in the context of sensor-based personal wellness management systems is presented in [20].

In our approach we aim at facilitating self-reflection and identification of stress-related patterns by visualizing links between stress-related data and different aspects of person’s activities, to raise awareness of the contexts leading to stress unbalance.

2.1 Information sources

LifelogExplorer [8] shows users information about their arousal based on data coming from several information sources.

We use The Discrete Tension Indicator (DTI-2)—wristband developed by Philips Research [19] that enables stress detection in everyday environments. DTI-2 measurements include skin conductance, body and environment temperatures, 3D acceleration. Skin conductance reflects the activity of sweat glands controlled by the sympathetic nervous system and therefore it can be used for estimating the arousal level of a person [2]. Based on the DTI-2 measurements, we estimate user’s arousal per minute using the procedure described in detail in [9]. The main steps of the procedure include filtering and smoothing the signal followed by discretization into personalized arousal levels (see Fig. 1).

Fig. 1
figure 1

Translating the raw skin conductance signal into discrete arousal categories

Full size image

The other information source we use is the digital calendar of the user. Calendars are widely used to reinforce ones memory in daily life situations, mostly remind the user about activities to come. In our approach, calendar is also very useful to look at the past and provide the structure for identifying regularities in behavioral patterns. Calendar data can be seen as a time series of activities with information about what, where, with whom, when and on whose initiative happened in the past. Currently, we support automatic extraction of calendar information from MS Outlook, but the same approach can be used for other calendar types (e.g. Google calendar, MacOSX iCal).

2.2 LifelogExplorer

LifelogExplorer provides comprehensible and concise interactive visualizations of the user data, including data stored in digital calendars and sensor data. Correlations between user reactions and the context can be explored with different levels of granularity. We assume inherent incompleteness of collected information; therefore, it remains up to the user to decide which behavior patterns are relevant and can be changed. The effects of interventions aimed at behaviour changes can be monitored in the tool using trend views.

LifelogExplorer offers two types of views on the user’s arousal: calendar views and aggregated views. Calendar views have clear benefits for understandability since they show the arousal information on top of a usual calendar representation. Each aggregated view shows the arousal footprints for one of the dimensions of users activities, like subjects or locations of calendar events, with the main purpose of enabling the user to identify long-term patterns. The tool also provides a number of advanced functionalities to give the user more control.

Visualization of arousal levels In LifelogExplorer we represent arousal level using colors, ranging from dark blue to dark red, corresponding to the gradation between very low and very high arousal level, with level labelling personalized per user. Light yellow, being in the middle of the colour scale used, represents the “average” arousal, which for many users corresponds to an active working mode, e.g. when involved into a lively conversation.

Stress footprint Stress footprint is a visual representation of user’s aggregated arousal levels in a specific context. For example, a footprint can represent arousal for a day of the week, or for a particular subject of calendar events. The footprint is visualized as a half pie chart (see Fig. 3), where its size reflects the relative time spent within this context. Intuitively, the longer the period represented, the larger the footprint. The half pie chart is divided into a number of ordered colored slices, each representing a level of arousal. Size of each slice represents the relative amount of time a person experienced a particular arousal level in this context.

Fig. 2
figure 2

A calendar view with arousal level information

Full size image
Fig. 3
figure 3

Examples of aggregated views in LifelogExplorer

Full size image

Calendar views LifelogExplorer augments the calendar with information about arousal levels in one of the two forms: stripe glyphs, with the same blue-to-red color scheme, (see Fig. 2a) and line glyphs (see Fig. 2b). On the top of each day, a day footprint summarizes arousal levels to ease the estimation of the stressfulness of the day as a whole. Line glyphs of arousal are created by plotting the arousal estimates based on the sensor data vertically, following the direction of time axis, and then mirroring the plot. The lines created in that way are further joined and filled-in to create an impression of a solid object. The view provides more detail about the acute arousal responses and gives more freedom of interpretation to the user, since it offers no coloring. Additionally, the tool offers custom days views, in which selected (possibly nonconsecutive) days are shown next to each other in one of the two calendar views. The selection of days can be made by the user, or automatically generated based on the user request from an aggregated view (see below).

Aggregated views LifelogExplorer can generate multiple aggregated views to summarize the arousal data for different dimensions, such as meeting subjects, locations, attendees, as well as time related dimensions such as hours of the day, days of the week, weeks, months (see Fig. 3). Such views are composed of multiple footprints and their goal is to facilitate understanding of user behavior patterns over longer periods in various contexts. The user can click either on one of the footprints or on a slice of a footprint to generate a custom days view with all the days comprised in this footprint/slice.

Figure 3a shows stress footprints for days of a week. A single footprint comprises the data of multiple days, i.e. Monday represents all Mondays within the considered time period. For cyclic dimensions, like week or hours of the day, we use a circular layout as the default layout to ease understanding and exploration. In the shown view, one can spot that Thursdays are the most “stressful” days for that user and Saturdays look the most calm ones. The smaller size of the footprints for Saturday and Sunday reflect that fact that this user was only occasionally wearing the sensor wristband during weekends; this indicates that the information for these days is only partial and might be biased.

Figure 3b shows an example of a trend view allowing to observe changes over time by enabling to compare arousal footprints per subject of calendar events over consecutive months (days or weeks could be also chosen). It can be seen e.g. that “Club meetings” were particularly stressful in September, but they were relatively calm in November.

Histogram view is an advanced functionality of LifelogExplorer that allows the users to change the categorization of their stress levels (choose the number of categories and the boundaries between them), as shown in Fig. 1.

3 Approach evaluation

We conducted a non-experimental study at a vocational school. We were mainly interested in evaluating user engagement in viewing their own data, their ability to learn from it and to apply self-coaching. To evaluate these aspects LifelogExplorer logged participant’s use of the system, hence avoiding affecting the users’ natural behavior and preserving ecological validity. We also discuss here what functionalities of the tool were mostly used, and what kinds of visualizations users prefer. Since observation of behavior using logs does not give insight in the reasons of users’ choices and preferences, an on-line questionnaire and semi-structured interviews have been employed at the end of the study. The interviews were conducted by a coaching expert from HCC and focused on discovering if the users were able to make sense of the visualized information, learn from it and make self-triggered interventions to alter their behavior.

Participants The study involved 21 participants (15 males, 6 females). Their age ranged from 23 to 56 \((\mu =41.0, \sigma =11.59)\). They were not using drugs, antidepressants and were not suffering from a disease resulting in a fever or excessive sweating. There was no preselection related to stress or fitness. 2 of the 21 participants joined the study a week later (and finishing it a week later), after learning about it from their colleagues at the vocational school and expressing their interest to participate. The participants did not get any financial compensation; we were interested to see whether they are intrinsically interested in seeing information extracted from their data.

Protocol The study duration was four weeks. At the beginning of the study the participants were given a short tutorial about the use of DTI-2 and LifelogExplorer. They were asked to wear the DTI-2 wristband during their working hours, and they were allowed to wear the wristband outside their working hours occasionally or systematically, if they wished. They received a link for downloading LifelogExplorer with their individual user codes and were free to install it on the PCs of their choice, at work and/or at home.

During the study, the participants downloaded the measurements from the DTI-2 wristband by connecting it to a PC with the LifelogExplorer tool running at the moments of their choice, with a request to do it at least once: at the end of the study. LifelogExplorer automatically detects device connection, downloads the data and updates all visualization elements. To provide context to the measurements, the participant’s work schedules were automatically uploaded to LifelogExplorer. The participants were free to decide when and which views on their data they want to explore, if any. The participants were not supervised directly; interventions were made only at participant’s requests.

4 Results

We have collected 20 days of measurements per participant on average \((\sigma = 5.92)\) along with an average of 90 calendar entries per participant \((\sigma = 31.30)\). 7 of the participants chose to wear the device also during some of the weekend days and 5 of the participants expressed their interest in using the system for an additional week, which they were allowed to do. Out of the full group, two persons did not collect any data—one due to getting ill at the beginning of the study and another because of consistently forgetting to wear the device. Two more participants are excluded from further analysis due to the low quality of collected physiological measurements (with more than 50 % of the data filtered out as too noisy).

Analysis of tool use The average number of sessions with the LifelogExplorer amounted to 18.2 \((\sigma =14.1)\) per user (compared to 20 days with measurements per user, on average). In principle, the users could have just used the tool for downloading the data from the device without exploring it any further. To differentiate between “data upload only” sessions and sessions in which the user actually viewed his data and avoid the bias the arousal visualisation set as default view, we configured the tool so that the initial view showed the calendar without arousal information. To see a view with some arousal information, the user had to click a button choosing the visualization type. We call a session active session if the user viewed his/her arousal information. 86.7 % of user sessions were active sessions. Further we give statistics for active sessions only.

The number of active sessions of two users were outliers, being very high: 41 and 47, respectively. After removing these two outliers, the data for the number of active sessions per user fits the normal distribution with the mean value 11.6 and standard deviation 6.0, according to Shapiro–Wilk Normality Test [17], with \(W= 0.89\) for \(p=0.05\).

During the study, the users spent a total of 95 min using the tool on average per user. The mean duration of each session was 6.5 min and the average number of actions per session amounted to 25. In terms of visualization style preference, the participants looked at stripe glyph views in 91 % of the active sessions, while they looked at line glyph views in 46 % of the sessions. Only in 6.6 % of the active sessions the users did not use any of the calendar views on the arousal (but aggregated views and histogram views).

All users accessed the aggregated views at some point, with the average number of 8 aggregated views per user during the whole study period; 89 % of the users went there further than looking at the default aggregated view selection (“Days of the Week”). The aggregated views were used in 37 % of the sessions and the custom days views in 26 % of sessions. 37 % of the custom days views were called from an aggregated view, to understand on which days certain things from the aggregated view happened, in the other 63 % of the custom views, the user defined the days he wants to see and compare. 14 % of the users also made use of the trend view functionality with tracking trends by weeks being the most popular option (which is logical, taking into account the duration of the study and weekly schedules of the users). The histogram view was looked at in 23 % of the sessions and in 53 % of these sessions the users customized the categorization of arousal.

Interestingly, 59 of 300 active sessions were started within 1 h after a previous active session of this user (the mean time till the follower session was 11.54 min, \(\sigma = 11.98\,\)min). 16 of 19 users had such “follower” sessions. We considered three clusters of sessions: follower sessions (following another session within 1 h), sessions being followed, and stand-alone sessions and stated that the three clusters have very similar characteristics both from the perspective of the session duration and the number of user activities in the sessions. The only difference was related to uploading the sensor data: it took place in 67 % of sessions being followed, only 15 % of the follower sessions and in 67 % of the stand alone sessions, indicating that the follower sessions were primarily used for taking another look at the data. Because of the anonymity, we could not contact the users having such follower sessions, and we can only make a conjecture that the follower sessions are caused by self-reflection taking place after the first session and the need for more information due to it.

61 % of the sessions took place in the first two weeks of the study, which means that the users explored their data in the second half of the study less frequently but did not lose interest in it. Moreover, 58 % of the users spent more time per session, on average, in the last 2 weeks of the study than in the first 2 weeks.

The big variation in the number of sessions per user suggests big differences between users’ interest in exploring their data. The relatively high number of activities and time spent on average per session suggests users interest in the information they get. Statistics for the use of aggregated views suggests that the participants were interested in exploring their long-term behavior patterns, which is especially encouraging taking into account the study duration. The use of histogram views and custom days views suggest that some users are interested in having control over the way the information is shown to them. Finally, clear usage preference towards the color based visualization in the calendar indicates that most of the users prefer interpreted visualizations of their arousal.

Qualitative feedback In order to get feedback on the practical usefulness of our approach, a coaching expert from HCC conducted interviews with 7 participants of the study and the other 12 participants were asked to filled in the on-line questionnaire, from which 8 did so. We chose for interviews by en external expert since they are more objective and avoid the risk of introducing a bias by the creators of LifelogExplorer. Below we present the results of both the interviews and the questionnaire, as well as some typical comments given by the participants on learning about behavioral patterns and triggering behavior changes (the comments are translated from Dutch to English).

12 out of 15 participants who provided feedback indicated that they have learned something new about themselves based on the presented information, providing comments like “Recognizable, but also surprising to see that certain activities, such as meetings, are experienced as stressful.”, “You learn and see what particular activities have impact on your energy and state of arousal” Most negative comments focused on the belief of the users that they already knew about their stress reactions. "I knew that classes require varying energy.” “I knew that repetitive activities can cost equal energy.”

Although 12 out of 15 participants indicated they learnt about themselves, only 5 (33 %) indicated that they are able to change something in their lives based on what they have learned. Some of their comments are “Yes, my stress level was going down during the breaks. Probably, it might help to take breaks between classes and meetings more often”; “Yes, dropping activities that did not require direct action helped me to lower my stress level”; “Yes, I better schedule my time”. Participants who felt they were not able to act upon the presented information indicated that stress they experience is part of their job and they do not think it can be reduced. Some mentioned that the results can only be obtained if weekly meetings with a discussion of stress-related issues would take place and the changes are tried out. Some of their comments are: “Some results are just part of the job.”; “I still need to give lessons to these groups.”; “No, I do not think I can do anything to reduce my stress, especially on busier days.” These results suggest that many people, even being able to identify behavioural patterns linked to stress, are not able to translate this knowledge into self-advices and need an additional assistance of a coach or peers.

5 Conclusions and Future Work

In this paper we presented the results of an unsupervised study on longitude monitoring of arousal in the context of daily events of the user and evaluated how the participants used LifelogExplorer—a tool visualizing relations between arousal and different aspects of daily events. The results of the study show that most participants increased their self-awareness of arousal-related patterns and some of them were able to generate practically relevant interventions to improve their stress balance based on what they learnt. This is especially encouraging since the study was unsupervised and it was conducted in a real-life setting.

To further support users in identifying meaningful stress patterns and to reduce incompleteness of the lifelog we plan to include additional information sources, like telephone and internet activities logs. We would also like to further explore possibilities of engaging users in effective and practically useful behavior change interventions, allowing options for automated monitoring of the intervention effects and generating recommendations guiding the user to the objectives (s)he chooses.