A Bayesian Assessment of Real-World Behavior During Multitasking
Multitasking is common in everyday life, but its effect on activities of daily living is not well understood. Critical appraisal of performance for both healthy individuals and patients is required. Motor activities during meal preparation were monitored in healthy individuals with a wearable sensor network during single and multitask conditions. Motor performance was quantified by the median frequencies (f m) of hand trajectories and wrist accelerations. The probability that multitasking occurred based on the obtained motor information was estimated using a Naïve Bayes Model, with a specific focus on the single and triple loading conditions. The Bayesian probability estimator showed task distinction for the wrist accelerometer data at the high and low value ranges. The likelihood of encountering a certain motor performance during well-established everyday activities, such as preparing a simple meal, changed when additional (cognitive) tasks were performed. Within a healthy population, the probability of lower acceleration frequency patterns increases when people are asked to multitask. Cognitive decline due to aging or disease might yield even greater differences.
KeywordsWearable sensors Activities of daily living Cognitive loading Executive function Motor control
Much can be learned about the brain by studying motor coordination . Motor behavior, defined as the combination of movements that produce purposeful or intended actions, emerges due to a synergy between a range of systems . The systems involved in this behavior are bounded by certain parameters and they have evolved to work within real-world constrains. Everyday living activities arise through the complex interaction of these factors and dysfunction within these factors will generate alternative behaviors. The occurrence of large changes in everyday living behavior can be an indicator that (patho)physiological changes are emerging. This is also the reason that at present, the diagnosis of disorders such as Alzheimer’s disease still heavily depend on the clinical history  and the observed behavioral changes by relatives and friends. Furthermore, there is growing evidence that indicates a link exists between activities of daily living (ADL) and executive dysfunction in patients suffering from early dementia .
It is unclear how changes in certain parameters might affect the behavior under real-world conditions. The complex interactions underlying behavior can be better understood by e.g., exploring effects of cognitive loading in a healthy populations. This information will be particularly interesting if it represents behavior that is common in the real-world. To our knowledge, no detailed assessments of multitasking within an ADL context exists, in which the tasks cover a range of complex everyday tasks. It still remains ambiguous to what extent everyday living is affected by cognitive ability. Current cognitive loading experiments often consist of experimental designs that do not capture the real-world performance of the biological system as they would occur on a daily basis. Issues with this standard experimental approach have been highlighted in a paper that investigated postural prioritization during multitasking . It showed that well-established principles of “posture first” strategies, wherein individuals favors execution of motor components over execution of cognitive components, degraded when measurements were taken in more complex environments. This kind of decrement in motor performance can have devastating real-world impacts if it leads to for example a fall or injury. To what degree our complex motor performance is affected by cognitive loading needs to be explored further.
In the case that complex ADL motor performance is affected by cognitive loading then logic would imply that we might be able to predict cognitive loading by monitoring motor behavior itself. Studies have been performed on more skilled tasks such as handwriting , but activities that stretch over several phases which make up a complex everyday task have not been researched under multitasking conditions. In this explorative study, motor performance is investigated, while variable levels of cognitive loading are introduced. The probability that multitasking occurred based on motor performance data (hand trajectories and wrist accelerations) will be estimated with a Naïve Bayes Model. The model is a simple probabilistic method based on Bayes Theorem. In practice, Naïve Bayes models often compete rather well compared to more sophisticated models . The simplicity, large community of users and ease of implementation makes the Naïve Bayes Model an ideal candidate for initial exploration of real-world multitasking.
The aim of this study is to investigate to what extent meal preparation is influenced by multitasking. It is hypothesized that multitasking will increase the probability of observing “slower” motion patterns, as defined by a decrease in the median frequency.
In total 21 (8 male, 13 female), healthy participants were recruited, with a mean age of 23 (±3) years, an average height of 170 (±8) cm, and an average weight of 67 (±12) kg. All participants gave written and informed consent to volunteer for the study and the protocol was approved by King’s College London BDM Research Ethics Subcommittee.
Sensors on the arm and back were kept in place by double-sided tape and straps. The head sensor was placed on a non-slip elastic headband.
Subjects were asked to perform the task of preparing a meal during a 40-s trial. The meal preparation consisted of making as many sandwiches as possible and it was constructed to include several items from the Motor Activity Log (MAL) for the upper extremity . Participants were asked to butter and cut as many slices of bread as possible within 40 s.
Subjects were instructed to speak freely and/or perform an additional cognitive activity (stroop task) during certain trials, while always performing the aforementioned everyday motor task. Four conditions were implemented and these consisted of just performing the motor task (single-task condition), performing the motor task while speaking (dual-task condition with speech) or while conducting a stroop task (dual-task condition with stroop task), and finally performing the motor activity concurrently with both speaking and a cognition task (triple task condition). The single task condition consisted of a trial at the start of the experiment and one at the end. A total of three trials were recorded for all other conditions. Conditions were pseudo-randomized in order to eliminate sequence effects in the outcomes.
The cognitive loading task consisted of a specific audio-spatial assignment. The auditory spatial task utilized a spatial stroop stimulus and it was presented through a wireless stereo headphone. Within one trial, three stimuli were given, with 10 s between each stimulus. The subjects were requested to response to unilateral aural stimuli. The stimuli consisted of the words “Left” and “Right” delivered through either the left or right headphone speaker. If the word matches the side, it was presented to (i.e., “Left” in the left ear) the result is congruous and therefore the appropriate response was to tell the researcher it was correct by shaking the head up and down. If incongruous, the subject is asked to state it was incorrect by shaking sideways.
Stimuli generation for the stroop task, as well as the data acquisition was performed in Matlab R2014a (MathWorks Inc., Natick, MA, USA).
Stroop Task Response Detection
Varying the values of a and b will provide the continuous wavelet transform coefficients C a,b indicating how closely the wavelet is correlated to the original signal. These coefficients are of course dependent on the selected waveform (ψ) and function (f). A larger value for C a,b shows a greater similarity between ψ and f.
The time at which a certain stimulus was given was subtracted from the time when a response was detected. This value represented the response time of the subject. A window size of 10 s was used to identify any responses, as the stimuli were generated at a .1 Hz rate. The response was labeled “incorrect” if no response was found. Identified responses were compared to the expected response. If the response was expected to occur within a specific direction (yaw or pitch) the response was labeled “correct.” Otherwise, the response was deemed “incorrect.”
In which t yaw (1) and t pitch (1) are the time points at the start of the response and t yaw(n) and t pitch(n) are indicating the end of the response. If any overlap is detected, the signal with the highest average energy is identified as the leading signal (1 is assigned) and the other signal is seen as the crosstalk signal (assigning it a value of .5). If both signals are equal in terms of average energy, they are both assigned a value of .5 and it can be stated that it is inconclusive which response the subject wanted to give.
A truth matrix consisting of dichotomized outcomes allows for easy assessment of performance. The first two cells of each row can be summed and if this value is greater than 1 the performance can be labeled as correct. This simple computation provides a quick top level view of the provided responses. The summed outcomes were labeled as extracted responses. Response detection was further validated during a small pilot trial (see supplementary information).
A total of four trials could not be analyzed due to data corruption (two single and two multitask trials). These trials were therefore excluded from further statistical testing.
Boxplots were used to visualize the data. A rank transformation procedure was used in order to apply an analysis of variance on the data , with groups consisted of the four conditions (single task, dual with speech task, dual with stroop task, and triple task). The ranked f m was used as the dependent variable. Subsequently, a non-parametric Kruskal-Wallis tests were performed upon acceleration and position data to establish if any differences were present between conditions .
The probability that an observation belongs to a certain class (posterior probabilities) were estimated using the predictor space, which was defined by instances on a 2D–grid. The posterior probability that a classification is C i for a given observation was computed by multiplying the conditional joint density of the predictors for a certain class with the class prior probability distribution and dividing it all by the joint density of the predictors . It was hypothesized that a condition with an increased probability for lower f m should yield a lower functional performance.
All data analysis and statistics were performed in Matlab R2014a (MathWorks Inc., Natick, MA, USA).
The ranked analysis of variance showed no significant difference between conditions for hand trajectory (F(3195) = 0.3170, p = 0.81) and acceleration (F(3195) = 2.556, p = 0.06). The Kruskal-Wallis test also found no significant differences for hand trajectory (H(3) = 0.246, p = 0.97) nor acceleration (H(3) = 6.852, p = 0.08).
Applying this model for (same dataset) prediction would generate a misclassification of 38%, with most of the misclassification occurring in the f m of acceleration region between 15.8 and 17.2 Hz covering the .4 to .6 range of task probability. This region contained 33% of all data points. Values outside this region yielded a relative good probability for separating the two tasks across all subjects. In general, higher f m values for accelerations were found in the single task, while low values more likely indicated subjects performing a triple task.
Results showed no difference in hand trajectories between the conditions when traditional statistical methods such as the analysis of variance and Kruskal-Wallis test were used. However, visualization, analysis of variance and the Kruskal-Wallis test indicated a clear trend towards lower f m for multitasking when the acceleration data was explored. The Bayesian probability estimator showed that differences existed in the probability estimates between the extremes (single and triple tasking). This differentiation between the single and triple task was also observed when the number of prepared sandwiches were counted. Participants completed less sandwiches when they were multitasking. It suggests that subjects will become “slower” both in f m accelerations, as well as in overall functional performance, when they are requested to multitask. The motor differences appears to be too small to be subjectively perceived as a decline in performance by subjects, but they become apparent by applying a simple Naïve Bayes model. The f m feature has previously been used to successfully classify different activities of daily living [16, 23], provides a relatively simple and thus informative metric. However, other features should be explored in order to determine if discrimination can be further improved.
Our human perception bias often exists in quantifying our own performance and this bias is also found in caretakers assessing activities of daily living in those who suffer from a decline in cognitive abilities . A more objective approach to unobtrusively track function will therefore benefit both patients and clinical professionals. This kind of technology can especially impact those older adults who are living alone and the change attitude towards technologies can positively influence the uptake of these devices . It is important to consider that the activities completed in this study are very natural and intuitive. Thus, the finding of any differentiation between single and multitasking in healthy activities of daily living is therefore a very intriguing and warrants further investigation.
Accelerations obtained from mobile devices have already been used for automatically detecting different states of physical activity . Measuring wrist accelerations might therefore provide a smart and acceptable method for monitoring, as there is no need to define any task specific constrains. This data can be easily gathered by an unobtrusive wrist worn accelerometer. However, confirmation of the presented results in other datasets is still required.
Naïve Bayes models are very attractive for estimating general probability within real-time domains making it a suitable model for real-world tracking . It also provides a computational inexpensive method for differentiating between tasks and it is relatively easy to implement. A key aspect of the model is that independence is assumed between the predictors, which may on occasions be violated. More sophisticated models should be applied in future studies to minimize assumptions, but these initial results show that even a simple method might be able to detect an increased probability of certain motor behavior occurring when healthy individuals start to multitask.
Although, real-world interaction is noisier, more heterogeneous and less repeatable than the induced stroop task, the induced task does reflect the domain of interest a lot better. The results found in this study seem comparable with other real-world scenarios. It has already been proven that mobile phone use has a detrimental influence on driving performance  and interestingly enough, even practice seems to be unable to eliminate the disruptive effects of concurrent cell phone use on driving . Real-world multitasking, might therefore be strongly engrained into our behavior. This makes it easier to robustly monitor and assess any potential changes. It would also indicate that we should investigate cognitive load effects in the case of human-machine interfaces in order to make them more ecological valid.
This study shows that even simple everyday tasks performed by healthy individuals can be affected by multitasking for certain individuals. The potential to monitor this with an unobtrusive wearable sensor provides an interesting approach for further exploration in relevant patient populations, such as Parkinson’s disease (PD).
Parkinson’s disease (PD) is a progressive neurodegenerative disorder that affects the central nervous system and is primarily found in patients over 50 years of age. Symptoms include difficulty with motor skills such as walking and writing, as well as uncontrollable shaking (tremor), and general lethargy. These symptoms are caused by the death of neurons in the midbrain that control movement by generating dopamine, a neurotransmitter that modulates neural pathways and allows for smooth, controlled movement . In later stages of the disease, patients may experience trouble with emotional control and dementia [29, 30]. Studies have shown that early movement impairments and cognitive deficits can provide insight into the underlying neurodegenerative processes . In the case of Parkinson’s disease, changes in physical movement typically precede changes in language and behavior. Measurements of movement are therefore particularly valuable as indicators of the earliest stages of neural dysfunction. In addition, impairments in PD are exacerbated under simple dual-task conditions requiring the simultaneous performance of cognitive or motor tasks when compared to healthy controls [32, 33, 34]. This provides further evidence that the aforementioned method of monitoring ADL under a range of conditions might be able to accurately predict changes at the executive level.
An increased probability of finding low median frequencies (f m) for wrist accelerations was found during complex multitasking compared a single activity. It shows that even in healthy individuals who are performing everyday tasks, changes can arise in motor performance due to multitasking. Differentiation based on probability is possible at the extreme ends of the recorded values, while overlap exists within the midrange. It is likely that certain patient populations will show even more pronounced differences in motor performance during multitasking. The opportunity to measure this with a modest wearable sensor makes it of interest for further research in relevant patient populations.
Compliance with Ethical Standards
Conflict of Interests
The authors declare that they have no conflict of interest.
Informed consent was obtained from all individual participants included in the study.
This research was supported through a fellowship from the Brain Sciences Foundation (BSFFELLOW01-2013JB) granted to Dr. Jeroen Bergmann.
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Ethical approval was given by King’s College London: Biomedical Sciences, Dentistry, Medicine and Natural & Mathematical Sciences Research Ethics Subcommittee (BDM RESC).
- 7.Rish I, editor An empirical study of the naive Bayes classifier. InIJCAI 2001 workshop on empirical methods in artificial intelligence; 2001 Aug 4: IBM New York.Google Scholar
- 14.Hartmann DL. Objective analysis. ATM S 552 Notes. 2008:51–86.Google Scholar
- 17.Neal DK. Goodness of fit tests for normality. Mathematica Educ Res. 1996;5Google Scholar
- 19.Conover WJ, Iman RL. Rank transformations as a bridge between parametric and nonparametric statistics. Am Stat. 1981;35(3):124–9.Google Scholar
- 20.Gibbons JD, Chakraborti S. Nonparametric statistical inference: Springer; 2011.Google Scholar
- 21.Lowd D, Domingos P, editors. Naive Bayes models for probability estimation. Proceedings of the 22nd international conference on Machine learning; 2005: ACM.Google Scholar
- 22.Friedman J, Hastie T, Tibshirani R. The elements of statistical learning: Springer series in statistics Springer, Berlin; 2001.Google Scholar
- 23.Wan-Young C, Purwar A, Sharma A, editors. Frequency domain approach for activity classification using accelerometer. 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society; 2008 20–25 Aug. 2008.Google Scholar
- 32.Rochester L, Hetherington V, Jones D, Nieuwboer A, Willems AM, Kwakkel G, et al. Attending to the task: interference effects of functional tasks on walking in Parkinson’s disease and the roles of cognition, depression, fatigue, and balance. Arch Phys Med Rehabil. 2004;85(10):1578–85.CrossRefPubMedGoogle Scholar
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.