Over the course of this decade, both mobile devices and mobile device applications designed for the express purpose of tracking physiological and cognitive performance metrics are nearly ubiquitous in our everyday endeavors. Though the idea of tracking our own various daily metrics is nothing new, the advent of technological innovations that carry the capacity to store, sort and share these ever-accruing amounts of data presents to us unchartered territory regarding our relationship to and understanding and interpretation of these metrics both singularly (as snapshots) and in the aggregate (over time). This work explores the form and function of the Quantified Self Movement. It discusses the current state of analyzing and interpreting information accrued from various metrics, specifically the issues that arise in synchronizing heterogeneous metrics, and in the context of an AugCog framework, proposes a method of approach to analyze multivariate data by curation based on the simultaneity of measured events.
1 The Concept of Self-Quantification
First coined in 2007 by the editors of Wired, “The Quantified Self” is a movement that by 2012 had accrued countless online social networking forums and more than 70 meetup groups worldwide. Though the notion of self-tracking information may arguably pre-date history, in our modern era the sheer quantity of available metrics to track, continuously, is unparalleled. Benjamin Franklin famously recorded his “13 Virtues” in a daily journal, Buckminster Fuller maintained a diary of his daily life and ideas in which he referred to himself as, “Guinea Pig B,” but not until Stephen Wolfram recently stated that, “[o]ne day I’m sure everyone will routinely collect all sorts of data about themselves,” (as does he on his own self) has it become apparent that the level of self-tracking various and heterogeneous information has reached near-ubiquity.
With the advent of wearable technologies that possess the ability to communicate, we have the capability of not only recording our information but of transmitting it to: linked devices and cloud databases as well as social networks and other interested parties. Biosensing applications are one of the largest arenas for growth with regard to the Internet of Things , and expounding upon the IoT concept, Wolfram Research is now developing algorithms for linking and analyzing multiply tracked physiological metrics for the creation of wholistic diagnostics and assessments for healthcare applications .
A previous work described a method approach to instantiate a ubiquitous AugCog framework, specifically one that addresses: how to meaningfully interpret combined objective and subjective metrics (e.g., physiological data and mood), how to efficiently and intelligently utilize heterogeneous measurements made in real-time and over time, and how to determine the limits of cognitive load to assist in the delegation of task work functions in human-computer interactions . With regard to commercially obtainable devices, this work discusses the current state of the Quantified Self movement for both personal and research applications, and proposes the instantiation of the Quantified Self in an AugCog framework.
2 Quantification: Have Device, Will Take Data
2.1 The Quantified Persona
With wearable technologies, the movement of the Quantified Self is purposed for the measurement of all aspects of daily life. Recent polls show that 45 % of US adults manage at least 1 chronic condition, and 69 % of US adults track at least one health indicator either for themselves or for someone else . Of those managing two or more chronic conditions, 56 % say that self-tracking has affected their overall approach to maintaining their health status .
On a wearable or mobile device platform, some common daily metrics to track may be blood pressure, heart rate, hours slept and to some degree by actigraphy, quality of sleep; dietary and caloric intake, and the number of steps walked (or run); for example, the iPhone 6 and the Galaxy S5 each carry a suite of health-tracking apps, and the panoply of wearables such as the Fitbit, Jawbone Up, and Nike Fuelband all utilize similar sensors that result with similar metrics. Though the variables measured are essentially the same, each device platform implements its own unique calculation algorithm, and each platform plus each embedded sensor array carry their own set of latency and measurement error parameters. None of these parameters are described to the consumer such that the consumer is left with a deficient understanding of how to interpret a measurement within realistic error bounds and specificity. A recent BBC investigative report lamented the measurement discrepancy betwixt each of four popular trackers, the response to which by a developer of one of the four in question underlined that there is both a communication gap between the manufacturer and the consumer as well as an inherent misunderstanding on the part of the consumer in terms of how to best utilize the technology and interpret the resulting data .
The resulting data itself presents what is heretofore referred to as The Quantified Conundrum. Tracking multiple variables that each likely relate to only singular applications presents a quandary: how does one visualize the time-aggregated and time-specific data from one application, let alone across multiple applications and/or devices? And, perhaps more importantly in the context of commercial devices, who owns the data that’s been collected? A community of biohackers are beginning to answer these questions, for both ordinary and medical devices. For example, FluxStream is an open-source framework with many APIs available to visualize personal data across different applications, the code for which is freely posted on GitHub . Though it seems the obvious solution, an app to manage and visualize data from various devices and applications is a nontrivial idea: a consumer actually has little to no control over the storage and usage of the data accrued on and across apps, devices and services, and though a multitude of developers have attempted to alleviate this problem, not many have succeeded. As of the time of this manuscript, there is still no one-stop-shop app for aggregating and visualizing the myriad metrics a consumer may collect on his or herself, but a laundry list of both live and now-defunct apps and APIs is available on a blog popular to Quantified Selfers, called LifeStream .
The concept of The Quantified Conundrum is thrust into deeper and darker waters in the context of mobile medical devices. Type 1 diabetics must monitor their blood sugar frequently through the course of the day, and for a parent of a diabetic child, that once meant round-the-clock testing, during both waking and resting hours (nighttime hypoglycemia is a serious problem for Type 1 adolescents). With the advent of the Continuous Glucose Monitor (CGM), a hair-width sensor placed under the skin logs and reports blood sugar levels every few minutes, via a short-range connection to an external monitor. Should the levels plummet somewhere perilous, the monitor sounds an alarm. This technology was revolutionary in 2012, but owing to the short-range connection, the external monitor had to be within 100 ft of the patient such that outside of resting hours, this technological solution became less useful for a primary caregiver.
When managing the logistics of this monitor became another job, one tech-savvy parent of a diabetic child coded an Android app that allowed him to monitor from afar on his mobile phone his child’s CGM data that had been subsequently transmitted to the cloud. After the app had proven to work, he tweeted a screenshot of the CGM results and other frustrated, medical device owning, biohacking parents, patients, and caregivers followed suit as comrades-in-arms (included in this proclaimed “revolution” was the very real development of a bionic pancreas) . Interestingly, the response to this revolution came from the manufacturer of that particular CGM, in the form of an FDA cleared (January, 2015) docking station that transmits a patient’s CGM data via bluetooth to up to five telephones residing anywhere in the world . Though this new product is a fortuitous uprgrade from the original setup, the new design implies that while the data should be easily shared amongst guardian/caregiving parties, it is still ultimately owned by the manufacturer and not by the data provider.
2.2 The Quantified Task Force
Relating together physiological and cognitive informatics has been of interest across research communities to assess, for example: the neurocognitive and physiological interplay in human sexuality, [24, 25], the dynamics of human cognition in psychosomatic and interoceptive processes [16, 21, 23, 26], and cognitive load resulting from delegated tasks in complex operational environments [4, 14, 17]. Though the results of such measurements are intended for different purposes, the metrics themselves are similar if not identical and when the level of complexity in assessment increases, the same resulting data problems arise, irrespective of origin.
The appropriate synchronization and combination of time-sensitive physiological and cognitive measurements requires a well-executed experimental design and a subsequent visualization scheme for analyzing the data that may potentially also include with it observational meta-data recorded over the course of the experiment; this is an inherently difficult task. Within complex operational environments, the capability to assess cognitive workload is imperative to an efficient and successful outcome of the operation but is often logistically problematic to conduct. In a paper from 2003, Gevins and Smith stated that, “perhaps the most basic issue in the study of cognitive workload is the problem of how to measure it,” . Electroencephalogram (EEG), functional near-infrared (fNIR), electrocardiogram (ECG), electrooculogram (EOG), skin conductance, plethysmography, heart rate variability, eye-tracking, pressure sensing and haptic feedback are some of the many various measures commonly acquired in cognitive workload research, though a systematic approach to interpreting these data in concert has yet to be developed.
A performer on DARPA’s AugCog program described in 2002 a method to define an Index of Cognitive Activity with eye-tracking measurements, but when eye-tracking was combined with EEG there was no immediately understandable way to relate the two, and it was acknowledged that the study design should be simplified in order to unveil a discernible relationship between them . The intended purpose of AugCog was to, “develop technologies to mitigate sensory or cognitive overload and restore operational effectiveness by extending the information management capacity of the warfighter,” and over the course of the program various methods were developed to corroborate heterogeneous physiological metrics, such as eye tracking, skin conductance, and blood flow , but combining these with cognitive assessment via EEG or fNIR presented engineering obstacles then and continues to do so now . While progress has been made on the fronts of dry-use EEG monitors  and compact arrays of singular physiological sensors as well as combined cognitive and physiological sensors [12, 15], there still remain technological and analytical grounds to break. With no packaged, single-solution sensor suite and no patented synchronization algorithm, we thus find ourselves immersed in another layer of The Quantified Conundrum.
3 Conundrum: Have Data, Will Infer Meaning
Whether the data-gathering expedition originated from either a commercial or research prospectus, the data-gatherer is ultimately directed towards data analysis. In both cases, there is prolific difficulty in the foundational matching of various metrics in time and sampling frequency in a manner that will foster meaningfully sensical results. The difficulties partly reside in the fact that these metrics are separately recorded, but moreover, such issues are also products of the magnitude of data processing that frequently occurs at the level of either the device or the application interface. With little to no control over the basic parameters of the measurements to be made, the subsequent interpretation of the results may be cursory only, owing to the lack of available analytical details. For example, because the calculating algorithms of and the error bounds on the measurements are both unknown in commercial health trackers, the tracked data can be only be meaningfully interpreted by changes in trends. Similarly problematic with research tools, the investigator is largely limited by the level of processing imposed upon the data before it is output or reported. Of greater importance are both the underlying intention set to collect the information as well as an a priori understanding of the various measurements individually, i.e., if different measurements result in redundancy, is the redundancy itself intentional, perhaps to speak ground truth against comparative measurands, or is it not intentional, in which case what version of the measurement provides the best option?
In an AugCog framework, we consider intentionally curated information to divulge relationships between measured factors such that those relationships exhibit a profile of the experimental state, in time. In particular, any one given measurement may vary with any number of stimuli, and some variables may be interdependently linked such that the relationships betwixt them are not obvious until the confounding factors are removed. For example, heart rate variability may temporally increase with every other cognitive and physiological metric and thus to remove the effects on other variables driven by heart rate variability, we normalize each by heart rate variability to find potential second-order effects. Second order effects may elucidate response characteristics that are unique to particular stressors, whether environmental or internal in origin, but are only discovered in the analysis post-process.
With respect to data from self-tracking devices, an instantiated AugCog method approach could entail importing multiple variables into an API that aggregates data across various platforms and visualizes the variables in a time series analysis to determine with what factors the variables dynamically change. In a complex operational environment, the challenge is to determine what suite of sensors delivers the most relevant combination of information while minimally encumbering the humans-in-the-loop. For example, sopite syndrome is a complex that arises from exposure to real or apparent motion, defined by a number of nonspecific symptoms such as yawning and skin pallor without nauseagenic stimulus, and that results in a reduced ability to focus on an assigned task [10, 11], and hence, the human subject experiencing the distress is already physically and cognitively taxed such that surpassing a critical number of sensors in the array could prove to be analytically superfluous at best and deleterious at worst. In both cases, the garnered understanding of the first and second-order relationships between various metrics gives rise to a further understanding of how to approach the development of learning algorithms that could provide real-time predictions of events that may be informative of impending physiological and/or cognitive distress. This type of information is exceptional and unique to our modern, quantifying era, and as such it is ultimately important to concerned parties, whether self-trackers, caregivers and medical providers, or assignment-delegating computers and thus worthy of future work and inquiry.
Cummings, M.: Technology impedances to augmented cognition. Ergonomics in Design (Views & Provocations), 25–27 (2010)
Hurley, D.: Wired: Diabetes patients are hacking their way toward a bionic pancreas (2014). http://www.wired.com/2014/12/diabetes-patients-hacking-together-diy-bionic-pancreases/
Fox, S., Duggan, M.: Tracking for health (2013). http://www.pewinternet.org/2013/01/28/tracking-for-health/
Gevins, A., Smith, M.: Neurophysiological measures of cognitive workload during human-computer interaction. Theor. Issues in Ergon. Sci. 4, 113–131 (2003)
Ikehara, C., Crosby, M.: Assessing cognitive load with physiological sensors. In: Proceedings of the 38th Hawaii International Conference on System Sciences - 2005 (2005)
Lewington, L.: BBC: Why activity trackers deliver mismatched fitness data (2015). http://www.bbc.com/news/technology-31113602
Life Stream (2013). http://lifestreamblog.com/lifelogging
Hoskins, M.: Healthline: newsflash: Dexcom SHARE gets FDA clearance! (2015). http://www.healthline.com/diabetesmine/newsflash-dexcom-share-gets-fda-clearance/
Marshall, S.P.: The index of cognitive activity: measuring cognitive workload. In: IEEE 7 Human Factors Meeting, Scottsdale, Arizona, pp. 7–5–7–9, August 2002
Matsangas, P., McCauley, M.: Sopite syndrome: a revised definition. Aviati. Space Environ. Med. 85, 672–673 (2014)
Matsangas, P., McCauley, M.: Yawning as a behavioral marker of mild motion sickness and sopite syndrome. Aviati. Space Environ. Med. 85, 658–661 (2014)
Matthews, R., McDonald, N., Hervieux, P., Turner, P., Steindorf, M.: A wearable physiological sensor suite for unobtrusive monitoring of physiological and cognitive state. In: 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS 2007, pp. 5276–5281, August 2007
Matthews, R., Turner, P., McDonald, N., Ermolaev, K., Manus, T., Shelby, R., Steindorf, M.: Real time workload classification from an ambulatory wireless eeg system using hybrid eeg electrodes. In: 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS 2008, pp. 5871–5875, August 2008
Mehler, B., Reimer, B., Coughlin, J.: Sensitivity of physiological measures for detecting systematic variations in cognitive demand from a working memory task: An on-road study across three age groups. Hum. Factors 54, 396–412 (2012)
Orbach, T.: Methods and systems for physiological and psycho-physiological monitoring and uses thereof, U.S. Patent App. 11/884,775, 4 September 2008. http://www.google.com/patents/US20080214903
Paulus, M., Stein, M.: Interoception in anxiety and depression. Brain Struct. Funct. 214, 451–463 (2010)
Ryu, K., Ayung, M.: Evaluation of mental workload with a combined measure based on physiological indices during a dual task of tracking and mental arithmetic. Int. J. Industr. Ergon. 35, 991–1009 (2005)
Skinner, A., Russo, C., Baraniecki, L., Maloof, M.: Ubiquitous augmented cognition (2014). http://2014.hci.international/index.php
Swan, M.: Sensor mania! the internet of things, wearable computing, objective metrics, and the quantified self 2.0. J. Sensor. Actuator Netw. 1, 217–253 (2012)
The BodyTrack Team (2015). http://fluxstream.org
Tomaka, J., Blascovitch, J., KIbler, J., Earnst, J.: Cognitive and physiological antecedents of of threat and challenge appraisal. J. Personailty Soc. Psychol. 1, 63–72 (1997)
Weintraub, K.: Quantified self: The tech based route to a better life? (2013). http://www.bbc.com/future/story/20130102-self-track-route-to-a-better-life
Widerhiold, B., Jang, D., Kim, S., Wiederhold, M.D.: Physiological monitoring as an objective tool in virtual reality therapy. Cyberpychology Behav. 5, 77–82 (2002)
Winzce, J., Hoon, P., Soon, E.: Sexual arousal in women: a comparison of cognitive and physiological responses by continuous measurement. Arch. Sex Behav. 2, 121–133 (1977)
Winzce, J., Vendetti, E., Barlow, D., Mavissakalian, M.: The effects of a subjective monitoring task in the physiological measure of genital response to erotic stimulation. Arch. Sex Behav. 9, 533–545 (1980)
Zaki, J., Davis, J., Ochsner, K.: Overlapping activity in anterior insult during interoception and emotional experience. NeuroImage 62, 493–499 (2012)
Editors and Affiliations
Rights and permissions
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Russo, C.R. (2015). The Quantified Self. In: Schmorrow, D.D., Fidopiastis, C.M. (eds) Foundations of Augmented Cognition. AC 2015. Lecture Notes in Computer Science(), vol 9183. Springer, Cham. https://doi.org/10.1007/978-3-319-20816-9_49
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-20815-2
Online ISBN: 978-3-319-20816-9
eBook Packages: Computer ScienceComputer Science (R0)