The Wearable Multimodal Monitoring System: A Platform to Study Falls and Near-Falls in the Real-World
Falls are particularly detrimental and prevalent in the aging population. To diagnose the cause of a fall current medical practice relies on expensive hospital admissions with many bulky devices that only provide limited diagnostic information. By utilizing the latest wearable technology, the Wearable Multimodal Monitoring System (WMMS) presented here offers a better solution to the problem of fall diagnostics and has the potential to predict these falls in real-time in order to prevent falls or, at least, mitigate their severity. This highly integrated system has been designed for real-life long-term monitoring of movement disorder patients. It contains multiple wearable and wireless biosensors that simultaneously and continuously monitor cardiovascular, autonomic, motor, and electroencephalographic (EEG) activity, in addition to receiving critical patient feedback about symptoms. Initial pilot data show that the system is comfortable and easy to use, and provides high quality data streams capable of detecting near-falls and other motor disturbances.
KeywordsWireless electroencephalography Skin conductance response Electrodermal activation Heart-rate variability Blood pressure Wearability Fall prediction
One in three adults aged 65 and older fall each year . Older adults are hospitalized for fall-related injuries five times more often than they are for injuries from other causes . With the population aging, the number of falls and the related costs will increase. Falls are more frequent in patients with advanced Parkinson’s disease (PD) and they occur at even earlier stages and more frequently in patients with atypical parkinsonian disorders [3, 4, 5]. About half of PD patients who fall will require medical care for fall-related injuries, and many never recover to their pre-fall motor and independence baseline [4, 6].
Current medical practice for diagnosing falls relies on expensive hospital admissions to determine if cardiologic, blood pressure, balance, gait, or seizure disturbances caused a fall. Patients are connected for short periods to bulky, single-function devices that can provide only limited diagnostic information as this information is confined to the hospital setting after a fall has occurred. Currently, advanced technologies may allow using inexpensive and wearable multisensor devices on outpatients to determine the causes of their near-falls and falls as well as collect other critical diagnostic information in a daily life setting.
Only recently has technology evolved to allow scientists to continuously record multiple data streams from the body in everyday life in a comfortable and unobtrusive way. This technology has been used for real-world applications such as stress monitoring , as well as gait and vital sign assessment and fall detection (see review ). The Wearable Multimodal Monitoring System (WMMS) improves upon previous technology by integrating and synchronizing multiple data streams in real-time while also recording valuable patient feedback via a smartphone. The system is an extension of on-going work to build a Multi-Aspect Real-world Integrated Neuroimaging (MARIN) system to study stress [9, 10]. The WMMS takes advantage of the original MARIN system architecture by utilizing some of the same devices, and also includes new devices and mobile applications specifically engineered for the study of falls in movement disorder patients. Our multimodal monitoring system is aimed to diagnose the causes of falls and near-falls so appropriate treatments can be undertaken to prevent subsequent occurrences.
2 The Wearable Multimodal Monitoring System (WMMS)
The Wearable Multimodal Monitoring System (WMMS) is a highly integrated system designed for real-life long-term monitoring of patients susceptible to falls. It uses several state-of-the-art microelectronics and communication technologies in a mobile, wireless data collection and computing platform with multiple, wearable biosensors that simultaneously and continuously monitor cardiovascular, autonomic, motor, and neurological activity in the daily life environment. We chose these modalities as they can capture the most common causes of intrinsic falls unrelated to accidents. In addition, the system requests and receives critical patient feedback about symptoms and other outcome measures. It is envisioned that the data collected by this system will be suitable for the creation of algorithms that can go beyond diagnosis, to the prediction of falls. These algorithms could then be implemented in next-generation systems to alert the patient when conditions and behaviors exist that increase their risk of falling.
Handheld Electronic Device. An Android smartphone serves both as the data hub for the sensors and the graphic user interface for the patient. The WMMS uses several Android-based applications. The main widget provides an event monitoring panel which allows the patient to log notes for salient events and answer related questions. Additionally, the WMMS includes three interactive applications designed to assess symptom severity and a suite of inventories that gauges non-motor functions (i.e., mood).
Event Monitoring Panel. The event monitoring panel is a widget that is available to the patient when the phone is turned on. It includes six different buttons that the patient can select to log salient events (right center of Fig. 1): falls and near-falls, medication, loss of consciousness, meals, dizziness, and tremors or dyskinesias. When selecting a button, the patient is directed to answer questions about the event. Each button follows a pathway of questions designed by clinicians to capture important associated information in a uniform way. This information will be helpful in the development of fall prediction algorithms, because it provides subjective information that adds context to the continuous physiological data streams.
Applications. In order to collect data about symptoms throughout the day, we have developed three interactive apps that can be easily accessed by the patient. The first app is our mobile version of a force transducer based tapping task. The parameters replicate those of a study of Huntington’s Disease patients where outcome measures (variability of tapping intervals and tap frequency) were found to be sensitive enough to distinguish between carriers (pre-manifest) and age-matched healthy controls . This task has also been performed with PD patients with success . The task is performed via a touch screen and does not utilize a force transducer; however, we expect that this mobile setup will still be sensitive enough to quantify motor impairment. We will compare this new app with the Movement Disorder Society-United Parkinson Disease Rating Scale (MDS-UPDRS) tapping score. The second app developed for the WMMS is a Baseline Measurements App that guides the user visually and verbally through a series of movements taken from clinical motor scales. It then creates an output file that time stamps each movement start and end to allow for easy data analysis of the time synced multimodal data streams. The third app is an Everyday Activity App which follows the same design as the Baseline Measurement App, but it asks the user to perform everyday activities such as walking around a room and typing on a keyboard. Both of these movement apps will allow us to explore how the time-synced multimodality data tracks motor function in the clinic and at home during clinically relevant and everyday actions.
Inventory Suite. The Inventory Suite contains questionnaires validated in the literature for the study of non-motor symptoms such as mood. These inventories have been translated to an easy to use mobile format. The suite will be deployed at least once a day, but some shorter inventories will be repeated throughout the day. It contains the Stress Visual Analog Scale, Fatigue Visual Analog Scale, Self-Assessment Manikin, Pittsburgh Sleep Diary, Beck Depression Inventory, and Spielberger State Anxiety Inventory. The data collected from these questionnaires will be used to relate different perceived states with physiological data and will provide potentially important predictive information for falls.
3 Pilot Study Results
To show feasibility and to evaluate the WMMS we collected preliminary data from six healthy young controls (36 ± 9 yrs.) at the U.S. Army Research Laboratory (ARL). Participants spent three hours interacting with the software developed for the WMMS and four hours of their normal work day wearing the system. We also collected preliminary data from three patients and one age-matched healthy control participant (65.2 ± 7.6 yrs.) recruited at the UCSD Movement Disorder Center. These participants interacted with the software and received clinical evaluations over a three hour period under close supervision. Participants at both sites signed informed consent before entering into the study.
3.1 Comfort Ratings/Ease of Use
3.2 Quality of WMMS Data Compared to Medical Instruments
One of the goals of the pilot study held in the clinic was to compare the quality of data collected from the WMMS to data collected from gold-standard medical instruments.
3.3 Capturing Relevant Events
During the time while patients were in the clinic, any relevant events were noted on the smartphone, so that those events were timelocked to the physiological data streams.
4 Fall Prediction Algorithm Development
Successful fall detection algorithms have been developed using acceleration data collected from various parts of the body utilizing techniques such as less sensitive simple threshold based algorithms  or more complex and precise machine learning algorithms  including the use of Bayesian models . While a wide literature has established that basic fall detection is a relatively simple process requiring only acceleration data from a few sites on the body, fall prediction algorithms have not yet been attempted on real-world data. Moreover, to our knowledge diagnosing the multiple causes for a fall has not been attempted. The WMMS offers an integrated view as it will utilize real-world continuous multimodal data streams collected on healthy controls and fall-prone subjects to begin to build prediction profiles for the various causes of falls. These profiles could be used for the prediction and intervention of falls in this population and the ageing population at large. We will use the recently validated definition of near-falls by Maidan et al. : A stumble event or loss of balance that would result in a fall if sufficient recovery mechanisms were not activated. At least two of the following mechanisms should be activated to be determined as a near-fall: (1) unplanned movement of arms or/and legs; (2) unplanned change in stride length, (3) lowering of the center of mass, (4) unplanned change in stride velocity and (5) trunk tilt.
Once fall and near-fall events are identified via user input and/or basic fall detection algorithms, we will then utilize the rich multimodal dataset (including those data obtained through the pathways, questionnaires, and motor apps) to probe for which preceding features are predictive of falls. We will do this by applying recent groundbreaking machine learning techniques [19, 20, 21, 22, 23] to these multimodal data streams. Techniques include but are not limited to fuzzy logic  and Bayesian probability models . The techniques will not only mine physiological and physical activities from multimodal sensors, but also handle context awareness, and subject specific models and personalization. Once an optimal fall prediction algorithm has been created, it will then be deployed in real-time on the smartphone and will trigger questions for the user or an emergency call to the study coordinators or 911 if questions are left unanswered or the patient reports an injury.
This paper has described a novel multimodal system designed to study falls in movement disorder patients. Overall the WMMS functioned well during the short pilot study. Patients and younger healthy adults found the system to be comfortable and easy to use throughout the study. The system provided continuous high-quality datastreams from a variety of users (from young healthy adults to older movement disorder patients). These datastreams were of high enough quality to be comparable to gold-standard medical equipment and to discriminate between events of interest and normal movement patterns. Moving forward, we aim to continue to collect data for the pilot study and to incorporate any advances in technology in order to update the system. For example, we chose the HealthStats BPro to measure blood pressure because it is the best non-cuff semi-continuous blood pressure device available on the market. However, during the pilot study we noted that the device lacked sufficient data collection frequency to accurately measure sudden blood pressure drops. New emerging technology utilizing Pulse Transit Time (PTT) [25, 26] may provide an alternative option. The WMMS employs a very flexible architecture and can be modified to incorporate new sensor technology as it becomes available. With continued improvement and data collection, we can deploy fall detection algorithms and initialize fall prediction algorithm development. We believe the WMMS holds the promise of safer and more independent living, not only for movement disorder patients but also for the ageing population at large.
This research was sponsored by the Army Research Laboratory (ARL) under Cooperative Agreement W911NF-10-2-0022 and the ARL Postdoctoral Fellowship Program administered by Oak Ridge Associated Universities. This material has been reviewed by the Walter Reed Army Institute of Research, and there is no objection to its presentation and/or publication. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Department of the Army of the Department of Defense or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein.
- 10.Doty, T.J., Hairston, W.D., Kellihan, B., Canady, J., Oie, K.S., McDowell, K.: Developing a wearable real-world neuroimaging system to study stress. In: 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER), pp. 786–789 (2013)Google Scholar
- 11.Bechtel, N., Scahill, R.I., Rosas, H.D., Acharya, T., van den Bogaard, S.J.A., Jauffret, C., Say, M.J., Sturrock, A., Johnson, H., Onorato, C.E., Salat, D.H., Durr, A., Leavitt, B.R., Roos, RCa, Landwehrmeyer, G.B., Langbehn, D.R., Stout, J.C., Tabrizi, S.J., Reilmann, R.: Tapping linked to function and structure in premanifest and symptomatic Huntington disease. Neurology 75(24), 2150–2160 (2010)CrossRefGoogle Scholar
- 12.Reilmann, R., Ellerbrock, M., Sass, C., Heger, T., Berg, D., Maetzler, W.: Quantitative motor (Q-Motor) deficits in tapping (digitomotography) distinguish Parkinson’s disease from control subjects and correlate to the UPDRS III-A step towards objective outcomes for motor deficits in clinical trials? Mov. Disord. 28, S174–S175 (2013)Google Scholar
- 17.Zhang, M., Sawchuk, A.A.: Context-aware fall detection using a Bayesian network. In: Proceedings of the 5th ACM International Workshop on Context-Awareness for Self-Managing Systems, pp. 10–16, New York, NY, USA (2011)Google Scholar
- 19.Broccard, F.D., Mullen, T., Chi, Y.M., Peterson, D., Iversen, J.R., Arnold, M., Kreutz-Delgado, K., Jung, T.-P., Makeig, S., Poizner, H., Sejnowski, T., Cauwenberghs, G.: Closed-loop brain–machine–body interfaces for noninvasive rehabilitation of movement disorders. Ann. Biomed. Eng. 42(8), 1573–1593 (2014)CrossRefGoogle Scholar
- 21.Müller, K.-R., Krauledat, M., Dornhege, G., Curio, G., Blankertz, B.: Machine learning and applications for brain-computer interfacing. In: Smith, Michael J., Salvendy, Gavriel (eds.) HCII 2007. LNCS, vol. 4557, pp. 705–714. Springer, Heidelberg (2007)Google Scholar
- 23.Sullivan, T.J., Deiss, S.R., Jung, T.-P., Cauwenberghs, G.: A brain-machine interface using dry-contact, low-noise EEG sensors. In: IEEE International Symposium on Circuits and Systems, ISCAS 2008, pp. 1986–1989 (2008)Google Scholar
- 26.Thomas, S.S., Nathan, V., Zong, C., Akinbola, E., Aroul, A.L.P, Philipose, L., Soundarapandian, K., Shi, X., Jafari, R.: BioWatch #x2014; a wrist watch based signal acquisition system for physiological signals including blood pressure. In: 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 2286–2289 (2014)Google Scholar