Advertisement

Validity of Consumer Activity Wristbands and Wearable EEG for Measuring Overall Sleep Parameters and Sleep Structure in Free-Living Conditions

  • Zilu LiangEmail author
  • Mario Alberto Chapa MartellEmail author
Research Article
Part of the following topical collections:
  1. Special Issue: Computational Health Science

Abstract

Consumer sleep tracking technologies offer an unobtrusive and cost-efficient way to monitor sleep in free-living conditions. Technological advances in hardware and software have significantly improved the functionality of the new gadgets that recently appeared in the market. However, whether the latest gadgets can provide valid measurements on overall sleep parameters and sleep structure such as deep and REM sleep has not been examined. In this study, we aimed to investigate the validity of the latest consumer sleep tracking devices including an activity wristband Fitbit Charge 2 and a wearable EEG-based eye mask Neuroon in comparison to a medical sleep monitor. First, we confirmed that Fitbit Charge 2 can automatically detect the onset and offset of sleep with reasonable accuracy. Second, analysis found that both consumer devices produced comparable results in measuring total sleep duration and sleep efficiency compared to the medical device. In addition, Fitbit accurately measured the number of awakenings, while Neuroon with good signal quality had satisfactory performance on total awake time and sleep onset latency. However, measuring sleep structure including light, deep, and REM sleep remains to be challenging for both consumer devices. Third, greater discrepancies were observed between Neuroon and the medical device in nights with more disrupted sleep and when the signal quality was poor, but no trend was observed in Fitbit Charge 2. This study suggests that current consumer sleep tracking technologies may be immature for diagnosing sleep disorders, but they are reasonably satisfactory for general purpose and non-clinical use.

Keywords

Sleep Fitbit Neuroon Wearable devices EEG Validation Self-tracking mHealth 

Notes

Acknowledgements

This work was supported by JSPS KAKENHI Grant-in-Aid for Research Activity Start-up (Grant Number 16H07469) and an internal research grant from the National Institute of Advanced Industrial Science and Technology of Japan. The authors would like to thank the anonymous reviewers for their valuable comments and suggestions to improve the quality of the paper.

Compliance with Ethical Standards

Conflict of Interest

The authors certify that there is no conflict of interest involved in this manuscript and this study. The opinions expressed in this paper are those of the authors and do not represent the views of the second author’s company.

Ethical Approval

Ethics approval was obtained from the Ethic Committee of the University of Tokyo (Ethics ID: KE16–83). All participants provided informed consent.

References

  1. 1.
    Liang Z, Chapa-Martell MA (2015) Framing self-quantification for individual-level preventive health care. In: Proceedings of the International Conference on Health Informatics, pp 336–343Google Scholar
  2. 2.
    Dittmar A, Axisa F, Delhomme G, Gehin C (2004) New concepts and technologies in home care and ambulatory monitoring. Stud Health Technol Inform 108:9–35Google Scholar
  3. 3.
    Mantua J, Gravel N, Spencer R (2016) Reliability of sleep measures from four personal health monitoring device compared to research-based actigraphy and polysomnography. Sensors 16:646CrossRefGoogle Scholar
  4. 4.
    Meltzer LJ, Hiruma LS, Avis K, Montgomery-Downs H, Valentin J (2015) Comparison of a commercial accelerometer with polysomnography and actigraphy in children and adolescents. Sleep 38(8):1323–1330.  https://doi.org/10.5665/sleep.4918 CrossRefGoogle Scholar
  5. 5.
    De Zambotti M, Baker FC, Colrain IM (2015) Validation of sleep-tracking technology compared with polysomnography in adolescents. Sleep 38(9):1461–1468.  https://doi.org/10.5665/sleep.4990 CrossRefGoogle Scholar
  6. 6.
    De Zambotti M, Godino JG, Baker FC et al (2016) The boom in wearable technology: cause for alarm or just what is needed to better understand sleep? Sleep 39(9):1761–1762.  https://doi.org/10.5665/sleep.6108 CrossRefGoogle Scholar
  7. 7.
    Buysse DJ (2014) Sleep health: can we define it? Does it matter? Sleep 37(1):9–17.  https://doi.org/10.5665/sleep.3298 CrossRefGoogle Scholar
  8. 8.
    Coates TJ, Killen JD, George J, Marchini E, Silverman S, Thoresen C (1982) Estimating sleep parameters: a multitrait-multimethod analysis. J Consult Clin Psychol 50(3):345–352.  https://doi.org/10.1037/0022-006X.50.3.345 CrossRefGoogle Scholar
  9. 9.
    Carskadon MA, Dement WC (2015) Normal human sleep: an overview. In: Kryger MH, Roth T, Dement WC (eds) Principle and practice of sleep medicine, 4th edn. Elsevier Saunders, Philadelphia, pp 13–23Google Scholar
  10. 10.
    Hall M (2010) Behavioral medicine and sleep: concept, measures and methods. In: Steptoe A (ed) Handbook of behavioral medicine: methods and applications. Springer, New York, pp 749–765.  https://doi.org/10.1007/978-0-387-09488-5_49 CrossRefGoogle Scholar
  11. 11.
    Buysse DJ, Reynolds CF, Monk TH, Berman SR, Kupfer DJ (1989) The Pittsburgh sleep quality index: a new instrument for psychiatric practice and research. Psychiatry Res 28(2):193–213.  https://doi.org/10.1016/0165-1781(89)90047-4 CrossRefGoogle Scholar
  12. 12.
    Carney CE, Buysse DJ, Ancoli-Israel S, Edinger JD, Krystal AD, Lichstein KL, Morin CM (2012) The consensus sleep diary: standardizing prospective sleep self-monitoring. Sleep 35(2):287–302.  https://doi.org/10.5665/sleep.1642 CrossRefGoogle Scholar
  13. 13.
    Hublin C, Partinen M, Koskenvuo M, Kaprio J (2007) Sleep and mortality: a population-based 22-year follow-up study. Sleep 30(10):1245–1253.  https://doi.org/10.1093/sleep/30.10.1245 CrossRefGoogle Scholar
  14. 14.
    Kushida CA, Littner MR, Morgenthaler T, Alessi CA, Bailey D, Coleman J Jr, Friedman L, Hirshkowitz M, Kapen S, Kramer M, Lee-Chiong T, Loube DL, Owens J, Pancer JP, Wise M (2005) Practice parameters for the indications for polysomnography and related procedures: an update for 2005. Sleep 28(4):499–519.  https://doi.org/10.1093/sleep/28.4.499 CrossRefGoogle Scholar
  15. 15.
    Ohayon M, Wickwire EM, Hirshkowitz M, Albert SM, Avidan A, Daly FJ, Dauvilliers Y, Ferri R, Fung C, Gozal D, Hazen N, Krystal A, Lichstein K, Mallampalli M, Plazzi G, Rawding R, Scheer FA, Somers V, Vitiello MV (2017) National sleep foundation’s sleep quality recommendations: first report. Sleep Health 3(1):6–19.  https://doi.org/10.1016/j.sleh.2016.11.006 CrossRefGoogle Scholar
  16. 16.
    Hirshkowitz M (2004) Normal human sleep: an overview. Med Clin North Am 88:51–65CrossRefGoogle Scholar
  17. 17.
    Keenan SA (1999) Normal human sleep. Respir Care Clin N Am 5:319–331Google Scholar
  18. 18.
    Sadeh A (2011) The role and validity of actigraphy in sleep medicine: an update. Sleep Med Rev 15(4):259–267.  https://doi.org/10.1016/j.smrv.2010.10.001 CrossRefGoogle Scholar
  19. 19.
    Ancoli-Israel S, Cole R, Alessi C, Chambers M, Moorcroft W, Pollak C (2003) The role of actigraphy in the study of sleep and circadian rhythms. American Academy of sleep medicine review paper. Sleep 26(3):342–392.  https://doi.org/10.1093/sleep/26.3.342 CrossRefGoogle Scholar
  20. 20.
    Littner M, Kushida CA, Anderson WM et al (2002) Practice parameters for the role of actigraphy in the study of sleep and circadian rhythms: an update for 2002. Sleep 26(3):337–341CrossRefGoogle Scholar
  21. 21.
    Vitiello MV, Larsen LH, Drolet G et al (2002) Gender differences in subjective-objective sleep relationships in non-complaining healthy older adults. Sleep 25S:A61Google Scholar
  22. 22.
    Van Ravesteyn LM, Tulen JH, Kamperman AM et al (2014) Perceived sleep quality is worse than objective parameters of sleep in pregnant women with a mental disorder. J Clin Sleep Med 10(10):1137–1141.  https://doi.org/10.5664/jcsm.4118 Google Scholar
  23. 23.
    Lund HG, Rybarczyk BD, Perrin PB, Leszczyszyn D, Stepanski E (2013) The discrepancy between subjective and objective measures of sleep in older adults receiving CBT for comorbid insomnia. J Clin Psychol 69(10):1108–1120.  https://doi.org/10.1002/jclp.21938 CrossRefGoogle Scholar
  24. 24.
    Most EIS, Aboudan S, Scheltens P, van Someren EJW (2012) Discrepancy between subjective and objective sleep disturbances in early- and moderate-stage Alzheimer disease. Am J Geriatr Psychiatry 20(6):460–467.  https://doi.org/10.1097/JGP.0b013e318252e3ff CrossRefGoogle Scholar
  25. 25.
    O’Donnel D, Silva EJ, Munch M et al (2009) Comparison of subjective and objective assessments of sleep in healthy older subjectis without sleep complaints. J Sleep Res 18(2):254–263.  https://doi.org/10.1111/j.1365-2869.2008.00719.x CrossRefGoogle Scholar
  26. 26.
    Tsuchiyama K, Nagayama H, Kudo K, Kojima K, Yamada K (2003) Discrepancy between subjective and objective sleep in patients with depression. Psychiatry Clin Neurosci 57(3):259–264.  https://doi.org/10.1046/j.1440-1819.2003.01114.x CrossRefGoogle Scholar
  27. 27.
    Kolla BP, Mansukhani S, Mansukhani MP (2016) Consumer sleep tracking devices: a review of mechanisms, validity and utility. Expert Rev Med Devices 13(5):497–506.  https://doi.org/10.1586/17434440.2016.1171708 CrossRefGoogle Scholar
  28. 28.
    Shelgikar AV, Anderson PF, Stephens MR (2016) Sleep tracking, wearable technology, and opportunities for research and clinical care. Chest 150(3):732–743.  https://doi.org/10.1016/j.chest.2016.04.016 CrossRefGoogle Scholar
  29. 29.
    Ong AA, Gillespie MB (2016) Overview of smartphone applications for sleep analysis. World J Otorhinolaryngol Head Neck Surg 2(1):45–49.  https://doi.org/10.1016/j.wjorl.2016.02.001 CrossRefGoogle Scholar
  30. 30.
    Ko PT, Kientz JA, Choe EK, Kay M, Landis CA, Watson NF (2015) Consumer sleep technologies: a review of the landscape. J Clin Sleep Med 11(12):1455–1461.  https://doi.org/10.5664/jcsm.5288 CrossRefGoogle Scholar
  31. 31.
    Kelly JM, Strecker RE, Bianchi MT (2012) Recent developments in home sleep-monitoring devices. International Scholarly Research Network Neurology, Article ID 768794, 10 pagesGoogle Scholar
  32. 32.
    Montgomery-Downs HE, Insana SP, Bond JA (2012) Movement toward a novel activity monitoring device. Sleep Breath 6(3):913–917CrossRefGoogle Scholar
  33. 33.
    De Zambotti M, Claudatos S, Inkelis S et al (2015) Evaluation of a consumer fitness-tracking device to access sleep in adults. Chronobiol Int 32(7):1024–1028.  https://doi.org/10.3109/07420528.2015.1054395 CrossRefGoogle Scholar
  34. 34.
    Bhat S, Ferraris A, Gupta D, Mozafarian M, DeBari VA, Gushway-Henry N, Gowda SP, Polos PG, Rubinstein M, Seidu H, Chokroverty S (2015) Is there a clinical role for smartphone sleep apps? Comparison of sleep cycle detection by a smartphone application to polysomnography. J Clin Sleep Med 11(7):709–715.  https://doi.org/10.5664/jcsm.4840 Google Scholar
  35. 35.
    Shirazi AS, Clawson J, Hassanpour Y, Tourian MJ et al (2013) Already up? Using mobile phones to track & share sleep behavior. Int J Hum Comput Stud 71(9):878–888.  https://doi.org/10.1016/j.ijhcs.2013.03.001 CrossRefGoogle Scholar
  36. 36.
    Behar J, Roebuck A, Shahid M, Daly J, Hallack A, Palmius N, Stradling J, Clifford GD (2015) SleepAp: an automated obstructive sleep apnoea scorning application for smartphones. IEEE J Biomed Health Inform 19(1):325–331.  https://doi.org/10.1109/JBHI.2014.2307913 CrossRefGoogle Scholar
  37. 37.
    Liang Z, Ploderer B (2016) Sleep tracking in the real world: a qualitative study into barriers for improving sleep. In: Proceedings of OZCHI 2016: 537–541Google Scholar
  38. 38.
    Liang Z, Ploderer B, Chapa-Martell MA (2017) Is Fitbit fit for sleep-tracking? Sources of measurement errors and proposed countermeasures. In: Proceedings of Pervasive Health 2017Google Scholar
  39. 39.
    Natale V, Leger D, Martoni M et al (2014) The role of actigraphy in the assessment of primary insomnia: a retrospective study. Sleep Med 15(1):111–115.  https://doi.org/10.1016/j.sleep.2013.08.792 CrossRefGoogle Scholar
  40. 40.
    Suh S, Nowakowski S, Bernert RA, Ong JC, Siebern AT, Dowdle CL, Manber R (2012) Clinical significance of night-to-night sleep variability in insomnia. Sleep Med 13(5):469–475.  https://doi.org/10.1016/j.sleep.2011.10.034 CrossRefGoogle Scholar
  41. 41.
    Buysse DJ, Cheng Y, Germain A, Moul DE, Franzen PL, Fletcher M, Monk TH (2010) Night-to-night sleep variability in older adults with and without chronic insomnia. Sleep Med 11(1):56–64.  https://doi.org/10.1016/j.sleep.2009.02.010 CrossRefGoogle Scholar
  42. 42.
    Natale V, Plazzi G, Martoni M (2009) Actigraphy in the assessment of insomnia: a quantitative approach. Sleep 32(6):767–771.  https://doi.org/10.1093/sleep/32.6.767 CrossRefGoogle Scholar
  43. 43.
  44. 44.
  45. 45.
    Yoshida M, Shinohara H, Kodama H (2015) Assessment of nocturnal sleep architecture by actigraphy and one-channel electroencephalography in early infancy. Early Hum Dev 91(9):519–526.  https://doi.org/10.1016/j.earlhumdev.2015.06.005 CrossRefGoogle Scholar
  46. 46.
    Rosenberg RS, Van Hout S (2013) The American Academy of Sleep Medicine inter-scorer reliability program: sleep stage scoring. J Clin Sleep Med 9(1):81–87.  https://doi.org/10.5664/jcsm.2350 Google Scholar
  47. 47.
    Liang Z, Ploderer B, Liu W, Nagata Y, Bailey J, Kulik L, Li X (2016) SleepExplorer: a visualization tool to make sense of correlations between personal sleep data and contextual factors. Pers Ubiquit Comput 20(6):985–1000.  https://doi.org/10.1007/s00779-016-0960-6 CrossRefGoogle Scholar
  48. 48.
    Iber C, Ancoli-Israel S, Chesson A et al (2007) The AASM manual for the scoring of sleep and associated events: rules, terminology and technical specifications, 1st edn. American Academy of Sleep Medicine, WestchesterGoogle Scholar
  49. 49.
    McCall C, McCall WV (2012) Objective vs. subjective measurements of sleep in depressed insomniacs: first night effect or reverse first night effect? J Clin Sleep Med 8(1):59–65Google Scholar
  50. 50.
    Ahmadi N, Shapiro GK, Chung SA, Shapiro CM (2009) Clinical diagnosis of sleep apnea based on single night of polysomnography vs. two nights of polysomnography. Sleep Breath 13(3):221–226.  https://doi.org/10.1007/s11325-008-0234-2 CrossRefGoogle Scholar
  51. 51.
    Tworoger SS, Davis S, Vitiello MV, Lentz MJ, McTiernan A (2005) Factors associated with objective (actigraphic) and subjective sleep quality in young adult women. J Psychosom Res 59(1):11–19.  https://doi.org/10.1016/j.jpsychores.2005.03.008 CrossRefGoogle Scholar
  52. 52.
    Evenson KR, Goto MM, Furberg RD (2015) Systematic review of the validity and reliability of consumer-wearable activity trackers. Int J Behav Nutr Phys Act 12(1):159.  https://doi.org/10.1186/s12966-015-0314-1 CrossRefGoogle Scholar
  53. 53.
    Higgins PA, Straub AJ (2006) Understanding the error of our ways: mapping the concepts of validity and reliability. Nurs Outlook 54(1):23–29.  https://doi.org/10.1016/j.outlook.2004.12.004 CrossRefGoogle Scholar
  54. 54.
    Benjamini Y (1988) Opening the box of a boxplot. Am Stat 42(4):257–262Google Scholar
  55. 55.
    Wilcoxon F (1945) Individual comparisons by ranking methods. Biom Bull 1(6):80–83.  https://doi.org/10.2307/3001968 CrossRefGoogle Scholar
  56. 56.
    Fay MP, Proschan MA (2010) Wilcoson-Mann-Whitney or t-test? On assumptions for hypothesis tests and multiple interpretations of decision rules. Stat Surv 4(0):1–39.  https://doi.org/10.1214/09-SS051 MathSciNetCrossRefzbMATHGoogle Scholar
  57. 57.
    Bradley JV (1968) Distribution-free statistical tests. Prentice-HallGoogle Scholar
  58. 58.
    Bland JM, Altman DG (1986) Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1(8476):307–310CrossRefGoogle Scholar
  59. 59.
    Meltzer L, Walsh C, Traylor J, Westin A (2012) Direct comparison of two new actigraphs and polysomnography in children and adolescents. Sleep 35(1):159–166.  https://doi.org/10.5665/sleep.1608 Google Scholar
  60. 60.
    Werner H, Molinari L, Guyer C, Jenni OG (2008) Agreement rates between actigraphy, diary, and questionnaire for children’s sleep patterns. Arch Pediatr Adolesc Med 162(4):350–358.  https://doi.org/10.1001/archpedi.162.4.350 CrossRefGoogle Scholar
  61. 61.
    Rosenberger ME, Buman MP, Haskell WL et al (2015) 24h or sleep, sedentary behavior, and physical activity with nine wearable devices. Med Sci Sports Exerc 48(3):457–465CrossRefGoogle Scholar
  62. 62.
    Kappenman ES, Luck SJ (2010) The effects of electrode impedance on data quality and statistical significance in ERP recordings. Psychophysiology 47(5):888–904.  https://doi.org/10.1111/j.1469-8986.2010.01009.x Google Scholar
  63. 63.
    Duun-Henriksen J, Kjaer TW, Looney D, et al. (2015) EEG signal quality of a subcutaneous recording system compared to standard surface electrodes. J Sensors 2015: Article 341208, 9 pagesGoogle Scholar
  64. 64.
    Baroni A, Bruzzese JM, Di Bartolo CA, Shatkin JP (2016) Fitbit Flex: an unreliable device for longitudinal sleep measures in a non-clinical population. Sleep Breath 20(2):853–854.  https://doi.org/10.1007/s11325-015-1271-2 CrossRefGoogle Scholar
  65. 65.
    Ferguson T, Rowlands AV, Olds T, Maher C (2015) The validity of consumer-level, activity monitors in healthy adults worn in free-living conditions: a cross-sectional study. Int J Behav Nutr Phys Act 12(1):42.  https://doi.org/10.1186/s12966-015-0201-9 CrossRefGoogle Scholar
  66. 66.
    Neuroon Open product overview. http://community.neuroonopen.com/product
  67. 67.
    Hein M, Lanquart JP, Loas G, Hubain P, Linkowski P (2017) Similar polysomnographic pattern in primary insomnia and major depression with objective insomnia: a sign of common pathophysiology? BMC Psychiatry 17(1):273.  https://doi.org/10.1186/s12888-017-1438-4 CrossRefGoogle Scholar
  68. 68.
    Lucey BP, McLeland JS, Toedebusch CD et al (2016) Comparison of a single-channel EEG sleep study to polysomnography. J Sleep Res 25(6):625–635.  https://doi.org/10.1111/jsr.12417 CrossRefGoogle Scholar
  69. 69.
    De Zambotti M, Baker FC, Willoughby AR et al (2016) Measures of sleep and cardiac functioning during sleep using a multi-sensory commercially-available wristband in adolescents: wearable technology to measure sleep and cardiac functioning. Physiol Behav 158:143–149CrossRefGoogle Scholar
  70. 70.
    Taibi DM, Landis CA, Vitiello MV (2013) Concordance of polysomnographic and actigraphic measurement of sleep and wake in older women with insomnia. J Clin Sleep Med 9(3):217–225.  https://doi.org/10.5664/jcsm.2482 Google Scholar
  71. 71.
    Blackwell T, Redline S, Ancoli-Isreal S et al (2008) Comparison of sleep parameters from actigraphy and polysomnography in older women: the SOF study. Sleep 31(2):283–291.  https://doi.org/10.1093/sleep/31.2.283 CrossRefGoogle Scholar
  72. 72.
    Cellini N, Buman MP, McDevitt EA et al (2013) Direct comparison of two actigrapy devices with polysomnographically recorded naps in healthy young adults. Chronobiol Int 30(5):691–698.  https://doi.org/10.3109/07420528.2013.782312 CrossRefGoogle Scholar
  73. 73.
    Kosecki D (2017) Your heart rate is the key to smarter sleep stages. Here’s why. Fitbit News, https://blog.fitbit.com/heart-rate-during-sleep-stages/

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.The University of TokyoTokyoJapan
  2. 2.CAC CorporationTokyoJapan

Personalised recommendations