Skip to main content
SpringerLink
Log in

Experiencer: An Open-Source Context-Sensitive Wearable Experience Sampling Tool

  • Conference paper
  • First Online:
Pervasive Computing Technologies for Healthcare (PH 2022)

Abstract

We introduce Experiencer, a newly developed Experience Sampling Method (ESM) software for commodity-level smartwatches. We designed this software mainly to address the compliance-related challenges, such as dropouts of study participants, that generations of ESM software solutions have faced. Dropouts are often caused by the inconvenient frequency and timing of the ESM prompts. This can partly be mitigated by utilizing physiological smartwatch sensors to learn which prompting moments are both convenient to the study participant and also relevant to the ESM study designer. Experiencer enables researchers to configure context-sensitive sampling protocols, providing access to raw sensor data, within the boundaries of European privacy legislation. In this paper, we describe the technical capabilities of our software, compare its features with the state-of-the-art, and showcase its application in studies that used Experiencer.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Aharony, N., Gardner, A., Sumter, C.: funf | open sensing framework (2021). http://www.funf.org/

  2. Arslan, R.C., Walther, M.P., Tata, C.S.: formr: A study framework allowing for automated feedback generation and complex longitudinal experience-sampling studies using R. Behav. Res. Methods 52(1), 376–387 (2020). https://doi.org/10.3758/s13428-019-01236-y

    Article  Google Scholar 

  3. Barrett, L.F., Barrett, D.J.: An introduction to computerized experience sampling in psychology. Soc. Sci. Comput. Rev. 19(2), 175–185 (2001). https://doi.org/10.1177/089443930101900204

    Article  Google Scholar 

  4. Batalas, N., Markopoulos, P.: Introducing tempest, a modular platform for in situ data collection. In: Proceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through Design, pp. 781–782 (2012)

    Google Scholar 

  5. Batalas, N., aan het Rot, M., Khan, V.J., Markopoulos, P.: Using tempest: End-user programming of web-based ecological momentary assessment protocols. Proceedings ACM Hum. Comput. Interact. 2(EICS), 1–24 (2018)

    Google Scholar 

  6. Becker, E.S., Goetz, T., Morger, V., Ranellucci, J.: The importance of teachers’ emotions and instructional behavior for their students’ emotions-an experience sampling analysis. Teach. Teach. Educ. 43, 15–26 (2014)

    Article  Google Scholar 

  7. van Berkel, N., Ferreira, D., Kostakos, V.: The experience sampling method on mobile devices. ACM Comput. Surv. (CSUR) 50(6), 93:1–93:40 (2017). https://doi.org/10.1145/3123988

  8. van Berkel, N., Goncalves, J., Lovén, L., Ferreira, D., Hosio, S., Kostakos, V.: Effect of experience sampling schedules on response rate and recall accuracy of objective self-reports. Int. J. Hum Comput Stud. 125, 118–128 (2019). https://doi.org/10.1016/j.ijhcs.2018.12.002

    Article  Google Scholar 

  9. Blaauw, F.J., et al.: Let’s get physiqual - an intuitive and generic method to combine sensor technology with ecological momentary assessments. J. Biomed. Inf. 63, 141–149 (2016). https://doi.org/10.1016/j.jbi.2016.08.001

    Article  Google Scholar 

  10. Bolger, N., Davis, A., Rafaeli, E.: Diary methods: capturing life as it is lived. Annu. Rev. Psychol. 54(1), 579–616 (2003)

    Article  Google Scholar 

  11. Carter, S., Mankoff, J., Heer, J.: Momento: support for situated ubicomp experimentation. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 125–134. Association for Computing Machinery (2007). https://doi.org/10.1145/1240624.1240644

  12. Collins, R.L., Kashdan, T.B., Gollnisch, G.: The feasibility of using cellular phones to collect ecological momentary assessment data: application to alcohol consumption. Exp. Clin. Psychopharmacol. 11(1), 73–78 (2003). https://doi.org/10.1037/1064-1297.11.1.73

    Article  Google Scholar 

  13. Csikszentmihalyi, M., Larson, R.: Validity and reliability of the experience-sampling method. In: Flow and the Foundations of Positive Psychology, pp. 35–54. Springer, Dordrecht (2014). https://doi.org/10.1007/978-94-017-9088-8_3

    Chapter  Google Scholar 

  14. Delespaul, P.A.E.G.: Technical note: devices and time-sampling procedures. In: Vries, M.W.d. (ed.) The Experience of Psychopathology: Investigating Mental Disorders in their Natural Settings, pp. 363–374. Cambridge University Press (1992). https://doi.org/10.1017/CBO9780511663246.033

  15. European Parliament: General Data Protection Regulation (GDPR) (2016). https://eur-lex.europa.eu/eli/reg/2016/679/2016-05-04

  16. Evans, B.: Paco-applying computational methods to scale qualitative methods, vol. 2016, no. 1, pp. 348–368 (2016). https://doi.org/10.1111/1559-8918.2016.01095

  17. Ferreira, D., Kostakos, V., Dey, A.K.: AWARE: mobile context instrumentation framework. Front. ICT 2, 6 (2015). https://doi.org/10.3389/fict.2015.00006

    Article  Google Scholar 

  18. Fuller-Tyszkiewicz, M., Skouteris, H., Richardson, B., Blore, J., Holmes, M., Mills, J.: Does the burden of the experience sampling method undermine data quality in state body image research? Body Image 10(4), 607–613 (2013). https://doi.org/10.1016/j.bodyim.2013.06.003

    Article  Google Scholar 

  19. Hafiz, P., Bardram, J.E.: The ubiquitous cognitive assessment tool for smartwatches: design, implementation, and evaluation study. JMIR Mhealth Uhealth 8(6), e17506 (2020). https://doi.org/10.2196/17506

    Article  Google Scholar 

  20. Hernandez, J., McDuff, D., Infante, C., Maes, P., Quigley, K., Picard, R.: Wearable ESM: differences in the experience sampling method across wearable devices. In: Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services, pp. 195–205. MobileHCI 2016, Association for Computing Machinery (2016). https://doi.org/10.1145/2935334.2935340

  21. Hicks, J., Ramanathan, N., Kim, D., Monibi, M., Selsky, J., Hansen, M., Estrin, D.: AndWellness: an open mobile system for activity and experience sampling. In: Wireless Health 2010, pp. 34–43. WH 2010, Association for Computing Machinery (2010). https://doi.org/10.1145/1921081.1921087

  22. Hoeksma, J.B., Sep, S.M., Vester, F.C., Groot, P.F.C., Sijmons, R., De Vries, J.: The electronic mood device: design, construction, and application. Behav. Res. Methods Instrum. Comput. 32(2), 322–326 (2000). https://doi.org/10.3758/BF03207801

    Article  Google Scholar 

  23. Intille, S., Haynes, C., Maniar, D., Ponnada, A., Manjourides, J.: \(\mu \)EMA: Microinteraction-based ecological momentary assessment (EMA) using a smartwatch. In: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp. 1124–1128. UbiComp 2016, Association for Computing Machinery (2016). https://doi.org/10.1145/2971648.2971717

  24. Intille, S.S., Rondoni, J., Kukla, C., Ancona, I., Bao, L.: A context-aware experience sampling tool. In: CHI 2003 Extended Abstracts on Human Factors in Computing Systems. pp. 972–973. CHI EA 2003, Association for Computing Machinery (2003). https://doi.org/10.1145/765891.766101

  25. Kaptein, M., Markopoulos, P., de Ruyter, B., Aarts, E.: Personalizing persuasive technologies: explicit and implicit personalization using persuasion profiles. Int. J. Hum Comput Stud. 77, 38–51 (2015)

    Article  Google Scholar 

  26. Khanshan, A.: Experiencer, the experience sampling method software (2021). https://experiencer.eu/

  27. Khanshan, A.: Experiencer ESM source code (2022). https://github.com/khnshn/Experiencer

  28. Khanshan, A.: From simulation to reality and back again: a hybrid approach to estimate the compliance of ESM study participants to different ESM protocols. In: 14th ACM SIGCHI Symposium on Engineering Interactive Computing Systems Doctoral Consortium, EICS DC 2022. 21–24 June 06 2022 (2022). http://eics.acm.org/eics2022/submission_dc.html

  29. Khanshan, A., Van Gorp, P., Nuijten, R., Markopoulos, P.: Assessing the influence of physical activity upon the experience sampling response rate on wrist-worn devices. Int. J. Environ. Res. Public Health 18(20), 10593 (2021). https://doi.org/10.3390/ijerph182010593

    Article  Google Scholar 

  30. Kheirkhahan, M., et al.: A smartwatch-based framework for real-time and online assessment and mobility monitoring. J. Biomed. Inf. 89, 29–40 (2019). https://doi.org/10.1016/j.jbi.2018.11.003

    Article  Google Scholar 

  31. Kini, S.: Please take my survey: compliance with smartphone-based EMA/ESM studies (2013). https://digitalcommons.dartmouth.edu/senior_theses/83/

  32. Larson, R., Csikszentmihalyi, M.: The experience sampling method. New Dir. Methodol. Soc. Behav. Sci. 15, 41–56 (1983)

    Google Scholar 

  33. Larson, R., Csikszentmihalyi, M.: The experience sampling method. In: Flow and the Foundations of Positive Psychology, pp. 21–34. Springer, Dordrecht (2014). https://doi.org/10.1007/978-94-017-9088-8_2

    Chapter  Google Scholar 

  34. Manini, T.M., et al.: Perception of older adults toward smartwatch technology for assessing pain and related patient-reported outcomes: pilot study. JMIR Mhealth Uhealth 7(3), e10044 (2019). https://doi.org/10.2196/10044

    Article  Google Scholar 

  35. Morren, M., van Dulmen, S., Ouwerkerk, J., Bensing, J.: Compliance with momentary pain measurement using electronic diaries: a systematic review. Eur. J. Pain 13(4), 354–365 (2009). https://doi.org/10.1016/j.ejpain.2008.05.010

    Article  Google Scholar 

  36. Myin-Germeys, I., Birchwood, M., Kwapil, T.: From environment to therapy in psychosis: a real-world momentary assessment approach. Schizophrenia Bull. 37(2), 244–247 (2011). https://doi.org/10.1093/schbul/sbq164

    Article  Google Scholar 

  37. Nuijten, R., et al.: Health promotion through monetary incentives: evaluating the impact of different reinforcement schedules on engagement levels with a mHealth app. Electronics 10(23), 2935 (2021)

    Article  Google Scholar 

  38. Park, S., Constantinides, M., Aiello, L.M., Quercia, D., Van Gent, P.: WellBeat: a framework for tracking daily well-being using smartwatches. IEEE Internet Comput. 24(5), 10–17 (2020). https://doi.org/10.1109/MIC.2020.3017867

    Article  Google Scholar 

  39. Pejovic, V., Lathia, N., Mascolo, C., Musolesi, M.: Mobile-based experience sampling for behaviour research. In: Tkalčič, M., De De Carolis, B., de de Gemmis, M., Odić, A., Košir, A. (eds.) Emotions and Personality in Personalized Services. HIS, pp. 141–161. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-31413-6_8

    Chapter  Google Scholar 

  40. Ponnada, A., Thapa-Chhetry, B., Manjourides, J., Intille, S.: Measuring criterion validity of microinteraction ecological momentary assessment (micro-EMA): Exploratory pilot study with physical activity measurement. JMIR Mhealth Uhealth 9(3), e23391 (2021). https://doi.org/10.2196/23391

    Article  Google Scholar 

  41. Ram, N., Brinberg, M., Pincus, A.L., Conroy, D.E.: The questionable ecological validity of ecological momentary assessment: considerations for design and analysis. Res. Hum. Dev. 14(3), 253–270 (2017). https://doi.org/10.1080/15427609.2017.1340052

    Article  Google Scholar 

  42. Ranjan, Y., et al.: RADAR-base: open source mobile health platform for collecting, monitoring, and analyzing data using sensors, wearables, and mobile devices. JMIR Mhealth Uhealth 7(8), e11734 (2019). https://doi.org/10.2196/11734

    Article  Google Scholar 

  43. Redmiles, E.M., Bodford, J., Blackwell, L.: I just want to feel safe: a diary study of safety perceptions on social media. In: Proceedings of the International AAAI Conference on Web and Social Media, vol. 13, pp. 405–416 (2019)

    Google Scholar 

  44. Rough, D.J., Quigley, A.: End-user development of experience sampling smartphone apps-recommendations and requirements. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 4(2), 1–19 (2020)

    Article  Google Scholar 

  45. Shahrestani, A., Van Gorp, P., Le Blanc, P., Greidanus, F., de Groot, K., Leermakers, J.: Unified health gamification can significantly improve well-being in corporate environments. In: 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 4507–4511 (2017). https://doi.org/10.1109/EMBC.2017.8037858. ISSN: 1558-4615

  46. Shin, D.H., Biocca, F.: Health experience model of personal informatics: the case of a quantified self. Comput. Hum. Behav. 69, 62–74 (2017). https://doi.org/10.1016/j.chb.2016.12.019

    Article  Google Scholar 

  47. Singh, G., Delamare, W., Irani, P.: D-SWIME: a design space for smartwatch interaction techniques supporting mobility and encumbrance. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pp. 1–13. Association for Computing Machinery (2018). https://doi.org/10.1145/3173574.3174208

  48. Stone, A.A., Shiffman, S.: Ecological momentary assessment (EMA) in behavorial medicine. Ann. Behav. Med. 16(3), 199–202 (1994). https://doi.org/10.1093/abm/16.3.199

    Article  Google Scholar 

  49. Timmermann, J., Heuten, W., Boll, S.: Input methods for the borg-RPE-scale on smartwatches. In: 2015 9th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth), pp. 80–83 (2015). https://doi.org/10.4108/icst.pervasivehealth.2015.259220. ISSN: 2153-1641

  50. Trull, T.J., Ebner-Priemer, U.W.: Using experience sampling methods/ecological momentary assessment (ESM/EMA) in clinical assessment and clinical research: Introduction to the special section. - PsycNET (2009). https://doi.org/10.1037/a0017653

  51. Van Gorp, P.: Gamebus - social health games for the entire family (2015). http://www.gamebus.eu

  52. Van Gorp, P., Surendranathan, A., Lesani, Y.: GameBus API guide (2021). http://devdocs.gamebus.eu

  53. Wang, X., et al.: HOPES: an integrative digital phenotyping platform for data collection, monitoring, and machine learning. J. Med. Internet Res. 23(3), e23984 (2021). https://doi.org/10.2196/23984

    Article  Google Scholar 

  54. Wen, C.K.F., Schneider, S., Stone, A.A., Spruijt-Metz, D.: Compliance with mobile ecological momentary assessment protocols in children and adolescents: a systematic review and meta-analysis. J. Med. Internet Res. 19(4), e6641 (2017). https://doi.org/10.2196/jmir.6641

    Article  Google Scholar 

  55. Wheeler, L., Reis, H.T.: Self-recording of everyday life events: origins, types, and uses. J. Pers. 59(3), 339–354 (1991). https://doi.org/10.1111/j.1467-6494.1991.tb00252.x

    Article  Google Scholar 

  56. Zhang, C., Wang, S., Aarts, H., Dastani, M.: Using cognitive models to train warm start reinforcement learning agents for human-computer interactions. arXiv:2103.06160 (2021). arxiv.org/abs/2103.06160

Download references

Acknowledgment

Special recognition to Jory Schoondermark and Barbara Montagne at GGz Centraal, as well as Karin Smolders and Lars Giling at the Eindhoven University of Technology who have contributed to the development of Experiencer through their input and feedback on early versions of Experiencer. Lastly, special appreciation goes to Samsung Nederland for their excellent support on the smartwatches we utilized for our research and development.

Author information

Authors and Affiliations

  1. Department of Industrial Design, Eindhoven Artificial Intelligence Systems Institute, Eindhoven University of Technology, 5612, Eindhoven, AZ, The Netherlands

    Alireza Khanshan & Panos Markopoulos

  2. Department of Industrial Engineering, Eindhoven University of Technology, 5612, Eindhoven, AZ, The Netherlands

    Pieter Van Gorp

Authors
  1. Alireza Khanshan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pieter Van Gorp
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Panos Markopoulos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alireza Khanshan .

Editor information

Editors and Affiliations

  1. University of Edinburgh, Edinburgh, UK

    Athanasios Tsanas

  2. Centre for Research and Technology Hellas, Thessaloniki, Greece

    Andreas Triantafyllidis

Rights and permissions

Reprints and permissions

Copyright information

© 2023 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Khanshan, A., Van Gorp, P., Markopoulos, P. (2023). Experiencer: An Open-Source Context-Sensitive Wearable Experience Sampling Tool. In: Tsanas, A., Triantafyllidis, A. (eds) Pervasive Computing Technologies for Healthcare. PH 2022. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 488. Springer, Cham. https://doi.org/10.1007/978-3-031-34586-9_21

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-34586-9_21

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-34585-2

  • Online ISBN: 978-3-031-34586-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation