Skip to main content
SpringerLink
Log in

Recommendations for Conducting Longitudinal Experience Sampling Studies

  • Chapter
  • First Online:
Advances in Longitudinal HCI Research

Part of the book series: Human–Computer Interaction Series ((HCIS))

Abstract

The Experience Sampling Method is used to collect participant self-reports over extended observation periods. These self-reports offer a rich insight into the individual lives of study participants by intermittently asking participants a set of questions. However, the longitudinal and repetitive nature of this sampling approach introduces a variety of concerns regarding the data contributed by participants. A decrease in participant interest and motivation may negatively affect study adherence, as well as potentially affecting the reliability of participant data. In this chapter, we reflect on a number of studies that aim to understand better participant performance with Experience Sampling. We discuss the main issues relating to participant data for longitudinal studies and provide hands-on recommendations for researchers to remedy these concerns in their own studies.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Larson R, Csikszentmihalyi M (1983) The experience sampling method. New Directions Methodol Soc Behav Sci

    Google Scholar 

  2. Csikszentmihalyi M, Larson R (1987) Validity and reliability of the Experience-Sampling method. J Nerv Ment Dis 175:526–536

    Article  Google Scholar 

  3. Csikszentmihalyi M, Larson R, Prescott S (1977) The ecology of adolescent activity and experience. J Youth Adolescence 6:281–294

    Article  Google Scholar 

  4. van Berkel N, Ferreira D, Kostakos V (2017) The experience sampling method on mobile devices. ACM Comput Surv 50(6):93:1–93:40

    Google Scholar 

  5. Smyth JM, Stone AA (2003) Ecological momentary assessment research in behavioral medicine. J Happiness Stud 4:35–52

    Article  Google Scholar 

  6. Hsieh G, Li I, Dey A, Forlizzi J, Hudson SE (2008) Using visualizations to increase compliance in experience sampling. In: Proceedings of the 10th international conference on ubiquitous computing, UbiComp ’08, (New York, NY, USA). ACM, pp 164–167

    Google Scholar 

  7. van Berkel N, Goncalves J, Hosio S, Kostakos V (2017) Gamification of mobile experience sampling improves data quality and quantity. In: Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies (IMWUT), vol 1, no 3, pp 107:1–107:21

    Google Scholar 

  8. Raento M, Oulasvirta A, Eagle N (2009) Smartphones: an emerging tool for social scientists. Sociol Methods Res 37(3):426–454

    Article  MathSciNet  Google Scholar 

  9. Ferreira D, Kostakos V, Schweizer I (2017) Human sensors on the move. Springer International Publishing, pp 9–19

    Google Scholar 

  10. van Berkel N, Goncalves J, Wac K, Hosio S, Cox AL (2020) Human accuracy in mobile data collection. Int J Hum-Comput Stud, p 102396

    Google Scholar 

  11. Westerink J, Ouwerkerk M, de Vries G, de Waele S, van den Eerenbeemd J, van Boven M (2009) Emotion measurement platform for daily life situations. In: 3rd international conference on affective computing and intelligent interaction and workshops, pp 1–6

    Google Scholar 

  12. Guha S, Wicker SB (2015) Spatial subterfuge: an experience sampling study to predict deceptive location disclosures. In: Proceedings of the 2015 ACM international joint conference on pervasive and ubiquitous computing, UbiComp ’15, (New York, NY, USA). Association for Computing Machinery, pp 1131–1135

    Google Scholar 

  13. Heron KE, Smyth JM (2010) Ecological momentary interventions: incorporating mobile technology into psychosocial and health behaviour treatments. Brit J Health Psychol 15(1):1–39

    Article  Google Scholar 

  14. Shaughnessy JJ, Zechmeister EB, Zechmeister JS (2011) Research methods in psychology. McGraw-Hill, New York

    Google Scholar 

  15. Armantier O, Topa G, Van der Klaauw W, Zafar B (2017) An overview of the survey of consumer expectations. Econ Policy Rev 23–2:51–72

    Google Scholar 

  16. Stein RE, Horwitz SM, Storfer-Isser A, Heneghan A, Olson L, Hoagwood KE (2008) Do pediatricians think they are responsible for identification and management of child mental health problems? Results of the AAP periodic survey. Ambulatory Pediatr 8(1):11–17

    Article  Google Scholar 

  17. van Berkel N, Budde M, Wijenayake S, Goncalves J (2018) Improving accuracy in mobile human contributions: an overview. In: Adjunct proceedings of the ACM international joint conference on pervasive and ubiquitous computing, pp 594–599

    Google Scholar 

  18. van Berkel N (2019) Data quality and quantity in mobile experience sampling. Phd thesis, The University of Melbourne

    Google Scholar 

  19. Hektner JM, Schmidt JA, Csikszentmihalyi M (2007) Experience sampling method: measuring the quality of everyday life. Sage

    Google Scholar 

  20. van Berkel N, Goncalves J, Hosio S, Sarsenbayeva Z, Velloso E, Kostakos V (2020) Overcoming compliance bias in self-report studies: a cross-study analysis. Int J Hum-Comput Stud 134:1–12

    Article  Google Scholar 

  21. Reiss S (2012) Intrinsic and extrinsic motivation. Teach Psychol 39(2):152–156

    Google Scholar 

  22. Eveleigh A, Jennett C, Blandford A, Brohan P, Cox AL (2014) Designing for dabblers and deterring drop-outs in citizen science. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI ’14, (New York, NY, USA). ACM, pp 2985–2994

    Google Scholar 

  23. Rotman D, Preece J, Hammock J, Procita K, Hansen D, Parr C, Lewis D, Jacobs D (2012) Dynamic changes in motivation in collaborative citizen-science projects. In: Proceedings of the ACM 2012 conference on computer supported cooperative work, CSCW ’12, (New York, NY, USA). ACM, pp 217–226

    Google Scholar 

  24. Measham TG, Barnett GB (2008) Environmental volunteering: motivations, modes and outcomes. Australian Geographer 39(4):537–552

    Article  Google Scholar 

  25. Deci E, Ryan RM (1985) Intrinsic motivation and self-determination in human behavior. Springer, Berlin

    Book  Google Scholar 

  26. Consolvo S, Walker M (2003) Using the experience sampling method to evaluate ubicomp applications. IEEE Pervas Comput 2:24–31

    Article  Google Scholar 

  27. Musthag M, Raij A, Ganesan D, Kumar S, Shiffman S (2011) Exploring micro-incentive strategies for participant compensation in high-burden studies. In: Proceedings of the 13th international conference on ubiquitous computing, UbiComp ’11, (New York, NY, USA). ACM, pp 435–444

    Google Scholar 

  28. Stone AA, Kessler RC, Haythomthwatte JA (1991) Measuring daily events and experiences: decisions for the researcher. J Personal 59(3):575–607

    Article  Google Scholar 

  29. Shih F, Liccardi I, Weitzner D (2015) Privacy tipping points in smartphones privacy preferences. In: Proceedings of the 33rd annual ACM conference on human factors in computing systems, CHI ’15, (New York, NY, USA). Association for Computing Machinery, pp 807–816

    Google Scholar 

  30. Tollmar K, Huang C (2015) Boosting mobile experience sampling with social media. In: Proceedings of the 17th international conference on human-computer interaction with mobile devices and services, MobileHCI ’15, (New York, NY, USA). Association for Computing Machinery, pp 525–530

    Google Scholar 

  31. van Berkel N, Goncalves J, Lovén L, Ferreira D, Hosio S, Kostakos V (2019) Effect of experience sampling schedules on response rate and recall accuracy of objective self-reports. Int J Hum-Comput Stud 125:118–128

    Article  Google Scholar 

  32. Eisele G, Vachon H, Lafit G, Kuppens P, Houben M, Myin-Germeys I, Viechtbauer W (2020) The effects of sampling frequency and questionnaire length on perceived burden, compliance, and careless responding in experience sampling data in a student population

    Google Scholar 

  33. Wheeler L, Reis HT (1991) Self-recording of everyday life events: origins, types, and uses. J Personal 59(3):339–354

    Article  Google Scholar 

  34. Barrett LF, Barrett DJ (2001) An introduction to computerized experience sampling in psychology. Soc Sci Comput Rev 19(2):175–185

    Article  Google Scholar 

  35. Bolger N, Davis A, Rafaeli E (2003) Diary methods: capturing life as it is lived. Ann Rev Psychol 54(1):579–616

    Article  Google Scholar 

  36. Kapoor A, Horvitz E (2008) Experience sampling for building predictive user models: a comparative study. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI ’08, (New York, NY, USA). Association for Computing Machinery, pp 657–666

    Google Scholar 

  37. Rosenthal S, Dey AK, Veloso M (2011) Using decision-theoretic experience sampling to build personalized mobile phone interruption models. In: Lyons K, Hightower J, Huang EM (eds) Pervasive computing. Springer, Berlin, Heidelberg, pp 170–187

    Chapter  Google Scholar 

  38. Mehrotra A, Vermeulen J, Pejovic V, Musolesi V (2015) Ask, but don’t interrupt: the case for interruptibility-aware mobile experience sampling. In: Adjunct proceedings of the 2015 ACM international joint conference on pervasive and ubiquitous computing and proceedings of the 2015 ACM international symposium on wearable computers, UbiComp/ISWC’15 Adjunct, (New York, NY, USA). Association for Computing Machinery, pp 723–732

    Google Scholar 

  39. van Berkel N, Goncalves J, Koval P, Hosio S, Dingler T, Ferreira D, Kostakos V (2019) Context-informed scheduling and analysis: improving accuracy of mobile self-reports. In: Proceedings of ACM SIGCHI conference on human factors in computing systems, pp 51:1–51:12

    Google Scholar 

  40. Lathia N, Rachuri KK, Mascolo C, Rentfrow PJ (2013) Contextual dissonance: design bias in sensor-based experience sampling methods. In: Proceedings of the 2013 ACM international joint conference on pervasive and ubiquitous computing, UbiComp ’13, (New York, NY, USA). Association for Computing Machinery, pp 183–192

    Google Scholar 

  41. Carstensen LL, Turan B, Scheibe S, Ram N, Ersner-Hershfield H, Samanez-Larkin GR, Brooks KP, Nesselroade JR (2011) Emotional experience improves with age: evidence based on over 10 years of experience sampling. Psychol Aging 26(1):21–33

    Article  Google Scholar 

  42. Savin-Williams RC, Demo DH (1983) Situational and transituational determinants of adolescent self-feelings. J Personal Soc Psychol 44(4):824

    Article  Google Scholar 

  43. Bailon C, Damas M, Pomares H, Sanabria D, Perakakis P, Goicoechea C, Banos O (2019) Smartphone-based platform for affect monitoring through flexibly managed experience sampling methods. Sensors 19(15):3430

    Article  Google Scholar 

  44. Berrocal A, Wac K (2018) Peer-vasive computing: leveraging peers to enhance the accuracy of self-reports in mobile human studies. In: Proceedings of the 2018 ACM international joint conference and 2018 international symposium on pervasive and ubiquitous computing and wearable computers. ACM, pp 600–605

    Google Scholar 

  45. Chang Y-L, Chang Y-J, Shen C-Y (2019) She is in a bad mood now: leveraging peers to increase data quantity via a chatbot-based ESM. In: Proceedings of the 21st international conference on human-computer interaction with mobile devices and services, MobileHCI ’19 (New York, NY, USA). Association for Computing Machinery

    Google Scholar 

  46. Solymosi R, Bowers K, Fujiyama T (2015) Mapping fear of crime as a context-dependent everyday experience that varies in space and time. Legal Criminol Psychol 20(2):193–211

    Article  Google Scholar 

  47. Kittur A, Chi EH, Suh B (2008) Crowdsourcing user studies with mechanical turk. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI ’08, (New York, NY, USA). Association for Computing Machinery, pp 453–456

    Google Scholar 

  48. Oleson D, Sorokin A, Laughlin G, Hester V, Le J, Biewald L (2011) Programmatic gold: targeted and scalable quality assurance in crowdsourcing. In: Workshops at the Twenty-Fifth AAAI conference on artificial intelligence

    Google Scholar 

  49. Rabbi M, Li K, Yan HY, Hall K, Klasnja P, Murphy S (2019) Revibe: a context-assisted evening recall approach to improve self-report adherence. In: Proceedings of the ACM Interaction Mobile Wearable Ubiquitous Technology, vol 3

    Google Scholar 

  50. Intille S, Kukla C, Ma X (2002) Eliciting user preferences using image-based experience sampling and reflection. In: CHI ’02 extended abstracts on human factors in computing systems, CHI EA ’02, (New York, NY, USA). Association for Computing Machinery, pp 738–739

    Google Scholar 

  51. Yue Z, Litt E, Cai CJ, Stern J, Baxter KK, Guan Z, Sharma N, Zhang GG (2014) Photographing information needs: the role of photos in experience sampling method-style research. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI ’14, (New York, NY, USA). Association for Computing Machinery, pp 1545–1554

    Google Scholar 

  52. Sprangers MA, Schwartz CE (1999) Integrating response shift into health-related quality of life research: a theoretical model. Soc Sci Med 48(11):1507–1515

    Article  Google Scholar 

  53. Schwartz CE, Sprangers MA, Carey A, Reed G (2004) Exploring response shift in longitudinal data. Psychol Health 19(1):51–69

    Article  Google Scholar 

  54. Ring L, Höfer S, Heuston F, Harris D, O’Boyle CA (2005) Response shift masks the treatment impact on patient reported outcomes (PROs): the example of individual quality of life in edentulous patients. Health Qual Life Outcomes 3(1):55

    Article  Google Scholar 

  55. Schwartz CE, Sprangers MA (2010) Guidelines for improving the stringency of response shift research using the thentest. Qual Life Res 19(4):455–464

    Article  Google Scholar 

  56. Christensen TC, Barrett LF, Bliss-Moreau E, Lebo K, Kaschub C (2003) A practical guide to experience-sampling procedures. J Happiness Stud 4(1):53–78

    Article  Google Scholar 

  57. Paruthi G, Raj S, Gupta A, Huang C-C, Chang Y-J, Newman MW (2017) Heed: situated and distributed interactive devices for self-reporting. In: Proceedings of the 2017 ACM international joint conference on pervasive and ubiquitous computing and proceedings of the 2017 ACM international symposium on wearable computers, UbiComp ’17, (New York, NY, USA). Association for Computing Machinery, pp 181–184

    Google Scholar 

  58. Gong Q, He X, Xie Q, Lin S, She G, Fang R, Han R, Chen Y, Xiao Y, Fu X et al (2018) LBSLAB: a user data collection system in mobile environments. In: Proceedings of the 2018 ACM international joint conference and 2018 international symposium on pervasive and ubiquitous computing and wearable computers, UbiComp ’18, (New York, NY, USA). Association for Computing Machinery, pp 624–629

    Google Scholar 

  59. Adams AT, Murnane EL, Adams P, Elfenbein M, Chang PF, Sannon S, Gay G, Choudhury T (2018) Keppi: a tangible user interface for self-reporting pain. In: Proceedings of the 2018 CHI conference on human factors in computing systems, CHI ’18 (New York, NY, USA). Association for Computing Machinery

    Google Scholar 

  60. Hernandez J, McDuff D, Infante C, Maes P, Quigley K, Picard R (2016) Wearable ESM: differences in the experience sampling method across wearable devices. In: Proceedings of the 18th international conference on human-computer interaction with mobile devices and services, MobileHCI ’16, (New York, NY, USA). Association for Computing Machinery, pp 195–205

    Google Scholar 

  61. Liono J, Salim FD, van Berkel N, Kostakos V, Qin AK (2019) Improving experience sampling with multi-view user-driven annotation prediction. In: IEEE international conference on pervasive computing and communications (PerCom, pp 1–11

    Google Scholar 

  62. Pielot M, Vradi A, Park S (2018) Dismissed! a detailed exploration of how mobile phone users handle push notifications. In: Proceedings of the 20th international conference on human-computer interaction with mobile devices and services, MobileHCI ’18 (New York, NY, USA). Association for Computing Machinery

    Google Scholar 

  63. Visuri A, van Berkel N, Okoshi T, Goncalves J, Kostakos V (2019) Understanding smartphone notifications’ user interactions and content importance. Int J Hum-Comput Stud 128:72–85

    Article  Google Scholar 

  64. Weber D, Voit A, Auda J, Schneegass S, Henze N (2018) Snooze! investigating the user-defined deferral of mobile notifications. In: Proceedings of the 20th international conference on human-computer interaction with mobile devices and services, MobileHCI ’18 (New York, NY, USA). Association for Computing Machinery

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Aalborg University, Aalborg, Denmark

    Niels van Berkel

  2. The University of Melbourne, Melbourne, Australia

    Vassilis Kostakos

Authors
  1. Niels van Berkel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vassilis Kostakos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Niels van Berkel .

Editor information

Editors and Affiliations

  1. Cyprus University of Technology, Limassol, Cyprus

    Evangelos Karapanos

  2. Westphalian University of Applied Scienc, Gelsenkirchen, Nordrhein-Westfalen, Germany

    Jens Gerken

  3. Aalborg University, Aalborg East, Denmark

    Jesper Kjeldskov

  4. Aalborg University, Aalborg East, Denmark

    Mikael B. Skov

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

van Berkel, N., Kostakos, V. (2021). Recommendations for Conducting Longitudinal Experience Sampling Studies. In: Karapanos, E., Gerken, J., Kjeldskov, J., Skov, M.B. (eds) Advances in Longitudinal HCI Research. Human–Computer Interaction Series. Springer, Cham. https://doi.org/10.1007/978-3-030-67322-2_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-67322-2_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-67321-5

  • Online ISBN: 978-3-030-67322-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation