Quality of Life Research

, Volume 25, Issue 3, pp 507–516 | Cite as

Advantages and psychometric validation of proximal intensive assessments of patient-reported outcomes collected in daily life

  • Eve B. CarlsonEmail author
  • Nigel P. Field
  • Josef I. Ruzek
  • Richard A. Bryant
  • Constance J. Dalenberg
  • Terrence M. Keane
  • David A. Spain
Special Section: PROs in Non-Standard Settings (by invitation only)



Ambulatory assessment data collection methods are increasingly used to study behavior, experiences, and patient-reported outcomes (PROs), such as emotions, cognitions, and symptoms in clinical samples. Data collected close in time at frequent and fixed intervals can assess PROs that are discrete or changing rapidly and provide information about temporal dynamics or mechanisms of change in clinical samples and individuals, but clinical researchers have not yet routinely and systematically investigated the reliability and validity of such measures or their potential added value over conventional measures. The present study provides a comprehensive, systematic evaluation of the psychometrics of several proximal intensive assessment (PIA) measures in a clinical sample and investigates whether PIA appears to assess meaningful differences in phenomena over time.


Data were collected on a variety of psychopathology constructs on handheld devices every 4 h for 7 days from 62 adults recently exposed to traumatic injury of themselves or a family member. Data were also collected on standard self-report measures of the same constructs at the time of enrollment, 1 week after enrollment, and 2 months after injury.


For all measure scores, results showed good internal consistency across items and within persons over time, provided evidence of convergent, divergent, and construct validity, and showed significant between- and within-subject variability.


Results indicate that PIA measures can provide valid measurement of psychopathology in a clinical sample. PIA may be useful to study mechanisms of change in clinical contexts, identify targets for change, and gauge treatment progress.


Experience sampling method Ecological momentary assessment Intensive longitudinal data Ambulatory assessment Traumatic stress Patient-reported outcomes 



We wish to thank Abbey Tillery, Marianne Kabour, Rob Wheeler, Donn Garvert, Luma Muhtadie, and Janet Neff for their work on the research and all of the research participants who generously contributed their time and effort to benefit others.  We thank John Nezlek for his comments and suggestions for statistical analyses. The contents do not represent the views of the U.S. Department of Veterans Affairs or the United States Government.


This study was funded by the National Institute of Mental Health (MH69876).

Compliance with ethical standards

Conflict of interest

All authors declare they have no conflict of interest. There are no commercial products associated with this research.

Ethical approval

All aspects of the conduct of this study were in accordance with the ethical standards of the VA Palo Alto Healthcare System and the Stanford University Institutional Review Board and with the 1964 Declaration of Helsinki and its later amendments.


  1. 1.
    Schwarz, N. (2012). Why researchers should think “real-time”: A cognitive rationale. In M. R. Mehl & T. S. Conner (Eds.), Handbook of research methods for studying daily life (pp. 22–42). New York: Guilford Press.Google Scholar
  2. 2.
    Reis, H. T., & Gable, S. L. (2000). Event-sampling and other methods for studying everyday experience. In H. T. Reis & C. M. Judd (Eds.), Handbook of research methods for social and personality psychology (pp. 190–222). New York: Wiley.Google Scholar
  3. 3.
    Nezlek, J. B. (2012). Diary methods for personality and social psychology. Los Angeles: Sage.Google Scholar
  4. 4.
    Myin-Germeys, I., Oorschot, M., Collip, D., Lataster, J., Delespaul, P., & van Os, J. (2009). Experience sampling research in psychopathology: Opening the black box of daily life. Psychological Medicine, 39(9), 1533–1547.CrossRefPubMedGoogle Scholar
  5. 5.
    Shrout, P. E., & Lane, S. P. (2012). Psychometrics. In M. R. Mehl & T. S. Conner (Eds.), Handbook of research methods for studying daily life (pp. 302–320). New York: Guilford.Google Scholar
  6. 6.
    Kashdan, T. B., & Nezlek, J. B. (2012). Whether, when, and how is spirituality related to well-being? Moving beyond single occasion questionnaires to understanding daily process. Personality and Social Psychology Bulletin, 38, 1523–1535.CrossRefPubMedGoogle Scholar
  7. 7.
    Nezlek, J. B., Sorrentino, R. M., Yasunaga, S., Otsubo, Y., Allen, M., Kouhara, S., & Shuper, P. (2008). Cross-cultural differences in reactions to daily events as indicators of cross-cultural differences in self-construction and affect. Journal of Cross-Cultural Psychology, 39, 685–702.CrossRefGoogle Scholar
  8. 8.
    Nezlek, J. B., & Gable, S. L. (2001). Depression as a moderator of relationships between positive daily events and day-to-day psychological adjustment. Personality and Social Psychology Bulletin, 27, 1692–1704.CrossRefGoogle Scholar
  9. 9.
    Wilhelm, P., & Schoebi, D. (2007). Assessing mood in daily life: Structural validity, sensitivity to change, and reliability of a short-scale to measure three basic dimensions of mood. European Journal of Psychological Assessment, 23, 258–267.CrossRefGoogle Scholar
  10. 10.
    Kimhy, D., Delespaul, P., Corcoran, C., Ahn, H., Yale, S., & Malaspina, D. (2006). Computerized experience sampling method (ESMc): Assessing feasibility and validity among individuals with schizophrenia. Journal of Psychiatric Research, 40, 221–230.CrossRefPubMedPubMedCentralGoogle Scholar
  11. 11.
    Granholm, E., Loh, C., & Swendsen, J. (2008). Feasibility and validity of computerized ecological momentary assessment in schizophrenia. Schizophrenia Bulletin, 34, 507–514.CrossRefPubMedPubMedCentralGoogle Scholar
  12. 12.
    Kimhy, D., Delespaul, P., Ahn, H., Cai, S., Shikhman, M., Lieberman, J. A., et al. (2010). Concurrent measurement of “real-world” stress and arousal in individuals with psychosis: Assessing the feasibility and validity of a novel methodology. Schizophrenia Bulletin, 36, 1131–1139.CrossRefPubMedPubMedCentralGoogle Scholar
  13. 13.
    Wheeler, L., & Reis, H. T. (1991). Self-recording of everyday life events: Origins, types, and uses. Journal of Personality, 59, 339–354.CrossRefGoogle Scholar
  14. 14.
    Csikszentmihalyi, M., & Larson, R. (1987). Validity and reliability of the experience-sampling method. Journal of Nervous and Mental Disease, 175(9), 526–536.CrossRefPubMedGoogle Scholar
  15. 15.
    Stone, A. A., & Shiffman, S. (1994). Ecological momentary assessment in behavioral medicine. Annals of Behavioral Medicine, 16, 199–202.Google Scholar
  16. 16.
    Ariely, D., & Carmon, Z. (2000). Gestalt characteristics of experiences: The defining features of summarized events. Journal of Behavioral Decision Making, 13, 191–201.CrossRefGoogle Scholar
  17. 17.
    Redelmeier, D. A., & Kahneman, D. (1996). Patients’ memories of painful medical treatments: Real-time and retrospective evaluations of two minimally invasive procedures. Pain, 66, 3–8.CrossRefPubMedGoogle Scholar
  18. 18.
    Redelmeier, D. A., Katz, J., & Kahneman, D. (2003). Memories of colonoscopy: A randomized trial. Pain, 104, 187–194.CrossRefPubMedGoogle Scholar
  19. 19.
    Kamarck, T. W., Shiffman, S., Muldoon, M. F., Sutton-Tyrrell, K., Gwaltney, C. J., Janicki, D. L., & Schwartz, J. E. (2007). Ecological momentary assessment as a resource for social epidemiology. In A. A. Stone, S. Shiffman, A. Atienza, & L. Nebeling (Eds.), The science of real-time data capture: Self-reports in health research (pp. 268–285). New York: Oxford University Press.Google Scholar
  20. 20.
    Carlson, E. B. (2001). Psychometric study of a brief screen for PTSD: Assessing the impact of multiple traumatic events. Assessment, 8, 431–441.CrossRefPubMedGoogle Scholar
  21. 21.
    Caspi, Y., Carlson, E. B., & Klein, E. (2007). Validation of a screening instrument for posttraumatic stress disorder in a community sample of Bedouin men serving in the Israeli Defense Forces. Journal of Traumatic Stress, 20, 529–539.CrossRefGoogle Scholar
  22. 22.
    Carlson, E. B., Waelde, L., Palmieri, P. A., Smith, S. R., McDade-Montez, E., & Macia, K. (in press). Development and validation of the Dissociative Symptoms Scale. Assessment.Google Scholar
  23. 23.
    Ehlers, A., & Clark, D. M. (2000). A cognitive model of posttraumatic stress disorder. Behaviour Research and Therapy, 38, 319–345.CrossRefPubMedGoogle Scholar
  24. 24.
    Stanton, A. L., Kirk, S. B., Cameron, C. L., & Danoff-Burg, S. (2000). Coping through emotional approach: Scale construction and validation. Journal of Personality and Social Psychology, 78, 1150–1169.CrossRefPubMedGoogle Scholar
  25. 25.
    Foa, E. B., Ehlers, A., Clark, D. M., Tolin, D. F., & Orsillo, S. M. (1999). The posttraumatic cognitions inventory (PTCI): Development and validation. Psychological Assessment, 11, 303–314.CrossRefGoogle Scholar
  26. 26.
    Bolton, E. E., Glenn, D. M., Orsillo, S., Roemer, L., & Litz, B. T. (2003). The relationship between self-disclosure and symptoms of posttraumatic stress disorder in peacekeepers deployed to Somalia. Journal of Traumatic Stress, 16, 203–210.CrossRefPubMedGoogle Scholar
  27. 27.
    Cordova, M. J., Ruzek, J. I., Benoit, M., & Brunet, A. (2003). Promotion of emotional disclosure following illness and injury: A brief intervention for medical patients and their families. Cognitive and Behavioral Practice, 10, 358–371.CrossRefGoogle Scholar
  28. 28.
    Emmons, R. A., McCullough, M. E., & Tsang, J.-A. (2003). The assessment of gratitude. In S. J. Lopez & C. R. Snyder (Eds.), Positive psychological assessment: A handbook of models and measures (pp. 327–341). Washington, DC: American Psychological Association.CrossRefGoogle Scholar
  29. 29.
    Feldman Barrett, E., & Barrett, D. J. (1999). Experience sampling program. Boston: Boston College.Google Scholar
  30. 30.
    Jacobs, N., Nicolson, N., Derom, C., Delespaul, P., van Os, J., & Myin-Germeys, I. (2005). Electronic monitoring of salivary cortisol sampling compliance in daily life. Life Sciences, 76, 2431–2443.CrossRefPubMedGoogle Scholar
  31. 31.
    Wichers, M. C., Myin-Germeys, I., Jacobs, N., Peeters, F., Kenis, G., Derom, C., et al. (2007). Evidence that moment-to-moment variation in positive emotions buffer genetic risk for depression: A momentary assessment twin study. Acta Psychiatrica Scandinavica, 115, 451–457.CrossRefPubMedGoogle Scholar
  32. 32.
    Delespaul, P., deVries, M., & Van Os, J. (2002). Determinants of occurrence and recovery from hallucinations in daily life. Social Psychiatry and Psychiatric Epidemiology, 37, 97–104.CrossRefPubMedGoogle Scholar
  33. 33.
    Myin-Germeys, I., van Os, J., Schwartz, J. E., Stone, A. A., & Delespaul, P. A. (2001). Emotional reactivity to daily life stress in psychosis. Archives of General Psychiatry, 58(12), 1137–1144.CrossRefPubMedGoogle Scholar
  34. 34.
    Geschwind, N., Peeters, F., Jacobs, N., Delespaul, P., Derom, C., Thiery, E., et al. (2010). Meeting risk with resilience: High daily life reward experience preserves mental health. Acta Psychiatra Scandinavica, 122, 129–138.CrossRefGoogle Scholar
  35. 35.
    Nezlek, J. B. (2011). Mutilevel modeling for social and personality psychology. Los Angeles: Sage.Google Scholar
  36. 36.
    Singer, J. D., & Willett, J. B. (2003). Applied longitudinal data analysis: Modeling change and event occurrence. New York: Oxford University Press.CrossRefGoogle Scholar
  37. 37.
    Dalenberg, C. J., & Carlson, E. B. (2012). Dissociation in posttraumatic stress disorder part II: How theoretical models fit the empirical evidence and recommendations for modifying the diagnostic criteria for PTSD. Psychological Trauma, 4, 551–559.CrossRefGoogle Scholar
  38. 38.
    Breslau, N., Kessler, R. C., Chilcoat, H. D., Schultz, L. R., Davis, G. C., & Andreski, P. (1998). Trauma and posttraumatic stress disorder in the community: The 1996 Detroit Area Survey of Trauma. Archives of General Psychiatry, 55, 626–631.CrossRefPubMedGoogle Scholar
  39. 39.
    Carlson, E. B., Dalenberg, C. J., & McDade-Montez, E. (2012). Dissociation in posttraumatic stress disorder part I: Definitions and review of research. Psychological Trauma, 4, 479–489.CrossRefGoogle Scholar
  40. 40.
    Kaniasty, K. (2005). Social support and traumatic stress. PTSD Research Quarterly, 16(2), 1–7.Google Scholar
  41. 41.
    Cordova, M. J., Cunningham, L. L. C., Carlson, C. R., & Andrykowski, M. A. (2001). Social constraints, cognitive processing, and adjustment to breast cancer. Journal of Consulting and Clinical Psychology, 69, 706–711.CrossRefPubMedGoogle Scholar
  42. 42.
    Christianson, S. (Ed.). (1992). Handbook of emotion and memory: Research and theory. Hillsdale, NJ: Erlbaum.Google Scholar
  43. 43.
    Myin-Germeys, I. (2012). Psychiatry. In M. R. Mehl & T. S. Conner (Eds.), Handbook of research methods for studying daily life (pp. 636–650). New York: Guilford Press.Google Scholar
  44. 44.
    Stone, A. A. (2007). Thoughts on the present state of real-time data capture. In A. A. Stone, S. Shiffman, A. Atienza, & L. Nebeling (Eds.), The science of real-time data capture: Self-reports in health research (pp. 361–370). New York: Oxford University Press.Google Scholar

Copyright information

© Springer International Publishing Switzerland (outside the USA)  2015

Authors and Affiliations

  • Eve B. Carlson
    • 1
    Email author
  • Nigel P. Field
    • 2
  • Josef I. Ruzek
    • 1
  • Richard A. Bryant
    • 3
  • Constance J. Dalenberg
    • 4
  • Terrence M. Keane
    • 5
    • 6
  • David A. Spain
    • 7
  1. 1.National Center for PTSD and VA Palo Alto Health Care SystemDepartment of Veterans AffairsMenlo ParkUSA
  2. 2.Pacific Graduate School of PsychologyPalo Alto UniversityPalo AltoUSA
  3. 3.School of PsychologyUniversity of New South WalesSydneyAustralia
  4. 4.Alliant International UniversitySan DiegoUSA
  5. 5.Departments of Psychiatry and PsychologyBoston University School of MedicineBostonUSA
  6. 6.National Center for PTSD and VA Boston Health Care SystemDepartment of Veterans AffairsBostonUSA
  7. 7.Department of SurgeryStanford University School of MedicineStanfordUSA

Personalised recommendations