Abstract
Experience sampling methods are widely used in clinical psychology to study affective dynamics in psychopathology. The present study examined whether affect ratings (valence and arousal) differed as a function of assessment schedule (signal- versus event-contingent) in a clinical sample and considered various approaches to modeling these ratings. A total of 40 community mental health center outpatients completed ratings of their affective experiences over a 21-day period using both signal-contingent schedules (random prompts) and event-contingent schedules (ratings following social interactions). We tested whether assessment schedules impacted 1) the central tendency (mean) and variability (standard deviation) of valence or arousal considered individually, 2) the joint variability in valence and arousal via the entropy metric, and 3) the between-person differences in configuration of valence-arousal landscapes via the Earth Mover’s Distance (EMD) metric. We found that event-contingent schedules, relative to signal-contingent schedules, captured higher average levels of pleasant valence and emotional arousal ratings. Moreover, signal-contingent schedules captured greater variability within and between individuals on arousal-valence landscapes compared to event-contingent schedules. Altogether, findings suggest that the two assessment schedules should not be treated interchangeably in the assessment of affect over time. Researchers must be cautious in generalizing results across studies utilizing different experience sampling assessment schedules.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
We used this approach, rather than Bolger et al. (2003) approach of dropping individuals with entries that are more than two standard deviations below the mean for the sample, because our specific analyses depend on a kernel density estimate of the two-dimensional space (e.g., valence and arousal). That is, a low number of completed surveys does not provide enough data to accurately estimate the person’s surface. Thus, we do not think the conventional wisdom applies here.
References
Barlow, D. H., & Farchione, T. J. (2018). Applications of the unified protocol for transdiagnostic treatment of emotional disorder. New York: Oxford University Press.
Berner, L. A., Crosby, R. D., Cao, L., Engel, S. G., Lavender, J. M., Mitchell, J. E., & Wonderlich, S. A. (2017). Temporal associations between affective instability and dysregulated eating behavior in bulimia nervosa. Journal of Psychiatric Research, 92, 183–190. https://doi.org/10.1016/j.jpsychires.2017.04.009.
Bolger, N., Davis, A., & Rafaeli, E. (2003). Diary methods: Capturing life as it is lived. Annual Review of Psychology, 54, 579–616. https://doi.org/10.1146/annurev.psych.54.101601.145030.
Brown, T. A., Di Nardo, P. A., & Barlow, D. H. (1994). Anxiety disorders interview schedule for DSM-IV (adult version). Albany: Graywind.
Carver, C. S., & Scheier, M. F. (1998). On the self-regulation of behavior. New York: Cambridge University Press.
Clifton, A., Pilkonis, P. A., & McCarty, C. (2007). Social networks in borderline personality disorder. Journal of Personality Disorders, 21, 434–441. https://doi.org/10.1521/pedi.2007.21.4.434.
Conner, T., & Bliss-Moreau, E. (2006). Sampling human experience in naturalistic settings. In S. Hesse-Biber & P. Leavy (Eds.), Emergent methods in social research (pp. 109–129). Thousand Oaks: Sage.
Crowe, E., Daly, M., Delaney, L., Carroll, S., & Malone, K.M. (2018). Psychiatry Research. Advanced online publication. https://doi.org/10.1016/j.psychres.2018.02.032.
Csikszentmihalyi, M., & Larson, R. (1987). Validity and reliability of the experience-sampling method. Journal of Nervous and Mental Disease, 175, 526–536. https://doi.org/10.1007/978-94-017-9088-8_3.
Dawood, S., & Pincus, A. L. (2018). Pathological narcissism and the severity, variability, and instability of depressive symptoms. Personality Disorders: Theory, Research, and Treatment, 9, 144–154. https://doi.org/10.1037/per0000239.
Ebner-Priemer, U. W., Kuo, J., Welch, S. S., Thielgen, T., Witte, S., Bohus, M., & Linehan, M. M. (2006). A valence-dependent group specific recall bias of retrospective self-reports: A study of borderline personality disorder in everyday life. Journal of Nervous and Mental Disease, 194, 774–779. https://doi.org/10.1097/01.nmd.0000239900.46595.72.
Ebner-Priemer, U. W., Kuo, J., Kleindienst, N., Welch, S. S., Reisch, T., Reinhard, I., Lieb., Linehan, & Bohus (2007). State affective instability in borderline personality disorder assessed by ambulatory monitoring. Psychological Medicine, 37, 961–970. https://doi.org/10.1017/S0033291706009706.
Ebner-Priemer, U. W., Kuo, J., Scholtz, W., Kleindienst, N., Rosenthal, Z., Detterer, L., Linehan, M., & Bohus, M. (2008). Distress and affective dysregulation in patients with borderline personality disorder: A psychophysiological ambulatory monitoring study. Journal of Nervous and Mental Disorders, 196, 314–320. https://doi.org/10.1097/NMD.0b013e31816a493f.
Ellison, W.D., Levy, K.N., Newman, M.G., Pincus, A.L., Wilson, S.J., & Molenaar, P.C.M. (2019). Dynamics among borderline personality and anxiety features in psychotherapy outpatients: An exploration of nomothetic and idiographic patterns. Personality Disorders: Theory, Research, and Treatment. Advanced online publication. doi: https://doi.org/10.1037/per0000363.
Farmer, A. S., & Kashdan, T. B. (2014). Affective and self-esteem instability in the daily lives of people with generalized social anxiety disorder. Clinical Psychological Science, 2, 187–201. https://doi.org/10.1177/2167702613495200.
Himmelstein, P. H., Woods, W. C., & Wright, A. G. C. (2019). A comparison of signal- and event-contingent ambulatory assessment of interpersonal behavior and affect in social situations. Psychological Assessment, 31, 952–960. https://doi.org/10.1037/pas0000718.
Hollenstein, T. (2007). State space grids: Analyzing dynamics across development. International Journal of Behavioral Development, 31, 384–396. https://doi.org/10.1177/0165025407077765.
Knowles, R., Tai, S., Jones, S. H., Highfield, J., Morriss, R., & Bentall, R. P. (2007). Stability of self-esteem in bipolar disorder: Comparisons among remitted bipolar patients, remitted unipolar patients and healthy controls. Bipolar Disorder, 9, 490–495. https://doi.org/10.1111/j.1399-5618.2007.00457.x.
Kopala-Sibley, D. C., Zuroff, D. C., Russell, J. J., Moskowitz, D. S., & Paris, J. (2012). Understanding heterogeneity in borderline personality disorder: Differences in affective reactivity explained by the traits of dependency and self-criticism. Journal of Abnormal Psychology, 121, 680–691. https://doi.org/10.1037/a0028513.
Koval, P., Pe, M. L., Meers, K., & Kuppens, P. (2013). Affective dynamics in relation to depressive symptoms: Variable, unstable, or inert? Emotion, 13, 1132–1141. https://doi.org/10.1037/a0033579.
Lawrence, M. A. (2016). ez: Easy analysis and visualization of factorial experiments. URL. https://cran.r-project.org/package=ez.
Liu, H., Xie, Q. W., & Lou, V. W. Q. (2019). Everyday social interactions and intra-individual variability in affect: A systematic review and meta-analysis of ecological momentary assessment studies. Motivation and Emotion, 43, 339–353. https://doi.org/10.1007/s11031-018-9735-x.
Loranger, A. W. (1999). International Personality Disorder Examination (IPDE). Odessa: Psychological Assessment Resources.
Mason, T., Lavender, J. M., Wonderlich, S. A., Steiger, H., Cao, L., Engel, S. G., Mitchell, J. E., & Crosby, R. D. (2017). Comfortably numb: The role of momentary dissociation in the experience of negative affect around binge eating. The Journal of Nervous and Mental Disease, 205, 335–339. https://doi.org/10.1097/NMD.0000000000000658.
Mehl, M. R., & Conner, T. S. (2012). Handbook of research methods for studying daily life. New York: The Guildford Press.
Mohr, C. D., Aprin, S., & McCabe, C. T. (2015). Daily affect variability and context-specific alcohol consumption. Drug and Alcohol Review, 34, 581–587. https://doi.org/10.1111/dar.12253.
Mohr, C. D., Zhang, M., & Schueller, S. M. (2017). Personal sensing: Understanding mental health using ubiquitous sensors and machine learning. Annual Review of Clinical Psychology, 13, 23–47. https://doi.org/10.1146/annurev-clinpsy-032816-044949.
Moskowitz, D. S., Russell, J. J., Sadikaj, G., & Sutton, R. (2009). Measuring people intensively. Canadian Psychology, 50, 131–140. https://doi.org/10.1037/a0016625.
Muehlenkamp, J. J., Engel, S. G., Crosby, R. D., Wonderlich, S. A., Simonich, H., & Mitchell, J. E. (2009). Emotional states preceding and following acts of non-suicidal self-injury in bulimia nervosa patients. Behavior Research and Therapy, 47, 83–87. https://doi.org/10.1016/j.brat.2008.10.011.
Myin-Germeys, I., Oorschot, M., Collip, D., Lataster, J., Delespau, P., & van Os, J. (2009). Experience sampling research in psychopathology: Opening the black box of daily life. Psychological Medicine, 39, 1533–1547. https://doi.org/10.1017/S0033291708004947.
Palmier-Claus, J. E., Taylor, P. J., Gooding, P., Dunn, G., & Lewis, S. W. (2012). Affective variability predicts suicidal ideation in individuals at ultra-high risk of developing psychosis: An experience sampling study. Journal of Clinical Psychology, 51, 72–83. https://doi.org/10.1111/j.2044-8260.2011.02013.x.
Pincus, A. L., Hopwood, C. J., & Wright, A. G. C. (in press). The interpersonal situation: An integrative framework for the study of personality, psychopathology, and psychotherapy. In D. Funder, J. F. Rauthmann, & R. Sherman (Eds.), Oxford handbook of psychological situations. New York: Oxford University Press.
Pisetsky, E. M., Crosby, R., Cao, L., Fitzsimmons-Craft, E. E., Mitchell, J. M., Engel, S. G., & Peterson, C. B. (2016). An examination of affect prior to and following episodes of getting drunk in women with bulimia nervosa. Psychiatry Research, 240, 202–208. https://doi.org/10.1016/j.psychres.2016.04.044.
Posner, J., Russell, J. A., & Peterson, B. S. (2005). The circumplex model of affect: An integrative approach to affective neuroscience, cognitive development, and psychopathology. Development and Psychopathology, 17, 715–734. https://doi.org/10.1017/S0954579405050340.
Ram, N., Conroy, D. E., Pincus, A. L., Hyde, A. L., & Molloy, L. (2012). Tethering theory to method: Using measures of intraindividual variability to operationalize individuals’ dynamic characteristics. In G. Hancock & J. Harring (Eds.), Advances in longitudinal modeling in the social and behavioral sciences (pp. 81–110). Charlotte: Information Age Publishing.
Ram, N., Coccia, M., Conroy, D. E., Lorek Dattiko, A., Orland, B., Pincus, A. L., Sliwinski, M., & Gerstorf, D. (2013). Behavioral landscapes and change in behavioral landscapes: A multiple time-scale density distribution approach. Research in Human Development, 10, 88–110. https://doi.org/10.1016/j.jrp.2016.06.010.
Ram, N., Benson, K., Brick, T. R., Conroy, D. E., & Pincus, A. L. (2017). Behavioral landscapes and earth mover’s distance. A new approach for studying individual differences in density distributions. Journal of Research in Personality, 69, 191–205. https://doi.org/10.1016/j.jrp.2016.06.010.
Reis, H. T., & Gable, S. L. (2000). Event-sampling and other methods for studying everyday experience. In H. T. Reis, & C. M. Judd (Eds.), Handbook of research methods in social and personality psychology (pp. 190–222). New York Cambridge Univ. Press.
Reisch, T., Ebner-Priemer, U. W., Tschacher, W., Bohus, M., & Linehan, M. M. (2008). Sequences of emotions in patients with borderline personality disorder. Acta Psychiatrica Scandinavica, 118, 42–48. https://doi.org/10.1111/j.1600-0447.2008.01222.x.
Roche, M. J., & Pincus, A. L. (2016). Precision assessment. An individualized and temporally dynamic approach to understanding patients in their daily lives. In U. Kumar (Ed.), The Wiley handbook of personality assessment (pp. 192–204). Chichester: Wiley, Ltd.
Rubner, Y., Tomasi, C., & Guibas, L. (2000). The earth mover’s distance as a metric for image retrieval. International Journal on Computer Vision, 40, 99–121.
Russell, J. A. (2003). Core affect and the psychological construction of emotion. Psychological Review, 110, 145–172. https://doi.org/10.1037/0033-295X.110.1.145.
Russell, J. J., Moskowitz, D. S., Zuroff, D. C., Sookman, D., & Paris, J. (2007). Stability and variability of affective experiences and interpersonal behavior in borderline personality disorder. Journal of Abnormal Psychology, 116, 578–588. https://doi.org/10.1037/0021-843X.116.3.578.
Sadikaj, G., Russell, J. J., Moskowitz, D. S., & Paris, J. (2010). Affect dysregulation in individuals with borderline personality disorder: Persistence and interpersonal triggers. Journal of Personality Assessment, 92, 490–500. https://doi.org/10.1080/00223891.2010.513287.
Sadikaj, G., Moskowitz, D. S., Russell, J. J., Zuroff, D. C., & Paris, J. (2013). Quarrelsome behavior in borderline personality disorder: Influence of behavioral and affective reactivity to perceptions of others. Journal of Abnormal Psychology, 122, 195–207. https://doi.org/10.1037/a0030871.
Santangelo, P., Reinhard, I., Mussgay, L., Steil, R., Sawitzki, G., Klein, C., et al. (2014). Specificity of affective instability in patients with borderline personality disorder compared to posttraumatic stress disorder, bulimia nervosa, and healthy controls. Journal of Abnormal Psychology, 123, 258–272. https://doi.org/10.1037/a0035619.
Santangelo, P. S., Limberger, M. F., Stiglmayr, C., Houben, M., Coosemans, J., Verleysen, G., Kuppens, P., Tuerlinckx, F., Vanpaemel, W., & Ebner-Priemer, U. W. (2016). Analyzing subcomponents of affective dysregulation in borderline personality disorder in comparison to other clinical groups using multiple e-diary datasets. Borderline Personality Disorder and Emotion Dysregulation, 3, 1–13. https://doi.org/10.1186/s40479-016-0039-z.
Scala, J. W., Levy, K. N., Johnson, B. N., Kivity, Y., Ellison, W. D., Pincus, A. L., et al. (2018). The role of negative affect and self concept clarity in predicting self-injurious urges in borderline personality disorders using ecological momentary assessment. Journal of Personality Disorders, 32, 36–57. https://doi.org/10.1521/pedi.2018.32.supp.36.
Schimmack, U., & Grob, A. (2000). Dimensional models of core affect: A quantitative by means of structural equation modeling. European Journal of Personality, 14, 325–345. https://doi.org/10.1002/1099-0984(200007/08)14:4<325::AID-PER380>3.0.CO;2-I.
Selby, E. A., Doyle, P., Crosby, R. D., Wonderlich, S. A., Engel, S. G., Mitchell, J. D., & Le Grange, D. (2012). Momentary emotion surrounding bulimic behaviors in women with bulimia nervosa and borderline personality disorder. Journal of Psychiatric Research, 46, 1492–1500. https://doi.org/10.1016/j.jpsychires.2012.08.014.
Shiffman, S., Stone, A. A., & Hufford, M. R. (2008). Ecological momentary assessment. Annual Review of Clinical Psychology, 4, 1–32. https://doi.org/10.1146/annurev.clinpsy.3.022806.091415.
Smyth, J. M., Wonderlich, S. A., Heron, K. E., Sliwinski, M. J., Crosby, R. D., Mitchell, J. E., & Engel, S. G. (2007). Daily and momentary mood and stress are associated with binge eating and vomiting in bulimia nervosa patients in the natural environment. Journal of Consulting and Clinical Psychology, 75, 629–638. https://doi.org/10.1037/0022-006X.75.4.629.
Stein, K. F. (1996). Affect instability in adults with a borderline personality disorder. Archives of Psychiatric Nursing, 10, 32–40. https://doi.org/10.1016/S0883-9417(96)80084-7.
Stone, A. A., & Shiffman, S. (1994). Ecological momentary assessment (EMA) in behavioral medicine. Annals of Behavioral Medicine, 16, 199–202. https://doi.org/10.1093/abm/16.3.199.
Thayer, R. E. (1989). The biopsychology of mood. New York: Oxford University Press.
Trull, T. J., & Ebner-Priemer, U. (2013). Ambulatory assessment. Annual Review of Clinical Psychology, 9, 151–176. https://doi.org/10.1146/annurev-clinpsy-050212-185510.
Trull, T. J., Solhan, M. B., Tragesser, S. L., Jahng, S., Wood, P. K., Piasecki, T. M., & Watson, D. (2008). Affective instability: Measuring a core feature of borderline personality disorder with ecological momentary assessment. Journal of Abnormal Psychology., 117, 647–661. https://doi.org/10.1037/a0012532.
Trull, T. J., Ebner-Priemer, U. W., Brown, W. C., Tomko, R. L., & Scheiderer, E. M. (2012). Clinical psychology. In M. R. Mehl & T. S. Conner (Eds.), Handbook of research methods for studying daily life (pp. 620–635). New York: Guilford Press.
Urbanek, S., and Y. Rubner, 2012. emdist: Earth Mover’s Distance. R package version 0.2-1. URL http://CRAN.R-project.org/package=emdist.
van Os, J., Delespaul, P., Wigman, J., Myin-Germeys, I., & Wichers, M. (2013). Beyond DSM and ICD: Introducing "precision diagnosis" for psychiatry using momentary assessment technology. World Psychiatry, 12, 113–117. https://doi.org/10.1002/wps.20046.
Vansteelandt, K., Claes, L., Muehlenkamp, J., Claes, L., Muehlenkamp, K., De Cuyper, K., Lemmens, J., Probst, M., Vanderlinden, J., & Pieters, G. (2013). Variability in affective activation predicts non-suicidal self-injury in eating disorders. European Eating Disorders Review, 21, 143–147. https://doi.org/10.1002/erv.2220.
Wand, M. (2015). KernSmooth: Functions for kernel smoothing supporting Wand and Jones (1995). R package version 2.23-15, URL http://CRAN.R-project.org/package=KernSmooth.
Wang, S., Roche, M. J., Pincus, A. L., Conroy, D. E., Rebar, A., & Ram, N. (2014). Interpersonal dependency and emotion in every day life. Journal of Research in Personality, 53, 5–12. https://doi.org/10.1016/j.jrp.2014.07.007.
Watson, D., & Tellegen, A. (1985). Toward a consensual structure of mood. Psychological Bulletin, 98, 219–235. https://doi.org/10.1037/0033-2909.98.2.219.
Wright, A. G. C., Gates, K. M., Arizmendi, C., Lane, S. T., Woods, W. C., & Edershile, E. A. (2019). Focusing personality assessment on the person: Modeling general, shared, and person specific processes in personality and psychopathology. Psychological Assessment, 31, 502–515. https://doi.org/10.1037/pas0000617.
Zautra, A. J., Berkhof, J., & Nicolson, N. A. (2002). Changes in affect interrelations as a function of stressful events. Cognition and Emotion, 16, 309–318. https://doi.org/10.1080/02699930143000257.
Acknowledgements
We thank J. Wesley Scala, M.S., and Emily A. Dowgwillo, M.S., for assistance in developing training material and coordinating the collection of data, and Kathleen Bohomey, Colin Carey, Caroline Curran, Wendi Falk, Sarah Forsythe, Laura Frey, Caroline Gooch, Jessica Grom, Brittani Hollern, Lauren Lipner, Kristin McLaughlin, Megan Moyer, Joanna Pantelides, Megan Parker, Jacqueline Proczynski, Carolina Ribo, Silvia Rizkallah, Aimee Sohnleitner, and Alyssa Spaw for their assistance in data collection. We also thank Jennifer Fox and Allison Clark for their assistance in the recruitment of participants. Finally, we thank Drs. William D. Ellison, Peter Molenaar and Joshua Smyth for consultation.
Funding
This research was supported by grants from The Pennsylvania State University Social Science Research Institute (principal investigator: Levy), and secondary grants to William D. Ellison from a dissertation grant from The Pennsylvania State University Research and Graduate Studies Office, and grants from American Psychoanalytic Association and International Psychoanalytic Association.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
Sindes Dawood, Michael N. Hallquist, Aaron L. Pincus, Nilam Ram, Michelle G. Newman, Stephen J. Wilson, and Kenneth N. Levy declare that they have no conflict of interest.
Ethical Approval
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed Consent
Informed consent was obtained from all individual participants included in the study.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Dawood, S., Hallquist, M.N., Pincus, A.L. et al. Comparing Signal-Contingent and Event-Contingent Experience Sampling Ratings of Affect in a Sample of Psychotherapy Outpatients. J Psychopathol Behav Assess 42, 13–24 (2020). https://doi.org/10.1007/s10862-019-09766-7
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10862-019-09766-7