Skip to main content

Advertisement

SpringerLink
Log in

Applying Generalizability Theory to Differentiate Between Trait and State in the Interpersonal Mindfulness Scale (IMS)

  • ORIGINAL PAPER
  • Published:
Mindfulness Aims and scope Submit manuscript

Abstract

Objectives

The Interpersonal Mindfulness Scale (IMS) is a recently developed measure to assess mindfulness in the context of interpersonal interactions. The IMS showed promising psychometric properties and is considered to be a trait measure, but its temporal reliability and ability to distinguish trait from state have not been rigorously examined using appropriate methods. Generalizability theory (G-theory) is increasingly used to differentiate between trait and state aspects and identify sources of error affecting a measure. This trait/state distinction is important because it is required for evaluation of long-term effects of mindfulness-based training, especially those targeting interpersonal relationships.

Methods

Responses of 116 participants who completed the scale at three occasions, with 1-month intervals, were analyzed using G-theory and person by item by occasion longitudinal observation design.

Results

The 27-item IMS demonstrated strong reliability and generalizability of scores across persons and occasions in measuring interpersonal mindfulness as a trait with G-coefficients of 0.84-0.91. Individual subscales were less reliable. Further investigation combining the most stable items into a trait-only measure did not result in a more reliable measure, nor did combining the least stable items into a state-only measure.

Conclusions

This study supported good psychometric properties of the full IMS as a measure of a trait with the total scores generalizable across people and occasions. These findings suggest that the IMS total trait scores can be used to reliably evaluate long-term effects of interventions aiming to enhance interpersonal mindfulness.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Arterberry, B. J., Martens, M. P., Cadigan, J. M., & Rohrer, D. (2014). Application of generalizability theory to the big five inventory. Personality and Individual Differences, 69, 98–103.

    Article  Google Scholar 

  • Baer, R. A., Smith, G. T., Hopkins, J., Krietemeyer, J., & Toney, L. (2006). Using self-report assessment methods to explore facets of mindfulness. Assessment, 13(1), 27–45. https://doi.org/10.1177/1073191105283504.

    Article  PubMed  Google Scholar 

  • Barreira, D., Garganta, J., Castellano, J., Machado, J., & Anguera, M. T. (2015). How elite-level soccer dynamics has evolved over the last three decades? Input from generalizability theory. Cuadernos de Psicologia del Deporte, 15(1), 51–62.

    Article  Google Scholar 

  • Bloch, R., & Norman, G. (2012). Generalizability theory for the perplexed: a practical introduction and guide: AMEE guide no. 68. Medical Teacher, 34, 960–992.

    Article  Google Scholar 

  • Brown, K. W., & Ryan, R. M. (2003). The benefits of being present: mindfulness and its role in psychological well-being. Journal of Personality and Social Psychology, 84(4), 822–884.

    Article  Google Scholar 

  • Cardinet, J., Pini, G., & Johnson, S. (2011). Applying generalizability theory using EduG. London: Routledge Academic.

    Book  Google Scholar 

  • Chiesa, A., Calati, R., & Serretti, A. (2011). Does mindfulness training improve cognitive abilities? A systematic review of neuropsychological findings. Clinical Psychology Review, 31, 449–464. https://doi.org/10.1016/j.cpr.2010.11.003.

    Article  PubMed  Google Scholar 

  • Cor, M. K., & Peeters, M. J. (2015). Using generalizability theory for reliable learning assessments in pharmacy education. Currents in Pharmacy Teaching & Learning, 7, 332–341. https://doi.org/10.1016/j.cptl.2014.12.003.

    Article  Google Scholar 

  • Cronbach, L. J., Rajaratnam, N., & Gleser, G. C. (1963). Theory of generalizability: a liberalization of reliability theory. British Journal of Statistical Psychology,16(2), 137-163. doi:https://doi.org/10.1111/j.2044-8317.1963.tb00206.x.

  • Ede, D. E., Walter, F. A., & Hughes, J. W. (2020). Exploring how trait mindfulness relates to perceived stress and cardiovascular reactivity. International Journal of Behavioral Medicine, 27. doi: https://doi.org/10.1007/s12529-020-09871-y.

  • Goleman, D., & Davidson, R. J. (2017). Altered traits: science reveals how meditation changes your mind, brain, and body. Penguin.

  • Goyal, M., Singh, S., Sibinga, E. M., Gould, N. F., Rowland-Seymour, A., Sharma, R., Berger, Z., Sleicher, D., Maron, D. D., Shihab, H. M., Ranasinghe, P. D., Linn, S., Saha, S., Bass, E. B., & Haythornthwaite, A. (2014). Meditation programs for psychological stress and well-being: a systematic review and meta-analysis. JAMA Internal Medicine, 174(3), 357–368. https://doi.org/10.1001/jamainternmed.2013.13018.

    Article  PubMed  PubMed Central  Google Scholar 

  • Hafenbrack, A. C., Cameron, L. D., Spreitzer, G. M., Zhang, C., Noval, L. J., & Shaffakat, S. (2020). Helping people by being in the present: mindfulness increases prosocial behavior. Organizational Behavior and Human Decision Processes, 159, 21–38. https://doi.org/10.1016/j.obhdp.2019.08.005.

    Article  Google Scholar 

  • Hilton, L., Hempel, S., Ewing, B. A., Apaydin, E., Xenakis, L., Newberry, S., Colaiaco, B., Maher, A. R., Shanman, R. M., Sorbero, M. E., & Maglione, M. A. (2017). Mindfulness meditation for chronic pain: a systematic review and meta-analysis. Journal of Psychosomatic Research, 57(1), 35–43. https://doi.org/10.1007/s12160-016-9844-2.

    Article  Google Scholar 

  • Howarth, A., Smith, J. G., Perkins-Porras, L., & Ussher, M. (2019). Effects of brief mindfulness-based interventions on health-related outcomes: a systematic review. Mindfulness, 10(10), 1957–1968. https://doi.org/10.1007/s12671-019-01163-1.

    Article  Google Scholar 

  • Hoyt, W. T., & Melby, J. N. (1999). Dependability of measurement in counselling psychology: an introduction to generalizability theory. The Counselling Psychologist, 27(3), 325–352.

    Article  Google Scholar 

  • Huisman, M. (2000). Imputation of missing item responses: some simple techniques. Quality and Quantity, 34(4), 331–351.

    Article  Google Scholar 

  • Kabat-Zinn, J. (1990). Full catastrophe living: using the wisdom of your body and mind to face stress, pain and illness. New York, NY: Delacourt.

    Google Scholar 

  • Kahriman-Pamuk, D., Uzun, N. B., Yildiz, T. G., & Haktanir, G. (2019). Reliability of indicators measuring early childhood education for sustainability: a study in Turkey using generalizability theory. International Journal of Early Childhood, 51, 193–206. https://doi.org/10.1007/s13158-019-00243-6.

    Article  Google Scholar 

  • Krägeloh, C. U., Henning, M. A., Medvedev, O. N., Feng, X. J., Moir, F., Billington, R., & Siegert, R. J. (2019). Mindfulness-based intervention research: characteristics, approaches, and developments. Routledge.

  • Lafave, M. R., & Butterwick, D. J. (2014). A generalizability theory study of athletic taping using the technical skill assessment instrument. Journal of Athletic Training, 49(3), 368–372. https://doi.org/10.4085/1062-6050-49.2.22.

    Article  PubMed  PubMed Central  Google Scholar 

  • Lei, P., Lei, G., Tian, J., Zhou, Z., Zhao, M., & Wan, C. (2014). Development and validation of the irritable bowel syndrome scale under the system of quality of life instruments for chronic diseases QLICD-IBS: combinations of classical test theory and generalizability theory. International Journal of Colorectal Diseases, 29, 1245–1255. https://doi.org/10.1007/s00384-014-1976-x.

    Article  Google Scholar 

  • Levenson, R. W., & Ruef, A. M. (1992). Empathy: a physiological substrate. Journal of Personality and Social Psychology, 63, 234–246.

    Article  Google Scholar 

  • McCaffrey, D. F., Oliveri, M. E., & Holtzman, S. (2018). A generalizability theory study to examine sources of score variance in third-party evaluations used in decision-making for graduate school admissions. ETS GRE Board Research Report. https://doi.org/10.1002/ets2.12225.

  • Medvedev, O. N., Krägeloh, C. U., Narayanan, A., & Siegert, R. J. (2017). Measuring mindfulness: applying generalizability theory to distinguish between state and trait. Mindfulness, 8, 1036–1046. https://doi.org/10.1007/s12671-017-0679-0.

    Article  Google Scholar 

  • Medvedev, O. N., Theadom, A., Barker-Collo, S., & Feigin, V. (2018). Distinguishing between enduring and dynamic concussion symptoms: applying generalisability theory to the Rivermead Post Concussion Symptoms Questionnaire (RPQ). PeerJ, 6, e5676.

    Article  Google Scholar 

  • Medvedev, O. N., Pratscher, S. D., & Bettencourt, A. (2020). Psychometric evaluation of the Interpersonal Mindfulness Scale using Rasch analysis. Mindfulness, 11, 2007–2015. https://doi.org/10.1007/s12671-020-01415-5.

    Article  Google Scholar 

  • Moll, S., Frolic, A., & Key, B. (2015). Investing in compassion: exploring mindfulness as a strategy to enhance interpersonal relationships in a healthcare practice. Journal of Hospital Administration, 4(6), 36–45. https://doi.org/10.5430/jha.v4n6p36.

    Article  Google Scholar 

  • Muthen, B., & Kaplan, D. (1985). A comparison of some methodologies for the factor analysis of non-normal Likert variables: a note on the size of the model. British Journal of Mathematical and Statistical Psychology, 45(1), 19–30. https://doi.org/10.1111/j.2044-8317.1992.tb00832.x.

    Article  Google Scholar 

  • O’Brien, J., Thompson, M. S., & Hagler, D. (2019). Using generalizability theory to inform optimal design for a nursing performance assessment. Evaluation & the Health Professions, 42(3), 297–327. https://doi.org/10.1177/0163278717735565.

    Article  Google Scholar 

  • Paterson, J., Medvedev, O. N., Sumich, A., Tautolo, E., Krägeloh, C. U., Sisk, R., McNamara, R. K., Berk, M., Narayanan, A., & Siegert, R. J. (2017). Distinguishing transient versus stable aspects of depression in New Zealand Pacific Island children using generalizability theory. Journal of Affective Disorders, 227, 698–704. https://doi.org/10.1016/j.jad.2017.11.075.

    Article  PubMed  Google Scholar 

  • Pratscher, S. D., Rose, A. J., Markovitz, L., & Bettencourt, B. A. (2018). Interpersonal mindfulness: investigating mindfulness in interpersonal interactions, co-rumination, and friendship quality. Mindfulness, 9(4), 1206–1215. https://doi.org/10.1007/s12671-017-0859-y.

    Article  Google Scholar 

  • Pratscher, S. D., Wood, P. K., King, L. A., & Bettencourt, B. A. (2019). Interpersonal mindfulness: scale development and initial construct validation. Mindfulness, 10, 1044–1061. https://doi.org/10.1007/s12671-018-1057-2.

    Article  Google Scholar 

  • Shavelson, R. J., & Webb, N. M. (1991). Generalizability theory: a primer (Vol. 1). Sage.

  • Shavelson, R. J., Webb, N. M., & Rowley, G. L. (1989). Generalizability theory. American Psychologist, 44, 599–612.

    Article  Google Scholar 

  • Simons, D. J., Shoda, Y., & Lindsay, D. S. (2017). Constraints on generality (COG): a proposed addition to all empirical papers. Perspectives on Psychological Science, 12(6), 1123–1128. https://doi.org/10.1177/1745691617708630.

    Article  PubMed  Google Scholar 

  • Swiss Society for Research in Education Working Group, (2006). EDUG user guide. IRDP, Euchatel, Switzerland.

  • Talsma, P. (2016). Assessing sensory panel performance using generalizability theory. Food Quality and Preference, 47, 3–9. https://doi.org/10.1016/j.foodqual.2015.02.019.

    Article  Google Scholar 

  • Truong, Q. C., Krägeloh, C. U., Siegert, R. J., Landon, J., & Medvedev, O. N. (2020). Applying generalizability theory to differentiate between trait and state in the Five Facet Mindfulness Questionnaire (FFMQ). Mindfulness, 1–11. https://doi.org/10.1007/s12671-020-01324-7.

  • Vispoel, W. P., Morris, C. A., & Kilinc, M. (2018). Applications of generalizability theory and their relations to classical test theory and structural equation modeling. Psychological Methods, 23(1), 1–26. https://doi.org/10.1037/met0000107.

    Article  PubMed  Google Scholar 

  • Wan, C., Li, H., Fan, X., Yang, R., Pan, J., Chen, W., & Zhao, R. (2014). Development and validation of the coronary heart disease scale under the system of quality of life instruments for chronic diseases QLICD-CHD: combinations of classical test theory and generalizability theory. Health and Quality of Life Outcomes, 12(82), 1–11.

    Google Scholar 

  • Webb, N. M., Shavelson, R. J., & Haertel, E. H. (2006). Reliability coefficients and generalizability theory. Handbook of Statistics, 26, 4–44. https://doi.org/10.1016/S0169-7161(06)26004-8.

    Article  Google Scholar 

  • Winkler-Schwartz, A., Marwa, I., Bajunaid, K., Mullah, M., Alotaibi, F. E., Bugdadi, A., Sawaya, R., Sabbagh, A. J., & Del Maestro, R. (2019). A comparison of visual rating scales and simulated virtual reality metrics in neurosurgical training: a generalizability theory study. World Neurosurgery, 127, 230–235. https://doi.org/10.1016/j.wneu.2019.03.059.

    Article  Google Scholar 

  • Wu, J., Hu, L., Zhang, G., Liang, Q., Meng, Q., & Wan, C. (2016). Development and validation of the nasopharyngeal cancer scale among the system of quality of life instruments for cancer patients (QLICP-NA V2.0): combined classical test theory and generalizability theory. Quality of Life Research, 25, 2087–2100. https://doi.org/10.1007/s11136-016-1251-4.

    Article  PubMed  Google Scholar 

  • Xu, W., Oei, T. P. S., Liu, X., Wang, X., & Ding, C. (2016). The moderating and mediating roles of self-acceptance and tolerance to others in the relationship between mindfulness and subjective well-being. Journal of Health Psychology, 21(7), 1446–1456. https://doi.org/10.1177/1359105314555170.

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

We would like to thank all participants who were involved in this study.

Author information

Authors and Affiliations

  1. School of Psychology, University of Waikato, Rm K 1.23, Hillcrest, 3240, Private Bag, Hamilton, 3105, New Zealand

    Rebecca A. Chalmers & Oleg N. Medvedev

  2. Department of Community Dentistry and Behavioral Science, Pain Research and Intervention Center of Excellence (PRICE), College of Dentistry, University of Florida, 2004 Mowry Road, Gainesville, FL, 32610-0404, USA

    Steven D. Pratscher

  3. University of Missouri, MO, Columbia, USA

    B. Ann Bettencourt

Authors
  1. Rebecca A. Chalmers
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Steven D. Pratscher
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. B. Ann Bettencourt
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Oleg N. Medvedev
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

RAC: designed and conducted the study, analyzed the data, and wrote the paper. SDP: collaborated with developing the study, collecting the data, and writing the manuscript. BAB: collaborated with developing the study, collecting the data, and editing the manuscript. ONM: collaborated with designing and conducting the study, analyzing the data, and writing the paper.

Corresponding author

Correspondence to Oleg N. Medvedev.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethics Statement

The study complied with the guidelines of the University of Missouri ethics committee and University of Waikato ethics committee, which are based on internationally accepted ethical standards.

Informed Consent

All participants involved in this study provided their informed consent.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic Supplementary Material

ESM 1

(DOCX 17 kb)

ESM 2

(DOCX 19 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chalmers, R.A., Pratscher, S.D., Bettencourt, B.A. et al. Applying Generalizability Theory to Differentiate Between Trait and State in the Interpersonal Mindfulness Scale (IMS). Mindfulness 12, 613–622 (2021). https://doi.org/10.1007/s12671-020-01520-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12671-020-01520-5

Keywords

Search

Navigation