Skip to main content
SpringerLink
Log in

Estimating anchor-based minimal important change using longitudinal confirmatory factor analysis

  • Published:
Quality of Life Research Aims and scope Submit manuscript

Abstract

Purpose

The minimal important change (MIC) is defined as the smallest within-individual change in a patient-reported outcome measure (PROM) that patients on average perceive as important. We describe a method to estimate this value based on longitudinal confirmatory factor analysis (LCFA). The method is evaluated and compared with a recently published method based on longitudinal item response theory (LIRT) in simulated and real data. We also examined the effect of sample size on bias and precision of the estimate.

Methods

We simulated 108 samples with various characteristics in which the true MIC was simulated as the mean of individual MICs, and estimated MICs based on LCFA and LIRT. Additionally, both MICs were estimated in existing PROMIS Pain Behavior data from 909 patients. In another set of 3888 simulated samples with sample sizes of 125, 250, 500, and 1000, we estimated LCFA-based MICs.

Results

The MIC was equally well recovered with the LCFA-method as using the LIRT-method, but the LCFA analyses were more than 50 times faster. In the Pain Behavior data (with higher scores indicating more pain behavior), an LCFA-based MIC for improvement was estimated to be 2.85 points (on a simple sum scale ranging 14–42), whereas the LIRT-based MIC was estimated to be 2.60. The sample size simulations showed that smaller sample sizes decreased the precision of the LCFA-based MIC and increased the risk of model non-convergence.

Conclusion

The MIC can accurately be estimated using LCFA, but sample sizes need to be preferably greater than 125.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

The R-code used to simulate and analyze the samples is provided in the Supplementary material.

The empirical example data can be obtained from the first author on reasonable request.

Code availability

The R-code is provided in the Supplementary material.

References

  1. FDA. (2019). Incorporating clinical outcome assessments into endpoints for regulatory decision-making. US Food and Drug Administration.

    Google Scholar 

  2. Jaeschke, R., Singer, J., & Guyatt, G. H. (1989). Measurement of health status. Ascertaining the minimal clinically important difference. Controlled Clinical Trials, 10, 407–415.

    Article  CAS  PubMed  Google Scholar 

  3. Terwee, C. B., Peipert, J. D., Chapman, R., Lai, J. S., Terluin, B., Cella, D., Griffith, P., & Mokkink, L. B. (2021). Minimal important change (MIC): A conceptual clarification and systematic review of MIC estimates of PROMIS measures. Quality of Life Research, 30(10), 2729–2754.

    Article  PubMed  PubMed Central  Google Scholar 

  4. King, M. T. (2011). A point of minimal important difference (MID): A critique of terminology and methods. Expert Review of Pharmacoeconomics & Outcomes Research, 11, 171–184.

    Article  Google Scholar 

  5. Terluin, B., Eekhout, I., & Terwee, C. B. (2017). The anchor-based minimal important change, based on receiver operating characteristic analysis or predictive modeling, may need to be adjusted for the proportion of improved patients. Journal of Clinical Epidemiology, 83, 90–100.

    Article  PubMed  Google Scholar 

  6. Vanier, A., Sebille, V., Blanchin, M., & Hardouin, J. B. (2021). The minimal perceived change: A formal model of the responder definition according to the patient’s meaning of change for patient-reported outcome data analysis and interpretation. BMC Medical Research Methodology, 21(1), 128.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Bjorner, J. B., Terluin, B., Trigg, A., Hu, J., Brady, K. J. S., & Griffiths, P. (2023). Establishing thresholds for meaningful within-individual change using longitudinal item response theory. Quality of Life Research, 32(5), 1267–1276.

    Article  PubMed  Google Scholar 

  8. Takane, Y., & Deleeuw, J. (1987). On the relationship between item response theory and factor-analysis of discretized variables. Psychometrika, 52(3), 393–408.

    Article  Google Scholar 

  9. Wirth, R. J., & Edwards, M. C. (2007). Item factor analysis: Current approaches and future directions. Psychological Methods, 12(1), 58–79.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  10. Embretson, S. E., & Reise, S. P. (2009). Item response theory for psychologists (2nd ed.). Lawrence Erlbaum.

    Google Scholar 

  11. Kamata, A., & Bauer, D. J. (2008). A note on the relation between factor analytic and item response theory. Structural Equation Modeling-a Multidisciplinary Journal, 15(1), 136–153.

    Article  Google Scholar 

  12. Samejima, F. (1996). The graded response model. In W. J. van der Linden & R. Hambleton (Eds.), Handbook of modern item response theory (pp. 85–100). Springer.

    Google Scholar 

  13. Chalmers, R. P. (2012). mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6), 1–29.

    Article  Google Scholar 

  14. Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48, 2.

    Article  Google Scholar 

  15. R Core Team. (2016). R: A language and environment for statistical computing. R Foundation for Statistical Computing.

    Google Scholar 

  16. Schuller, W., Terwee, C. B., Klausch, T., Roorda, L. D., Rohrich, D. C., Ostelo, R. W., Terluin, B., & de Vet, H. C. W. (2019). Psychometric properties of the Dutch-Flemish patient-reported outcomes measurement information system pain behavior item bank in patients with musculoskeletal complaints. The Journal of Pain, 20(11), 1328–1337.

    Article  PubMed  Google Scholar 

  17. Crins, M. H. P., Roorda, L. D., Smits, N., de Vet, H. C. W., Westhovens, R., Cella, D., Cook, K. F., Revicki, D., van Leeuwen, J., Boers, M., Dekker, J., & Terwee, C. B. (2016). Calibration of the Dutch-Flemish PROMIS pain behavior item bank in patients with chronic pain. European Journal of Pain, 20(2), 284–296.

    Article  CAS  PubMed  Google Scholar 

  18. Gasparini, A. (2018). rsimsum: Summarise results from Monte Carlo simulation studies. Journal of Open Source Software, 3(26), 739.

    Article  Google Scholar 

  19. Terluin, B., Griffiths, P., Trigg, A., Terwee, C. B., & Bjorner, J. B. (2022). Present state bias in transition ratings was accurately estimated in simulated and real data. Journal of Clinical Epidemiology, 143, 128–136.

    Article  PubMed  Google Scholar 

  20. Morris, T. P., White, I. R., & Crowther, M. J. (2019). Using simulation studies to evaluate statistical methods. Statistics in Medicine, 38(11), 2074–2102.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Deyo, R. A., & Centor, R. M. (1986). Assessing the responsiveness of functional scales to clinical change: An analogy to diagnostic test performance. J Chron Dis, 39, 897–906.

    Article  CAS  PubMed  Google Scholar 

  22. Terluin, B., Eekhout, I., Terwee, C. B., & de Vet, H. C. W. (2015). Minimal important change (MIC) based on a predictive modeling approach was more precise than MIC based on ROC analysis. Journal of Clinical Epidemiology, 68, 1388–1396.

    Article  PubMed  Google Scholar 

  23. Terluin, B., Eekhout, I., & Terwee, C. B. (2022). Improved adjusted minimal important change took reliability of transition ratings into account. Journal of Clinical Epidemiology, 148, 48–53.

    Article  PubMed  Google Scholar 

  24. Hays, R. D., Brodsky, M., Johnston, M. F., Spritzer, K. L., & Hui, K. K. (2005). Evaluating the statistical significance of health-related quality-of-life change in individual patients. Evaluation and the Health Professions, 28(2), 160–171.

    Article  PubMed  Google Scholar 

  25. Trigg, A., Lenderking, W. R., & Boehnke, J. R. (2023). Introduction to the special section: “Methodologies and considerations for meaningful change.” Qual Life Res. https://doi.org/10.1007/s11136-023-03413-1

    Article  PubMed  Google Scholar 

  26. Terluin, B., Koopman, J. E., Hoogendam, L., Griffiths, P., Terwee, C. B., & Bjorner, J. B. (2023). Estimating meaningful thresholds for multi-item questionnaires using item response theory. Quality of Life Research, 32(6), 1819–1830.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Funding

No funds, grants, or other support was received.

Author information

Authors and Affiliations

  1. Department of General Practice, Amsterdam UMC, Vrije Universiteit Amsterdam, de Boelelaan 1117, 1081 HV, Amsterdam, The Netherlands

    Berend Terluin

  2. Amsterdam Public Health research institute, Amsterdam, The Netherlands

    Berend Terluin & Caroline B. Terwee

  3. Medical Affairs Statistics, Bayer plc, Reading, UK

    Andrew Trigg

  4. SeeingTheta, 2 Chemin des Vaux, 49400, Saumur, France

    Piper Fromy

  5. Department of Epidemiology and Data Science, Amsterdam UMC, Vrije Universiteit Amsterdam, de Boelelaan 1117, 1081 HV, Amsterdam, The Netherlands

    Wouter Schuller & Caroline B. Terwee

  6. Spine Clinic, Provinciale weg 152-154, 1506 ME, Zaandam, The Netherlands

    Wouter Schuller

  7. QualityMetric, Johnston, Rhode Island, USA

    Jakob B. Bjorner

  8. Department of Public Health, University of Copenhagen, Copenhagen, Denmark

    Jakob B. Bjorner

  9. National Research Centre for the Working Environment, Copenhagen, Denmark

    Jakob B. Bjorner

Authors
  1. Berend Terluin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrew Trigg
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Piper Fromy
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wouter Schuller
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Caroline B. Terwee
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jakob B. Bjorner
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization: BT; Methodology: BT, AT, JBB; Formal analysis and investigation: BT; Writing—original draft preparation: BT; Writing—review and editing: BT, AT, PF, WS, CBT, JBB; Supervision: JBB.

Corresponding author

Correspondence to Berend Terluin.

Ethics declarations

Conflicts of interest

The authors have no relevant financial or non-financial interests to disclose.

Ethical approval

With respect to the Pain Behavior data, the Medical Ethical Committee of the VU Medical Center at Amsterdam (2013/20) approved the study and accepted a verbal-informed consent procedure.

Consent to participate

Verbal informed consent was obtained from all patients providing the Pain Behavior data.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 63 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Terluin, B., Trigg, A., Fromy, P. et al. Estimating anchor-based minimal important change using longitudinal confirmatory factor analysis. Qual Life Res 33, 963–973 (2024). https://doi.org/10.1007/s11136-023-03577-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-023-03577-w

Keywords

Search

Navigation