Skip to main content

Advertisement

SpringerLink
Log in

The importance of development standards for anchoring vignettes: an illustrative example from pediatric localized scleroderma quality of life

  • Published:
Quality of Life Research Aims and scope Submit manuscript

Abstract

Purpose

Anchoring vignettes (AVs) are a promising measurement technique to reduce bias in patient-reported outcome (PRO) measures by helping researchers understand differences in how individuals and groups interpret response options. However, little attention has been paid to ensure quality development of AVs, and their performance has not been well assessed in pediatric populations. In this study, we explore the application of a rigorous development process for AVs based upon current standards for PROs, as well as feasibility of AVs when administered to children and adolescents.

Methods

We developed AVs using a rigorous, patient-centered mixed methods process including three phases: (1) development, (2) a pilot study, and (3) a field test. Our proposed process included the generation of a conceptual framework based on the PRO, the Localized Scleroderma Quality of Life Instrument, and numerous vignette-specific considerations. We qualitatively explored readability and comprehension of the AVs (pilot study) and then analyzed ranking patterns within vignette sets (field test).

Results

Four sets of four vignettes were developed. Revisions were suggested at each phase of development. The pilot study demonstrated that children ≥ 10 years had no trouble indicating understanding of the AVs. In the field test, although appropriate rankings of vignettes were generally demonstrated by participants, the percentage of tied rankings was higher than expected in this pediatric group.

Conclusions

This work supports the need for rigorous developmental standards for AVs, as each stage of development suggested revisions. Additionally, AVs showed initial promise for use with pediatric populations; general feasibility and understanding were supported.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. U. S. Department of Health and Human Services, F. a. D. A., Center for Drug Evaluation and Research, Center for Biologics Evaluation and Research, and Center for Devices and Radiological Health. (2006). Guidance for industry: Patient-reported outcome measures: use in medical product development to support labeling claims: Draft guidance. Health Qual Life Outcomes, 4, 79. https://doi.org/10.1186/1477-7525-4-79.

    Article  Google Scholar 

  2. Reeve, B. B., Wyrwich, K. W., Wu, A. W., Velikova, G., Terwee, C. B., Snyder, C. F., et al. (2013). ISOQOL recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research. Quality of Life Research, 22(8), 1889–1905. https://doi.org/10.1007/s11136-012-0344-y.

    Article  PubMed  Google Scholar 

  3. Eignor D.R. (2013). The standards for educational and psychological testing: American psychological association.

  4. King, G., Murray, C. J., Salomon, J. A., & Tandon, A. (2004). Enhancing the validity and cross-cultural comparability of measurement in survey research. American Political Science Review, 98(1), 191–207.

    Article  Google Scholar 

  5. Grol-Prokopczyk, H., Freese, J., & Hauser, R. M. (2011). Using anchoring vignettes to assess group differences in general self-rated health. Journal of Health and Social Behavior, 52(2), 246–261. https://doi.org/10.1177/0022146510396713.

    Article  PubMed  PubMed Central  Google Scholar 

  6. King, G., & Wand, J. (2007). Comparing incomparable survey responses: Evaluating and selecting anchoring vignettes. Political Analysis, 15(1), 46–66.

    Article  Google Scholar 

  7. Schwartz, C. E. (2010). Applications of response shift theory and methods to participation measurement: A brief history of a young field. Archives of Physical Medicine Rehabilitation, 91(9), S38–S43.

    Article  Google Scholar 

  8. Bolt, D. M., Lu, Y., & Kim, J.-S. (2014). Measurement and control of response styles using anchoring vignettes: A model-based approach. Journal Psychological Methods, 19(4), 528.

    Article  Google Scholar 

  9. Primi R., Zanon C., Santos D., De Fruyt F., John O.P. (2016). Anchoring vignettes. European Journal of Psychological Assessment.

  10. Vonkova, H., Bendl, S., & Papajoanu, O. (2017). How students report dishonest behavior in school: Self-assessment and anchoring vignettes. The Journal of Experimental Education, 85(1), 36–53.

    Article  Google Scholar 

  11. Vonkova H., Zamarro G., Deberg V., Hitt C. (2015). Comparisons of student perceptions of teacher’s performance in the classroom: Using parametric anchoring vignette methods for improving comparability. EDRE Working Paper No. 2015–01

  12. Marksteiner T., Kuger S., Klieme E. (2018). The potential of anchoring vignettes to increase intercultural comparability of non-cognitive factors. Assessment in Education: Principles, Policy & Practice, 1–21.

  13. Vonkova, H., & Hrabak, J. (2015). The (in) comparability of ICT knowledge and skill self-assessments among upper secondary school students: The use of the anchoring vignette method. Computers Education, 85, 191–202.

    Article  Google Scholar 

  14. Vonkova, H. (2019). Life satisfaction among different groups of children: self-reports, differential scale usage and anchoring vignettes. Child Indicators Research. https://doi.org/10.1007/s12187-019-09629-3.

    Article  Google Scholar 

  15. Primi R., Santos D., John O.P., De Fruyt F., Hauck-Filho N. (2017). Dealing with person differential item functioning in social-emotional skill assessment using anchoring vignettes. In The Annual Meeting of the Psychometric Society, (pp. 275–286): Springer

  16. Grol-Prokopczyk, H., Verdes-Tennant, E., McEniry, M., & Ispány, M. (2015). Promises and pitfalls of anchoring vignettes in health survey research. Demography, 52(5), 1703–1728.

    Article  Google Scholar 

  17. Grol-Prokopczyk, H. (2018). In pursuit of anchoring vignettes that work: evaluating generality versus specificity in vignette texts. The Journals of Gerontology: Series B, 73(1), 54–63. https://doi.org/10.1093/geronb/gbx048.

    Article  Google Scholar 

  18. Jürges, H., & Winter, J. (2013). Are anchoring vignette ratings sensitive to vignette age and sex? Health Economics, 22(1), 1–13. https://doi.org/10.1002/hec.1806.

    Article  PubMed  Google Scholar 

  19. Zigler, C. K., Ardalan, K., Lane, S., Schollaert, K. L., & Torok, K. S. (2019). A novel patient-reported outcome for paediatric localized scleroderma: A qualitative assessment of content validity. British Journal of Dermatology. https://doi.org/10.1111/bjd.18512.

    Article  PubMed  Google Scholar 

  20. Zulian, F., Athreya, B. H., Laxer, R., Nelson, A. M., Feitosa de Oliveira, S. K., Punaro, M. G., et al. (2006). Juvenile localized scleroderma: Clinical and epidemiological features in 750 children. An international study. Rheumatology (Oxford), 45(5), 614–620. https://doi.org/10.1093/rheumatology/kei251.

    Article  CAS  Google Scholar 

  21. Zigler, C. K., Ardalan, K., Hernandez, A., Calliendo, A. E., Magee, K., Mann, C. M., et al. (2020). Exploring the impact of paediatric localized scleroderma on health-related quality of life: Focus groups with youth and caregivers. British Journal of Dermatology. https://doi.org/10.1111/bjd.18879.

    Article  PubMed  Google Scholar 

  22. Hopkins, D. J., & King, G. (2010). Improving anchoring vignettes: Designing surveys to correct interpersonal incomparability. Public Opinion Quarterly, 74(2), 201–222.

    Article  Google Scholar 

  23. Collins, D. (2003). Pretesting survey instruments: An overview of cognitive methods. Quality of Life Research, 12(3), 229–238.

    Article  Google Scholar 

  24. Wand, J. (2013). Credible comparisons using interpersonally incomparable data: Nonparametric scales with anchoring vignettes. American Journal of Political Science, 57(1), 249–262.

    Article  Google Scholar 

  25. Wand J., King G., Lau O. (2011). Anchors: Software for anchoring vignette data. Journal of Statistical Software.

  26. Stasiulis, E., Gladstone, B., Boydell, K., O'Brien, C., Pope, E., & Laxer, R. M. (2018). Children with facial morphoea managing everyday life: a qualitative study. British Journal of Dermatology, 179(2), 353–361. https://doi.org/10.1111/bjd.16449.

    Article  CAS  PubMed  Google Scholar 

  27. Murray, K. J., & Laxer, R. M. (2002). Scleroderma in children and adolescents. Rheumatic Diseases Clinics of North America, 28(3), 603–624.

    Article  Google Scholar 

  28. Wu, E. Y., Li, S. C., Torok, K. S., Virkud, Y. V., Fuhlbrigge, R. C., Rabinovich, C. E., et al. (2019). Baseline description of the juvenile localized scleroderma subgroup from the childhood arthritis and rheumatology research alliance legacy registry. ACR Open Rheumatology, 1(2), 119–124. https://doi.org/10.1002/acr2.1019.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Varni, J. W., Limbers, C. A., & Burwinkle, T. M. (2007). How young can children reliably and validly self-report their health-related quality of life? An analysis of 8,591 children across age subgroups with the PedsQL 4.0 Generic Core Scales. Health Qual Life Outcomes, 5, 1–1. https://doi.org/10.1186/1477-7525-5-1.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Riley, A. W. (2004). Evidence that school-age children can self-report on their health. Ambulatory Pediatrics, 4(4 Suppl), 371–376. https://doi.org/10.1367/a03-178r.1.

    Article  PubMed  Google Scholar 

  31. Rieffe, C., Ketelaar, L., & Wiefferink, C. (2010). Assessing empathy in young children: Construction and validation of an Empathy Questionnaire (EmQue). Personality and Individual Differences, 49, 362–367. https://doi.org/10.1016/j.paid.2010.03.046.

    Article  Google Scholar 

  32. von Davier, M., Shin, H. J., Khorramdel, L., & Stankov, L. (2018). The Effects of vignette scoring on reliability and validity of self-reports. Applied Psychological Measurement, 42(4), 291–306. https://doi.org/10.1177/0146621617730389.

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Rose Cannon, Jennifer Coias, Dafonso Davage, Stephanie Flores-Pollack, Smriti Prasad, and Noelle Teske for their invaluable assistance with this project, as well as Clement Stone for his insight and input into project design. We also would like to acknowledge the vital involvement of the participants and their families, whose experiences drive this work.

Funding

Dr. Torok was supported by The Nancy Taylor Foundation for Chronic Diseases, Inc. (PI: Torok) and NIH-NIAMS (1K23AR059722, PI: Torok). Dr. Jacobe was funded by the James N. Gilliam, M.D. Chair in Dermatology.

Author information

Authors and Affiliations

  1. Department of Population Health Sciences, Duke University School of Medicine, Durham, USA

    Christina K. Zigler & Theresa M. Coles

  2. Department of Dermatology, UT Southwestern Medical Center, Dallas, TX, USA

    Heidi Jacobe

  3. Division of Pediatric Rheumatology, Department of Pediatrics, Duke University School of Medicine, Durham, USA

    Kaveh Ardalan

  4. School of Education, University of Pittsburgh, Pittsburgh, USA

    Suzane Lane

  5. University of Pittsburgh School of Medicine, Pittsburgh, USA

    Kaila L. Schollaert & Kathryn S. Torok

  6. Division of Rheumatology, Department of Pediatrics, Children’s Hospital of Pittsburgh of UPMC, Pittsburgh, USA

    Kathryn S. Torok

Authors
  1. Christina K. Zigler
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Heidi Jacobe
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kaveh Ardalan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Theresa M. Coles
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Suzane Lane
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kaila L. Schollaert
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Kathryn S. Torok
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christina K. Zigler.

Ethics declarations

Conflict of interest

Dr. Zigler declares she has no conflicts of interest. Dr. Jacobe declares she has no conflicts of interest. Dr. Ardalan declares he has no conflicts of interest. Ms. Schollaert declares she has no conflicts of interest. Dr. Lane declares she has no conflicts of interest. Dr. Torok declares she has no conflicts of interest. Dr. Coles declares that she has no conflicts of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the University of Pittsburgh and UT Southwestern Institutional Review Boards (IRB) and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Specifically, the qualitative pilot study occurred at and was approved by the University of Pittsburgh IRB and the quantitative field test occurred at and was approved by both the University of Pittsburgh and the UT Southwestern IRBs.

Informed consent

Informed consent was obtained from all individual participants included in the study. Adult participant (> 18 yrs old) consent and parental consent/child assent (for minor participants) were obtained in accordance with each institution’s policy.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zigler, C.K., Jacobe, H., Ardalan, K. et al. The importance of development standards for anchoring vignettes: an illustrative example from pediatric localized scleroderma quality of life. Qual Life Res 29, 3263–3272 (2020). https://doi.org/10.1007/s11136-020-02575-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-020-02575-6

Keywords

Search

Navigation