Skip to main content
SpringerLink
Log in

Development and validation of a theoretical test in basic laparoscopy

  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

Testing of knowledge is an important component in a successful skills curriculum. Nonetheless, structured testing of basic procedure-relevant knowledge in the surgical domains is not ordinary practice. A regional need assessment showed insufficient knowledge regarding basic laparoscopy for first-year residents in obstetrics and gynecology. This study therefore aimed to develop and validate a framework for a theoretical knowledge test, a multiple-choice test, in basic theory related to laparoscopy.

Methods

The content of the multiple-choice test was determined by conducting informal conversational interviews with experts in laparoscopy. The subsequent relevance of the test questions was evaluated using the Delphi method involving regional chief physicians. Construct validity was tested by comparing test results from three groups with expected different clinical competence and knowledge levels: senior medical students, first-year residents, and chief physicians.

Results

The four conversational interviews resulted in the development of 47 test questions, which were narrowed down to 37 test questions after two Delphi rounds involving 12 chief physicians. Significant differences were found between the test scores from the senior medical students (n = 14) and the first-year residents (n = 52) (median test scores, 18 vs. 24, respectively; p = 0.001), and between the first-year residents and the chief physicians (n = 12) (median test scores, 24 vs. 33, respectively; p = 0.001). Internal consistency (Cronbach’s alpha) was 0.82. There was no evidence of differential item functioning between the three groups tested.

Conclusions

A newly developed knowledge test in basic laparoscopy proved to have content and construct validity. The formula for the development and validation of a theoretical test could potentially be used for any topics that require structured testing of knowledge.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Stefanidis D, Heniford BT (2009) The formula for a successful laparoscopic skills curriculum. Arch Surg 144:77–82

    Article  PubMed  Google Scholar 

  2. Zheng B, Hur H-C, Johnson S, Swanstrom LL (2010) Validity of using Fundamentals of Laparoscopic Surgery (FLS) program to assess laparoscopic competence for gynecologists. Surg Endosc 24:152–160

    Article  PubMed  Google Scholar 

  3. Kohls-Gatzoulis JA, Regehr G, Hutchison C (2004) Teaching cognitive skills improves learning in surgical skills courses: a blinded, prospective, randomized study. Can J Surg 47:277–283

    PubMed  Google Scholar 

  4. Peters JH, Fried GM, Swanstrom LL, Soper NJ, Sillin LF, Schirmer B, Hofmann K, SAGESFLS Committee (2004) Development and validation of a comprehensive program of education and assessment of the basic fundamentals of laparoscopic surgery. Surgery 135:21–27

    Article  PubMed  Google Scholar 

  5. Case SM, Swanson DB (1998) Constructing written test questions for the basic and clinical sciences, 2nd edn. National Board of Medical Examiners, Philadelphia

    Google Scholar 

  6. Downing SM, Yudkowsky R (2009) Written tests. Assessment in health professions education, 1st edn. Routledge, New York, pp 149–184

    Google Scholar 

  7. Schuwirth LWT, van der Vleuten CPM (2003) ABC of learning and teaching in medicine: written assessment. BMJ 22:643–645

    Article  Google Scholar 

  8. van der Vleuten CPM, Schhwirth LWT (2009) Written assessments: a practical guide for medical teachers, 3rd edn. Churchill Livingstone, London, pp 323–331

    Google Scholar 

  9. Schubert S, Ortwein H, Dumitsch A, Schwantes U, Wilhelm O, Kiessling C (2008) A situational judgement test of professional behaviour: development and validation. Med Teach 30:528–533

    Article  PubMed  Google Scholar 

  10. Schout BMA, Hendrikx AJM, Scheele F, Bemelmans BLH, Scherpbier AJJA (2010) Validation and implementation of surgical simulators: a critical review of present, past, and future. Surg Endosc 24:536–546

    Article  PubMed  CAS  Google Scholar 

  11. Norman GR, Streiner DL (2008) Validity. Health measurement scales: a practical guide to their development and use., 4th edn. Oxford University Press, Oxford, pp 247–276

    Google Scholar 

  12. Haladyna T, Downing S, Rodriguez M (2002) A review of multiple-choice item-writing guidelines for classroom assessment. Appl Meas Educ 15:309–334

    Article  Google Scholar 

  13. Graham B, Regehr G (2003) Delphi as a method to establish consensus for diagnostic criteria. J Clin Epidemiol 56:1150–1156

    Article  PubMed  Google Scholar 

  14. Swaminathan H (1990) Detecting differential item functioning using logistic regression procedures. Educ Meas 27:361–370

    Article  Google Scholar 

  15. Zumbo B (1999) A handbook on the theory and methods of differential item functioning (DIF). National Defense Headquarters, Ottawa

    Google Scholar 

  16. Cohen L, Manion L, Morrison K (2000) Interviews. Research methods in education, 5th edn. Routledge, New York, pp 265–292

    Book  Google Scholar 

  17. Wass V, Van der Vleuten C, Shatzer J, Jones R (2001) Assessment of clinical competence. Lancet 24:945–949

    Article  Google Scholar 

  18. Hawkins RE, Swanson DB (2008) Using written examinations to assess medical knowledge and its application: practical guide to the evaluation of clinical competence, 1st edn. Mosby, Philadelphia, pp 42–59

    Google Scholar 

  19. Fried GM (2006) Lessons from the surgical experience with simulators: incorporation into training and utilization in determining competency. Gastrointest Endosc Clin N Am 16:425–434

    Article  PubMed  Google Scholar 

  20. Scott DJ, Valentine RJ, Bergen PC, Rege RV, Laycock R, Tesfay ST, Jones DB (2000) Evaluating surgical competency with the American board of surgery in-training examination, skill testing, and intraoperative assessment. Surgery 128:613–622

    Article  PubMed  CAS  Google Scholar 

  21. Sloan DA, Donnelly MB, Schwartz RW, Strodel WE (1995) The objective structured clinical examination: the new gold standard for evaluating postgraduate clinical performance. Ann Surg 222:735–742

    Article  PubMed  CAS  Google Scholar 

  22. Swanstrom LL, Fried GM, Hoffman KI, Soper NJ (2006) Beta test results of a new system assessing competence in laparoscopic surgery. J Am Coll Surg 202:62–69

    Article  PubMed  Google Scholar 

  23. Larsen DP, Butler AC, Roediger HL (2008) Test-enhanced learning in medical education. Med Educ 42:959–966

    Article  PubMed  Google Scholar 

  24. Wentink M, Stassen LPS, Alwayn I, Hosman RJAW, Stassen HG (2003) Rasmussen’s model of human behavior in laparoscopy training. Surg Endosc 17:1241–1246

    Article  PubMed  CAS  Google Scholar 

Download references

Acknowledgments

We thank Karl Bang Christensen, Associate Professor, Department of Public Health, Unit of Biostatistics, for statistical help. We also thank Dorte Nielsen, Chief Physician, Department of Obstetrics and Gynecology, Rigshospitalet, University Hospital of Copenhagen; Jens Jørgen Kjer, Chief Physician, Department of Obstetrics and Gynecology, Rigshospitalet, University Hospital of Copenhagen; and Helle V. Clausen, Chief Physician, Department of Obstetrics and Gynecology, Herlev Hospital, University Hospital of Copenhagen for their participation in the interview part of the study. Furthermore, we thank all the chief physicians from the consensus panel.

Disclosures

Jeanett Strandbygaard, Mathilde Maagaard, Christian Rifbjerg Larsen, Lars Schouenborg, Christian Ottosen, Charlotte Ringsted, Teodor Grantcharov, Bent Ottesen, and Jette Led Sorensen have no conflicts of interest or financial ties to declare.

Author information

Authors and Affiliations

  1. Department of Obstetrics and Gynecolology, 4074, Rigshospitalet, University Hospital of Copenhagen, Blegdamsvej 9, 2100, Copenhagen, Denmark

    Jeanett Strandbygaard, Mathilde Maagaard, Lars Schouenborg, Christian Ottosen & Jette Led Sorensen

  2. Department of Obstetrics and Gynaecology, Hillerød Hospital, Dyrehavevej 29, 3400, Hillerød, Denmark

    Christian Rifbjerg Larsen

  3. Centre of Clinical Education, University of Copenhagen, Copenhagen and Capital Region, Blegdamsvej 9, 2100, Copenhagen, Denmark

    Charlotte Ringsted

  4. Department of Surgery, St. Michael’s Hospital, 30 Bond Street, Toronto, ON, M5B 1W8, Canada

    Teodor Grantcharov

  5. Juliane Marie Centre, Centre for Children, Women, and Reproduction, Blegdamsvej 9, 2100, Copenhagen, Denmark

    Bent Ottesen

Authors
  1. Jeanett Strandbygaard
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mathilde Maagaard
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christian Rifbjerg Larsen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lars Schouenborg
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Christian Ottosen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Charlotte Ringsted
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Teodor Grantcharov
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Bent Ottesen
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Jette Led Sorensen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jeanett Strandbygaard.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Strandbygaard, J., Maagaard, M., Larsen, C.R. et al. Development and validation of a theoretical test in basic laparoscopy. Surg Endosc 27, 1353–1359 (2013). https://doi.org/10.1007/s00464-012-2615-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-012-2615-7

Keywords

Search

Navigation