Journal of Cancer Education

, Volume 28, Issue 3, pp 591–596

Assessing the Effectiveness of a Grand Rounds CME Activity for Health-Care Professionals


    • CE Outcomes, LLC
  • P. Holder Nevins
    • CE Outcomes, LLC
  • J. Chad Williamson
    • CE Outcomes, LLC
  • Brian Tomlinson
    • Lymphoma Research Foundation

DOI: 10.1007/s13187-013-0507-8

Cite this article as:
Glauser, T.A., Nevins, P.H., Williamson, J.C. et al. J Canc Educ (2013) 28: 591. doi:10.1007/s13187-013-0507-8


The Lymphoma Research Foundation offers Grand Rounds continuing medical education (CME) activities on specific issues related to advances in the management of patients with lymphoma. The 2012 activity comprised interactive case studies presented by local lymphoma experts. A case-based survey was designed to assess whether the management choices of program participants are consistent with the evidence-based content of the CME activity. This survey was administered to participants 1 month after completion of the CME activity and also to a control group who did not participate in the educational program. Participants were more aware of the epidemiology of CD20-positive tumors than were controls and were more likely to appropriately diagnose primary mediastinal large B cell lymphoma (PMBCL), use evidence-based second-line therapy for PMBCL, and properly manage a patient with classic Hodgkin lymphoma that did not respond to standard therapy. Participants were also more confident than controls in their ability to interpret histology and cytogenetic testing for selecting an optimal treatment.


Case vignetteSurveyLymphoma


Lymphoma is the most common hematologic malignancy, with an estimated 79,000 new cases occurring in 2012 [1]. Of these, about 70,000 were non-Hodgkin lymphoma (NHL) and 9,000 were Hodgkin lymphoma (HL) [1]. The diagnosis and management of patients with both HL and NHL continues to evolve, and for both types of lymphoma, evolving treatments have improved the 5-year survival rate significantly since the 1970s [1]. To assist health-care professionals in keeping abreast of these ongoing changes, the Lymphoma Research Foundation hosts continuing medical education (CME) programs. The 2012 event, titled Chicago Lymphoma Rounds, provided a forum for health-care professionals who care for patients with lymphoma to meet regularly and address issues specific to the diagnosis and treatment of lymphoma. The format of the program was case-based education, including lectures from local lymphoma experts. The Lymphoma Research Foundation engaged CE Outcomes, LLC, to measure the effectiveness of this activity in improving the knowledge of participants and their likelihood of using current evidence-based regimens for their care of patients with lymphoma. This article details the findings from this research.


There were four learning objectives for the program: educating attendees on the latest developments in lymphoma diagnosis, treatment, and management; sharing best practices for treating lymphoma patients; improving clinical practice and optimizing lymphoma patient care; and building a network of health-care professionals who work with lymphoma patients.

CE Outcomes physicians reviewed the learning objectives and content to define a series of key measurement indicators focused on the management of patients with lymphoma. Measurement indicators are individual evidence-based statements that outline the health-care performance expectations associated with the content of an educational activity. Those identified from this program and used to develop the case vignette survey are listed in Table 1.
Table 1

Measurement indicators

Be aware that the value of interim PET imaging remains unclear [13, 14].

Be aware that a recent study by Gallamini et al. showed that for patients with positive PET after cycle 2 of ABVD, those with positive studies did better when switched to BEACOPP, compared with historical controls. They should also be aware that BEACOPP has a higher risk of AML/MDS and results in infertility. Further, a study by Schmitz et al. shows that patients with relapsed disease do better with high-dose chemo and stem cell transplant (SCT) than thosethat just get high-dose chemotherapy [1517].

Know that CD20 is expressed in 21 to 43 % of classical HL [18].

Know that a patient positive for CD79a, Pax-5, CD30, but negative for CD20, who has cells with pale cytoplasm, has a presentation consistent with PMBCL. Further, patients with PMBCL have a median age at diagnosis of 35 years, gender predominance is female, it usually manifests in the mediastinal/supraclavicular lymph nodes, and bone marrow involvement is rare [19].

Be aware that survival rates of patients with PMBCL treated with R-CHOP were significantly better (P = .003), compared with those randomized to CHOP [20].

Be aware that when multiple studies of agents to treat relapsed lymphoma are compared, RICE-SCT has a 71 % overall response, better than all others except EPOCH [2126].

Be aware that combination approaches to treating Waldenstrom’s macroglobulinemia that include chemotherapy with rituximab yield results that are at least as good as that seen with any agent alone [27].

Know that keratin expression in B-lineage neoplasms is seen in post-germinal center, aggressive lymphomas, and is more common in plasma cell neoplasms.

The case vignette-based survey was presented to program participants and a demographically similar group of nonparticipants. Case vignettes were designed to assess whether the diagnostic and therapeutic choices of participants were consistent with clinical evidence presented in the content of the educational activity. The case vignettes were also used to assess whether practice choices of participants were different from practice choices of nonparticipants. Case vignettes have gained considerable support for their value in predicting physician practice patterns. Results from recent research studies demonstrate that case vignettes (when compared with chart review and standardized patients) are a valid and comprehensive method to measure a physician’s process of care in actual clinical practice. Furthermore, case vignettes are more cost effective and less invasive than other means of measurement [2, 3]. The case vignettes presented are listed in Table 2.
Table 2


Case 1

A 29-year-old previously healthy man presented with rapidly enlarging masses on both sides of his neck over the past month. He had been asymptomatic otherwise and denied weight loss, fever, or night sweats. On physical exam, he had multiple, bilateral, rubbery cervical nodes, the largest 4 cm in size. Lymph node biopsy showed classical HL. On flow cytometry, the lymphoma cells were CD15 and CD20 positive and CD3, CD30, and CD4 negative. Staging PET/CT scans revealed FDG-avid bilateral cervical adenopathy, with involvement of the mediastinum and periaortic nodes but no involvement of the retroperitoneum. Bone marrow biopsy was negative. Before starting therapy, the patient asks about how his treatment will be monitored. After two cycles of ABVD, you elect to get a PET/CT scan, which is positive.

Case 2

A 41-year-old female presented to her primary care provider with worsening dyspnea on exertion. She reported fevers, night sweats, and an unintentional weight loss of 15 lbs. Physical examination demonstrated reduced breath sounds at the right lung base and no palpable lymphadenopathy. A chest x-ray showed a mediastinal mass with no lung infiltrates. The patient was staged by CT scans of the neck, chest, abdomen, and pelvis as well as PET scan, all of which show no other areas of disease. The mediastinal mass biopsy demonstrated sheets of medium- to large-sized cells containing abundant pale cytoplasm with positive staining for CD30, CD20, CD79a, and PAX-5. Bone marrow biopsy did not reveal involvement with lymphoma. Her ejection fraction is 65 %. She was medically fit with no significant comorbidities. Her ECOG performance status (PS) was 1. Six months later, she developed progressive disease.

Case 3

A 67-year-old man was evaluated for worsening fatigue and weakness. He had no neurological symptoms or bone pain. The physical examination did not reveal any lymphadenopathy. The CBC showed Hb, 8.4 g/dL; WBC, 12,000/mL; platelets, 110,000/mL; creatinine, 1.5 mg/dL; and serum protein electrophoresis, 2.9-g/dL monoclonal IgM protein. The bone marrow biopsy revealed 30 % infiltration with lymphoplasmacytic lymphoma. His ECOG PS was 2. A year later, he developed progressive disease. A re-biopsy demonstrated lymphoplasmacytoid cells with MUM1 and cytokeratin staining.

Case 4

A 30-year-old man was evaluated for generalized lymphadenopathy and splenomegaly. He reported low-grade fever, easy fatigability, and weight loss of 10 lbs. On physical exam, he had enlarged lymph nodes involving both sides of the neck, axilla, and inguinal regions. CT scan showed many lymph nodes measuring 2 to 3 cm in the mediastinum, mesenteric, and para-aortic area. Lab studies including CBC, complete metabolic panel, and LDH were unremarkable. A lymph node biopsy was interpreted as classical HL, though with an atypical phenotype, CD20 positive and CD30 negative. He received ABVD chemotherapy, but the lymph nodes remained unchanged after 2 months of treatment.

Additional survey items were included to assess barriers to the optimal management of patients with lymphoma and confidence of participating health-care providers in managing patients with lymphoma. All surveys were field tested with practicing members of the target audience community prior to implementation. Surveys and data analyses were conducted in conjunction with a clinician practicing in oncology. Surveys were reviewed and approved by the Lymphoma Research Foundation prior to distribution.

Survey Implementation

Survey instruments were distributed by CE Outcomes to participants at least 30 days after completing the activity. From the 66 eligible participants, a sample of 30 responses was collected by email and fax. The same survey instrument was distributed to a demographically similar group of clinicians, randomly selected from a proprietary database, who did not attend the CME program. Surveys were distributed by email, and responses were collected and analyzed in comparison with the responses of the participant group.

Data Analysis

Data were analyzed using PASW Statistics 18 (SPSS; Chicago, IL). Data were first arrayed using frequencies. T tests and chi squares were then used to test the differences between the mean evidence-based responses of the participants and the nonparticipants. Differences between the two groups are considered significant if the P value is .10 or less. An effect size was calculated to determine the amount of difference between the evidence-based responses of the participants and nonparticipants. Effect size was calculated using Cohen’s d formula [4] and is expressed as a nonoverlap percentage, or the percentage achieved by participants that was not reflected in the evidence-based responses of nonparticipants.


The survey was completed by 71 health-care providers who participated in the educational activity (participants) and 86 who did not (nonparticipants) (Table 3). For both groups, the mean number of lymphoma patients seen per week was 14. The majority of participants specialized in oncology, although five pathologist participants also took the survey. Nonparticipants all specialized in oncology. Survey respondents were primarily physicians (77 % of participants, 83 % of nonparticipants) and practiced in an urban area (80 and 57 % respectively).
Table 3





Patients seen/week (mean)



Patients with lymphoma seen/week (mean)



Gender (% male)

53 %

67 %

Years since graduation



Attended medical/NP/PA school in the USA

77 %

63 %



  Medical oncology

5 (16.7 %)

6 (20.0 %)


20 (66.7 %)

24 (80.0 %)


5 (16.7 %)




23 (76.7 %)

25 (83.3 %)


7 (23.3 %)

5 (16.7 %)


0 (0 %)

Practice location



24 (80.0 %)

17 (56.7 %)


6 (20.0 %)

12 (40.0 %)


1 (3.3 %)

Practice type


  Solo practice

5 (16.7 %)

6 (20.0 %)

  Group practice

10 (33.3 %)

18 (60.0 %)

  Medical school

5 (16.7 %)

2 (6.7 %)


1 (3.3 %)

  Nongovernment hospital

8 (26.7 %)

1 (3.3 %)


2 (6.7 %)

2 (6.7 %)


Major professional activity


  Direct patient care

24 (80.0 %)

29 (96.7 %)


  Medical education

2 (6.7 %)

  Medical research

4 (13.3 %)

1 (3.3 %)



30 (100 %)

30 (100 %)

Educating Attendees on the Latest Developments in Lymphoma Diagnosis, Treatment, and Management

Significantly more participants (77 %) than nonparticipants (53 %) correctly diagnosed PMBCL in the patient presented in case 2 (P = .058). Additionally, significantly more participants (60 %), compared with nonparticipants (33 %), were aware that CD20 is expressed in 21–43 % of classical Hodgkin lymphomas (P = .038).

Most participants (57 %) and nonparticipants (53 %) were not aware that keratin expression is seen in aggressive post-germinal center lymphomas. With respect to the value of interim PET/CT scans for monitoring patients with classical HL who are receiving treatment (case 1), 53 % of participants, as compared with 37 % of nonparticipants, were aware that the value remains unclear.

Sharing Best Practices for Treating Lymphoma Patients

Most participants (83 %) and nonparticipants (87 %) selected chemotherapy with rituximab as initial treatment for case 3. Significantly more participants (87 %), compared with nonparticipants (50 %), would appropriately perform a re-biopsy for case 4 (Fig. 1).
Fig. 1

Survey responses regarding management of the case presentation: patient with refractory classical HL

For case 2, significantly more participants (83 %) than nonparticipants (53 %) would appropriately treat using R-ICE and autologous stem cell transplant (P = .012) (Fig. 2). More participants (50 %), compared with nonparticipants (33 %), selected standard chemotherapy for NHL, such as cyclophosphamide, hydroxydaunorubicin, oncovin, prednisone (CHOP), as second-line therapy for case 4.
Fig. 2

Survey responses regarding treatment of the case presentation: patient with relapsed lymphoma

Improving Clinical Practice and Optimizing Lymphoma Patient Care

When case 1 had a positive PET/CT after two cycles of ABVD, 33 % of participants and 17 % of nonparticipants would change the treatment regimen to bleomycin, etoposide, Adriamycin, cyclophosphamide, Oncovin, procarbazine, and prednisone (BEACOPP). For case 2, similar percentages of participants (67 %) and nonparticipants (70 %) would use R-CHOP for six to eight cycles as initial therapy.


Similar percentages of participants (63 %) and nonparticipants (67 %) were “very confident” in their ability to manage patients with lymphoma, and 37 % of participants and 27 % of nonparticipants were “somewhat confident.” However, no participants perceived themselves as “not confident,” compared with 7 % of nonparticipants. With respect to confidence in ability to interpret histological and cytogenetic testing for selecting optimal treatment, 57 % of participants and 20 % of nonparticipants were “very confident,” 37 % of participants and 67 % of nonparticipants were “somewhat confident,” and 7 % of participants and 13 % of nonparticipants were “not confident” (P = .001).


More nonparticipants (53 %), compared with participants (37 %), considered incorporating biomarkers into management regimens as a “very significant” barrier to the optimal management of patients with lymphoma. Establishing criteria for maintenance therapy was seen as a “very significant” barrier by 43 % of both groups. Compared with nonparticipants, a higher percentage of participants viewed the following as “very significant” barriers to the optimal management of patients with lymphoma: optimal combination of multiple agents (37 % participants, 27 % nonparticipants), managing side effects of therapy (37 vs 30 %), cost effectiveness in selecting a therapy (30 vs 27 %), lack of algorithms for best treatment order (27 vs 23 %), and deciding when to initiate therapy (27 vs 23 %). However, the potential for combination chemotherapy to replace autologous stem cell transplant was seen as a “very significant” barrier by 23 % of participants and 40 % of nonparticipants.


The diagnosis of lymphoma is made by histologic, immunophenotypic, and cytogenetic evaluations. Establishing an accurate diagnosis is crucial, as there are major differences in the management of patients with lymphoma subtypes. Management of patients with lymphomas has dramatically improved in recent decades; however, research using novel agents and combinations of agents is ongoing. It is crucial for health-care providers who manage patients with lymphomas to incorporate the most recent information about diagnostic testing and management strategies into clinical practice.

Evidence suggests that multi-event, interactive educational initiatives result in durable improvements in cancer diagnosis and treatment, while single-event educational initiatives, such as didactic lectures or dissemination of printed guidelines, have little effect on physician behavior and performance [59]. The program Chicago Lymphoma Rounds, evaluated in this study, had several characteristics associated with successful CME programs [6]. The activity provided a forum for clinicians involved in caring for patients with lymphoma to meet regularly to share best practices and network with peers. Multiple educational exposures, such as those in this activity, have been associated with achieving desired objectives of CME programs [6]. The rounds also utilized local lymphoma experts to present the latest information on new therapies and advances in the management of lymphoma through both didactic lectures and the interactive case studies. Evidence suggests that educational programs delivered by influential physicians may be effective in facilitating changes in physician behavior [5]. A study on CME to improve colon cancer staging found that participant discussions with opinion leaders as part of a multimodal educational approach resulted in a significant improvement in lymph node assessment and staging even at 30 months post-intervention [7].

In some regards, the case vignette structure of this educational initiative was similar to that of tumor boards, where a multidisciplinary team reviews and decides upon the management approach for individual patients. Although their effectiveness has yet to be thoroughly evaluated, tumor boards have the potential to play a significant role in altering health-care provider behavior in planning patient care [7, 10]. Moreover, multidisciplinary tumor boards provide a significant educational opportunity for participants [7, 11]. One study found that over one third (34.6 %) of cases presented at a gynecologic oncology tumor board had alterations in disposition. Major and minor alterations primarily involved pathology reassignment [11]. In addition, over 25 % of the cases generated educationally centered discourse [11].

Case vignettes, such as those employed in the Chicago Lymphoma Rounds, not only are useful as a problem-solving approach to learning, but have also been shown to be an accurate method of measuring the quality of clinical practice when compared with both chart abstraction and standardized patient assessment [2, 3]. Case vignettes have the advantages of being inexpensive and easy to use, and they have been validated as assessment tools across diverse clinical settings, health-care provider populations, and diseases [3]. This case-based survey administered 30 days after completion of this CME activity demonstrated that the CME program was successful in many aspects and identified areas where further educational interventions are warranted. While significantly more participants than nonparticipants correctly diagnosed the patient in case 2, one quarter of participants still failed to select a correct diagnosis. The survey also found that participants were significantly more likely to know the percentage of CD20 expression in classical HL than nonparticipants; however, 40 % of participants still failed to identify the correct percentage. The topic of keratin expression in aggressive post-germinal center lymphomas remains an area of confusion and suggests that educational interventions of greater depth are needed. Although participants were better able than nonparticipants to identify appropriate strategies for monitoring lymphoma treatment, the role of interim radiologic studies (PET/CT) also remains an area where further education is needed.

A key survey finding was that participants were more likely than nonparticipants to correctly manage patients with progressive disease. Notably, participants were more likely to choose appropriate treatment in patients with progressive PMBCL and were more likely to choose re-biopsy in classical HL with an atypical phenotype nonresponsive to standard therapy.

Compared with nonparticipants, participants reported an increase in their confidence to manage patients with lymphoma, including their ability to interpret histological and cytogenetic testing for selecting optimal treatment. Participants and nonparticipants held somewhat different views on barriers to the management of patients with lymphoma. However, substantial percentages of both groups cited issues such as combination and maintenance therapy, biomarker use, and side-effect management as very significant barriers in managing patients with lymphoma. These topics should be incorporated into future CME programs.

This study has some limitations. It relied on evaluation of a CME intervention using a self-reported case vignette survey as a surrogate measure of respondents’ skills, knowledge, and attitudes. Although the use of case vignettes has been shown to provide valid and reliable data on clinicians’ actual practice patterns, vignettes do not capture all elements of quality care, and the results should not be viewed as the only measure of clinical competency [3], nor do the survey results reflect or predict actual improvements in patient outcomes. The survey was also administered to participants 1 month after completion of the CME activity, so the durability of improvements seen among participants over a longer time period cannot be predicted. Respondents were given a small honorarium to complete the study, which could influence participation rates and responses. Additionally, research has found that clinicians who respond to Internet-based surveys are frequently not representative of the general population of health-care professionals [12]; however, demographic characteristics of our sample were not different, in comparison to the American Medical Association 2010 data, from that of the population of physicians. The cross-sectional design of the study does not allow for causal inferences to be drawn.

Copyright information

© Springer Science+Business Media New York 2013