Abstract
Assessing programmed death ligand 1 (PD-L1) expression on tumor cells (TCs) using Food and Drug Administration-approved, validated immunoassays can guide the use of immune checkpoint inhibitor (ICI) therapy in cancer treatment. However, substantial interobserver variability has been reported using these immunoassays. Artificial intelligence (AI) has the potential to accurately measure biomarker expression in tissue samples, but its reliability and comparability to standard manual scoring remain to be evaluated. This multinational study sought to compare the %TC scoring of PD-L1 expression in advanced urothelial carcinoma, assessed by either an AI Measurement Model (AIM-PD-L1) or expert pathologists. The concordance among pathologists and between pathologists and AIM-PD-L1 was determined. The positivity rate of ≥ 1%TC PD-L1 was between 20–30% for 8/10 pathologists, and the degree of agreement and scoring distribution for among pathologists and between pathologists and AIM-PD-L1 was similar both scored as a continuous variable or using the pre-defined cutoff. Numerically higher score variation was observed with the 22C3 assay than with the 28–8 assay. A 2-h training module on the 28–8 assay did not significantly impact manual assessment. Cases exhibiting significantly higher variability in the assessment of PD-L1 expression (mean absolute deviation > 10) were found to have patterns of PD-L1 staining that were more challenging to interpret. An improved understanding of sources of manual scoring variability can be applied to PD-L1 expression analysis in the clinical setting. In the future, the application of AI algorithms could serve as a valuable reference guide for pathologists while scoring PD-L1.
Similar content being viewed by others
References
Agilent Technologies (2021) PD-L1 IHC 22C3 pharmDx instructions for use. https://www.agilent.com/cs/library/packageinsert/public/P03951E_20.pdf . Accessed 27 Mar 2024
Agilent Technologies (2021) PD-L1 IHC 28–8 pharmDx instructions for use. https://www.agilent.com/cs/library/packageinsert/public/PD04163_rev_02_SK00521-5_IFU.pdf . Accessed 27 Mar 2024
Ahn S, Kim K-M (2021) PD-L1 expression in gastric cancer: interchangeability of 22C3 and 28–8 pharmDx assays for responses to immunotherapy. Mod Pathol 34:1719–1727. https://doi.org/10.1038/s41379-021-00823-9
Bajorin DF, Witjes JA, Gschwend J, Schenker M, Valderrama BP, Tomita Y, Bamias A, Lebret T, Shariat S, Park SH, Ye D, Agerbaek M, Collette S, Unsal-Kacmaz K, Zardavas D, Koon HB, Galsky MD (2021) First results from the phase 3 CheckMate 274 trial of adjuvant nivolumab vs placebo in patients who underwent radical surgery for high-risk muscle-invasive urothelial carcinoma (MIUC). J Clin Oncol 39:Abstract 391. https://doi.org/10.1200/JCO.2021.39.6_suppl.391
Bajorin DF, Witjes JA, Gschwend JE, Schenker M, Valderrama BP, Tomita Y, Bamias A, Lebret T, Shariat SF, Park SH, Ye D, Agerbaek M, Enting D, McDermott R, Gajate P, Peer A, Milowsky MI, Nosov A, Neif Antonio J, Tupikowski K, Toms L, Fischer BS, Qureshi A, Collette S, Unsal-Kacmaz K, Broughton E, Zardavas D, Koon HB, Galsky MD (2021) Adjuvant nivolumab versus placebo in muscle-invasive urothelial carcinoma. N Engl J Med 384:2102–2114. https://doi.org/10.1056/NEJMoa2034442
Balar AV, Galsky MD, Rosenberg JE, Powles T, Petrylak DP, Bellmunt J, Loriot Y, Necchi A, Hoffman-Censits J, Perez-Gracia JL, Dawson NA, van der Heijden MS, Dreicer R, Srinivas S, Retz MM, Joseph RW, Drakaki A, Vaishampayan UN, Sridhar SS, Quinn DI, Durán I, Shaffer DR, Eigl BJ, Grivas PD, Yu EY, Li S, Kadel EE, Boyd Z, Bourgon R, Hegde PS, Mariathasan S, Thåström A, Abidoye OO, Fine GD, Bajorin DF (2017) Atezolizumab as first-line treatment in cisplatin-ineligible patients with locally advanced and metastatic urothelial carcinoma: a single-arm, multicentre, phase 2 trial. Lancet 389:67–76. https://doi.org/10.1016/s0140-6736(16)32455-2
Baxi V, Lee G, Duan C, Pandya D, Cohen DN, Edwards R, Chang H, Li J, Elliott H, Pokkalla H, Glass B, Agrawal N, Lahiri A, Wang D, Khosla A, Wapinski I, Beck A, Montalto M (2022) Association of artificial intelligence-powered and manual quantification of programmed death-ligand 1 (PD-L1) expression with outcomes in patients treated with nivolumab & #xb1; ipilimumab. Mod Pathol 35:1529–1539. https://doi.org/10.1038/s41379-022-01119-2
Bera K, Schalper KA, Rimm DL, Velcheti V, Madabhushi A (2019) Artificial intelligence in digital pathology—new tools for diagnosis and precision oncology. Nat Rev Clin Oncol 16:703–715
Bristol Myers Squibb (2022) OPDIVO® (nivolumab) [package insert]. https://packageinserts.bms.com/pi/pi_opdivo.pdf. Accessed 27 Mar 2024
Bristol Myers Squibb (2022) Yervoy®(ipilimumab) [package insert]. https://packageinserts.bms.com/pi/pi_yervoy.pdf. Accessed 27 Mar 2024
Büttner R, Gosney JR, Skov BG, Adam J, Motoi N, Bloom KJ, Dietel M, Longshore JW, López-Ríos F, Penault-Llorca F, Viale G, Wotherspoon AC, Kerr KM, Tsao M-S (2017) Programmed death-ligand 1 immunohistochemistry testing: A review of analytical assays and clinical implementation in non-small-cell lung cancer. J Clin Oncol 35:3867–3876. https://doi.org/10.1200/jco.2017.74.7642
CellCarta (2022) Pathotrainer: enabling pathology training and tuning in biomarker development. https://www.pathomation.com/pathotrainer-3/. Accessed 27 Mar 2024
Chen L-T, Satoh T, Ryu M-H, Chao Y, Kato K, Chung HC, Chen J-S, Muro K, Kang WK, Yeh K-H, Yoshikawa T, Oh SC, Bai L-Y, Tamura T, Lee K-W, Hamamoto Y, Kim JG, Chin K, Oh D-Y, Minashi K, Cho JY, Tsuda M, Sameshima H, Kang Y-K, Boku N (2020) A phase 3 study of nivolumab in previously treated advanced gastric or gastroesophageal junction cancer (ATTRACTION-2): 2-year update data. Gastric Cancer 23:510–519. https://doi.org/10.1007/s10120-019-01034-7
Cserni B, Bori R, Csörgő E, Oláh-Németh O, Pancsa T, Sejben A, Sejben I, Vörös A, Zombori T, Nyári T, Cserni G (2022) ONEST (Observers Needed to Evaluate Subjective Tests) suggests four or more observers for a reliable assessment of the consistency of histological grading of invasive breast carcinoma: A reproducibility study with a retrospective view on previous studies. Pathol Res Pract 229:153718. https://doi.org/10.1016/j.prp.2021.153718
Downes MR, Slodkowska E, Katabi N, Jungbluth AA, Xu B (2020) Inter- and intraobserver agreement of programmed death ligand 1 scoring in head and neck squamous cell carcinoma, urothelial carcinoma and breast carcinoma. Histopathology 76:191–200. https://doi.org/10.1111/his.13946
Eckstein M, Cimadamore A, Hartmann A, Lopez-Beltran A, Cheng L, Scarpelli M, Montironi R, Gevaert T (2019) PD-L1 assessment in urothelial carcinoma: a practical approach. Ann Transl Med 7:690. https://doi.org/10.21037/atm.2019.10.24
Eckstein M, Erben P, Kriegmair MC, Worst TS, Weiß C-A, Wirtz RM, Wach S, Stoehr R, Sikic D, Geppert CI, Weyerer V, Bertz S, Breyer J, Otto W, Keck B, Burger M, Taubert H, Weichert W, Wullich B, Bolenz C, Hartmann A, Erlmeier F (2019) Performance of the Food and Drug Administration/EMA-approved programmed cell death ligand-1 assays in urothelial carcinoma with emphasis on therapy stratification for first-line use of atezolizumab and pembrolizumab. Eur J Cancer 106:234–243. https://doi.org/10.1016/j.ejca.2018.11.007
Eckstein M, Wirtz RM, Pfannstil C, Wach S, Stoehr R, Breyer J, Erlmeier F, Gunes C, Nitschke K, Weichert W, Otto W, Keck B, Eidt S, Burger M, Taubert H, Wullich B, Bolenz C, Hartmann A, Erben P (2018) A multicenter round robin test of PD-L1 expression assessment in urothelial bladder cancer by immunohistochemistry and RT-qPCR with emphasis on prognosis prediction after radical cystectomy. Oncotarget 9:15001–15014. https://doi.org/10.18632/oncotarget.24531
Franklin C, Kader G, Mewborn D, Moreno J, Peck R, Perry M, Scheaffer R (2007) Guidelines for Assessment and Instruction in Statistics Education. https://web.archive.org/web/20130307004604/http:/www.amstat.org/education/gaise/GAISEPreK-12_Full.pdf. Accessed 27 Mar 2024
Fuchs CS, Doi T, Jang RW, Muro K, Satoh T, Machado M, Sun W, Jalal SI, Shah MA, Metges JP, Garrido M, Golan T, Mandala M, Wainberg ZA, Catenacci DV, Ohtsu A, Shitara K, Geva R, Bleeker J, Ko AH, Ku G, Philip P, Enzinger PC, Bang YJ, Levitan D, Wang J, Rosales M, Dalal RP, Yoon HH (2018) Safety and efficacy of pembrolizumab monotherapy in patients with previously treated advanced gastric and gastroesophageal junction cancer: phase 2 clinical keynote-059 trial. JAMA Oncol 4:e180013. https://doi.org/10.1001/jamaoncol.2018.0013
Genentech Inc (2022) TECENTRIQ® (atezolizumab) [package insert]. https://www.gene.com/download/pdf/tecentriq_prescribing.pdf. Accessed 27 Mar 2024
Herbst RS, Baas P, Kim DW, Felip E, Perez-Gracia JL, Han JY, Molina J, Kim JH, Arvis CD, Ahn MJ, Majem M, Fidler MJ, de Castro G Jr, Garrido M, Lubiniecki GM, Shentu Y, Im E, Dolled-Filhart M, Garon EB (2016) Pembrolizumab versus docetaxel for previously treated, PD-L1-positive, advanced non-small-cell lung cancer (KEYNOTE-010): a randomised controlled trial. Lancet 387:1540–1550. https://doi.org/10.1016/S0140-6736(15)01281-7
Hirsch FR, McElhinny A, Stanforth D, Ranger-Moore J, Jansson M, Kulangara K, Richardson W, Towne P, Hanks D, Vennapusa B, Mistry A, Kalamegham R, Averbuch S, Novotny J, Rubin E, Emancipator K, McCaffery I, Williams JA, Walker J, Longshore J, Tsao MS, Kerr KM (2017) PD-L1 immunohistochemistry assays for lung cancer: results from phase 1 of the Blueprint PD-L1 IHC assay comparison project. J Thorac Oncol 12:208–222. https://doi.org/10.1016/j.jtho.2016.11.2228
Kim B, Lee C, Kim YA, Moon KC (2020) PD-L1 expression in muscle-invasive urinary bladder urothelial carcinoma according to basal/squamous-like phenotype. Front Oncol 10:527385. https://doi.org/10.3389/fonc.2020.527385
Larkin J, Chiarion-Sileni V, Gonzalez R, Grob JJ, Rutkowski P, Lao CD, Cowey CL, Schadendorf D, Wagstaff J, Dummer R, Ferrucci PF, Smylie M, Hogg D, Hill A, Marquez-Rodas I, Haanen J, Guidoboni M, Maio M, Schoffski P, Carlino MS, Lebbe C, McArthur G, Ascierto PA, Daniels GA, Long GV, Bastholt L, Rizzo JI, Balogh A, Moshyk A, Hodi FS, Wolchok JD (2019) Five-year survival with combined nivolumab and ipilimumab in advanced melanoma. N Engl J Med 381:1535–1546. https://doi.org/10.1056/NEJMoa1910836
Lawson NL, Dix CI, Scorer PW, Stubbs CJ, Wong E, Hutchinson L, McCall EJ, Schimpl M, DeVries E, Walker J, Williams GH, Hunt J, Barker C (2020) Mapping the binding sites of antibodies utilized in programmed cell death ligand-1 predictive immunohistochemical assays for use with immuno-oncology therapies. Mod Pathol 33:518–530. https://doi.org/10.1038/s41379-019-0372-z
Lopez-Beltran A, Cimadamore A, Blanca A, Massari F, Vau N, Scarpelli M, Cheng L, Montironi R (2021) Immune checkpoint inhibitors for the treatment of bladder cancer. Cancers (Basel) 13:131. https://doi.org/10.3390/cancers13010131
Mansour MSI, Hejny K, Johansson F, Mufti J, Vidis A, Mager U, Dejmek A, Seidal T, Brunnström H (2021) Factors influencing concordance of PD-L1 expression between biopsies and cytological specimens in non-small cell lung cancer. Diagnostics (Basel) 11:1927. https://doi.org/10.3390/diagnostics11101927
Mok TSK, Wu Y-L, Kudaba I, Kowalski DM, Chul Cho B, Turna HZ, Castro G Jr, Srimuninnimit V, Laktionov KK, Bondarenko I, Kubota K, Lubiniecki GM, Zhang J, Kush D, Lopes G, Investigators K- (2019) Pembrolizumab versus chemotherapy for previously untreated, PD-L1-expressing, locally advanced or metastatic non-small-cell lung cancer (KEYNOTE-042): a randomised, open-label, controlled, phase 3 trial. Lancet 393:1819–1830. https://doi.org/10.1016/s0140-6736(18)32409-7
National Comprehensive Cancer Network (2022) NCCN Clinical practice guidelines in oncology. Bladder cancer. https://www.nccn.org/professionals/physician_gls/pdf/bladder.pdf . Accessed 27 Mar 2024
Prince EA, Sanzari JK, Pandya D, Huron D, Edwards R (2021) Analytical concordance of PD-L1 assays utilizing antibodies from FDA-approved diagnostics in advanced cancers: A systematic literature review. JCO Precis Oncol 5:953–973. https://doi.org/10.1200/PO.20.00412
Rehman JA, Han G, Carvajal-Hausdorf DE, Wasserman BE, Pelekanou V, Mani NL, McLaughlin J, Schalper KA, Rimm DL (2017) Quantitative and pathologist-read comparison of the heterogeneity of programmed death-ligand 1 (PD-L1) expression in non-small cell lung cancer. Mod Pathol 30:340–349. https://doi.org/10.1038/modpathol.2016.186
Reisenbichler ES, Han G, Bellizzi A, Bossuyt V, Brock J, Cole K, Fadare O, Hameed O, Hanley K, Harrison BT, Kuba MG, Ly A, Miller D, Podoll M, Roden AC, Singh K, Sanders MA, Wei S, Wen H, Pelekanou V, Yaghoobi V, Ahmed F, Pusztai L, Rimm DL (2020) Prospective multi-institutional evaluation of pathologist assessment of PD-L1 assays for patient selection in triple negative breast cancer. Mod Pathol 33:1746–1752. https://doi.org/10.1038/s41379-020-0544-x
Rimm DL, Han G, Taube JM, Yi ES, Bridge JA, Flieder DB, Homer R, West WW, Wu H, Roden AC, Fujimoto J, Yu H, Anders R, Kowalewski A, Rivard C, Rehman J, Batenchuk C, Burns V, Hirsch FR, Wistuba II (2017) A prospective, multi-institutional, pathologist-based assessment of 4 immunohistochemistry assays for PD-L1 expression in non-small cell lung cancer. JAMA Oncol 3:1051–1058. https://doi.org/10.1001/jamaoncol.2017.0013
Robert ME, Rüschoff J, Jasani B, Graham RP, Badve SS, Rodriguez-Justo M, Kodach LL, Srivastava A, Wang HL, Tang LH, Troncone G, Rojo F, Van Treeck BJ, Pratt J, Shnitsa I, Kumar G, Karasarides M, Anders RA (2023) High interobserver variability among pathologists using combined positive score to evaluate PD-L1 expression in gastric, gastroesophageal junction, and esophageal adenocarcinoma. Mod Pathol 36:100154. https://doi.org/10.1016/j.modpat.2023.100154
Scheel AH, Dietel M, Heukamp LC, Jöhrens K, Kirchner T, Reu S, Rüschoff J, Schildhaus H-U, Schirmacher P, Tiemann M, Warth A, Weichert W, Fischer RN, Wolf J, Buettner R (2016) Harmonized PD-L1 immunohistochemistry for pulmonary squamous-cell and adenocarcinomas. Mod Pathol 29:1165–1172. https://doi.org/10.1038/modpathol.2016.117
Schmid P, Adams S, Rugo HS, Schneeweiss A, Barrios CH, Iwata H, Dieras V, Hegg R, Im S-A, Shaw Wright G, Henschel V, Molinero L, Chui SY, Funke R, Husain A, Winer EP, Loi S, Emens LA (2018) Atezolizumab and nab-paclitaxel in advanced triple-negative breast cancer. N Engl J Med 379:2108–2121. https://doi.org/10.1056/NEJMoa1809615
Schwamborn K, Ammann J, Knüchel R, Hartmann A, Baretton G, Lasitschka F, Schirmacher P, Braunschweig T, Tauber R, Erlmeier F, Hieke-Schulz S, Weichert W (2019) Multicentric analytical comparability study of programmed death-ligand 1 expression on tumor-infiltrating immune cells and tumor cells in urothelial bladder cancer using four clinically developed immunohistochemistry assays. Virchows Arch 475:599–608. https://doi.org/10.1007/s00428-019-02610-z
Tsao MS, Kerr KM, Kockx M, Beasley MB, Borczuk AC, Botling J, Bubendorf L, Chirieac L, Chen G, Chou TY, Chung JH, Dacic S, Lantuejoul S, Mino-Kenudson M, Moreira AL, Nicholson AG, Noguchi M, Pelosi G, Poleri C, Russell PA, Sauter J, Thunnissen E, Wistuba I, Yu H, Wynes MW, Pintilie M, Yatabe Y, Hirsch FR (2018) PD-L1 immunohistochemistry comparability study in real-life clinical samples: results of Blueprint phase 2 project. J Thorac Oncol 13:1302–1311. https://doi.org/10.1016/j.jtho.2018.05.013
Ventana Medical Systems Inc (2020) VENTANA PD-L1 (SP263) assay [package insert]. https://diagnostics.roche.com/ch/de/products/tests/ventana-pd-l1-_sp263-assay2.html. Accessed 27 Mar 2024
Acknowledgements
The authors would like to thank Jennifer Schroeder and the Pharmalex team for their help with the statistical analysis; CellCarta, especially Elizabeth Ross, Senne Carpentier, and Pablo Reclusa, for managing sample staining and digitization; and Discovery Life Sciences, especially Gudrun Baenfer and Rolf Diezko, for managing sample selection and provisioning. Writing and editorial assistance were provided by Emily Motola, PharmD, Allyson Koyen Malashevich, PhD, and Agata Shodeke, PhD, of Spark (a division of Prime, New York, USA), funded by Bristol Myers Squibb, according to Good Publication Practice guidelines.
Funding
This study was sponsored by Bristol Myers Squibb.
Author information
Authors and Affiliations
Contributions
Authors contributions are as follows: study conception/design: Drs Kumar, Badve, Babadjanova, Karasarides and Hartmann; data acquisition: Drs Rüschoff, Kumar, Badve, Jasani, Rioux-Leclercq, Rojo, Martini, Cheng, Tretiakova, Mitchell, Anders, Robert, Brutus, Karasarides, and Hartmann, and Mr Glass; data analysis: Drs Kumar, Badve, Krause, Babadjanova, Pratt, and Karasarides, and Mr Fahy, Pyle, Le, and Baxi; data interpretation: Drs Rüschoff, Kumar, Badve, Jasani, Krause, Rioux-Leclercq, Rojo, Martini, Cheng, Tretiakova, Mitchell, Anders, Robert, Yu, Pratt, Brutus, Karasarides, and Hartmann, and Mr Baxi. All authors contributed to the writing and editing of this manuscript.
Corresponding authors
Ethics declarations
The study was conducted in compliance with BMS Bioethics Policy (https://www.bms.com/about-us/responsibility/position-on-key-issues/bioethics-policy-statement.html). The samples procured from Discovery Life Sciences were processed in accordance with all applicable EU and US regulations as specified on the company’s website (https://www.dls.com/resource-hub/faqs).
Conflict of Interest
Dr. Rüschoff has received support for travel and participation in advisory boards from Agilent, AstraZeneca, Bristol Myers Squibb, DSI, and Merck/MSD. Dr Badve has received grants or contracts from Agilent, Lilly, the National Institutes of Health (NIH), and the National Cancer Institute; consulting fees from Agendia, Dako-Agilent, Ventana, Roche, and Daiichi-Sankyo; and honoraria from Medscape, Targos-Discovery, and PER; and is the Director of the Indian Cancer Genome Atlas Foundation. Dr. Jasani has received consulting fees from Bristol Myers Squibb. Dr Rojo has received grants or contracts from AstraZeneca, Roche, and Pfizer; consulting fees and honoraria from Roche, AstraZeneca, Novartis, MSD, Bristol Myers Squibb, Lilly, Amgen, Astellas, Glaxo-Smith Kline, and Daiichi Sankyo; and support for meeting attendance and travel from MSD, Roche, and AstraZeneca. Dr Martini has received funding from the University of Washington and Bristol Myers Squibb. Dr Anders has received grants or contracts from BMS, RAPT Therapeutics, Stand Up2Cancer, the NIH, and Mark Foundation; consulting fees from Bristol Myers Squibb, Merck SD, and AstraZeneca; and support for attending meetings from Bristol Myers Squibb. Dr Robert has received consulting fees from Takeda Pharmaceuticals and Bristol Myers Squibb. Drs Babadjanova, Pratt, Karasarides, and Kumar are employees of, and hold stock in, Bristol Myers Squibb. Dr. Brutus is a former employee of PathAI and holds stock in PathAI. Dr Hartmann has received consulting fees from Bristol Myers Squibb, MSD, Roche, Cepheid, Qiagen, Janssen, AstraZeneca, Agilent, Eli Lilly, Phäon, Ipsen, Diaceutics, Illumina, Abbvie, and Nanostring; receives travel support/support for attending meetings for, and participates in, advisory boards for Bristol Myers Squibb, MSD, Roche, Cepheid, Qiagen, Janssen, AstraZeneca, Agilent, Lilly, Phäon, Ipsen, and Diaceutics; and has received other research support from AstraZeneca, Biocartis, Cepheid, Gilead, Illumina, Janssen, Nanostring, Novartis, Owkin, Qiagen, QUIP GmbH, Roche, and Sanofi. Drs Krause, Rioux-Leclercq, Cheng, Tretiakova, Mitchell and Yu, and Mr Fahy, Pyle, Le, Glass and Baxi have no competing interests to declare.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Josef Rüschoff and George Kumar co-lead authors.
George Kumar, Sergine Brutus and Maria Karasarides affiliation pertains to the time the study was conducted.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Rüschoff, J., Kumar, G., Badve, S. et al. Scoring PD-L1 Expression in Urothelial Carcinoma: An International Multi-Institutional Study on Comparison of Manual and Artificial Intelligence Measurement Model (AIM-PD-L1) Pathology Assessments. Virchows Arch (2024). https://doi.org/10.1007/s00428-024-03795-8
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s00428-024-03795-8