Minimal information about T cell assays: the process of reaching the community of T cell immunologists in cancer and beyond
- 1.3k Downloads
Many assays to evaluate the nature, breadth, and quality of antigen-specific T cell responses are currently applied in human medicine. In most cases, assay-related protocols are developed on an individual laboratory basis, resulting in a large number of different protocols being applied worldwide. Together with the inherent complexity of cellular assays, this leads to unnecessary limitations in the ability to compare results generated across institutions. Over the past few years a number of critical assay parameters have been identified which influence test performance irrespective of protocol, material, and reagents used. Describing these critical factors as an integral part of any published report will both facilitate the comparison of data generated across institutions and lead to improvements in the assays themselves. To this end, the Minimal Information About T Cell Assays (MIATA) project was initiated. The objective of MIATA is to achieve a broad consensus on which T cell assay parameters should be reported in scientific publications and to propose a mechanism for reporting these in a systematic manner. To add maximum value for the scientific community, a step-wise, open, and field-spanning approach has been taken to achieve technical precision, user-friendliness, adequate incorporation of concerns, and high acceptance among peers. Here, we describe the past, present, and future perspectives of the MIATA project. We suggest that the approach taken can be generically applied to projects in which a broad consensus has to be reached among scientists working in fragmented fields, such as immunology. An additional objective of this undertaking is to engage the broader scientific community to comment on MIATA and to become an active participant in the project.
KeywordsMinimal information about T cell assays Reporting guidelines Immune monitoring MIATA
For many years, the development of active immunotherapy to treat cancer was considered a pursuit with uncertain outcome that was met by increasing skepticism, in part resulting from failure to demonstrate clinical efficacy [1, 2]. Recent reports of success from randomized phase III trials [3, 4, 5] and the first approval for a therapeutic vaccine  now set the stage for a new era of immunotherapy with greater availability of biological samples from large controlled clinical trials on the horizon. A crucial role for T cell responses in active immunotherapy has been demonstrated in numerous animal models as well as in clinical trials [7, 8]. A large number of different assays for measuring T cell quality and function has been developed and applied in such clinical trials. The identification, development, and eventual validation of T cell biomarkers that correlate with product bioactivity and clinical response is a challenging and complex process, but is required for understanding and improving on response rates. This necessitates an integrated clinical development plan, a systematic and comprehensive standardized scheme for sample procurement/processing, and biomarker assays including data analysis to be applied. To this end, quality-enabling laboratory infrastructure needs to be established, which is essential for large and well-conducted trials, culminating in the enhanced potential to measure successful induction of clinical responses [9, 10]. Because the technical validation of assays for measuring cellular immune biomarkers is not trivial, systematic mono- and multi-center efforts have been initiated in the different fields of cancer immunology, autoimmunity and infectious diseases to harmonize and standardize T cell assays. These systematic efforts have led to the identification of critical factors that influence assay performance and the consequent formulation of guidelines to improve and harmonize T cell assays across laboratories [11, 12, 13, 14, 15, 16]. These harmonization efforts increased the awareness to the problem that, despite the vast body of literature describing results from T cell assays, the majority of scientific publications lack important information on crucial steps in the assay process, precluding a full understanding and accurate interpretation of published data sets and limiting the ability to reproduce results and compare data sets between centers or perform meta-analysis. Undoubtedly, having a more uniform and complete reporting structure would not only serve the broad interests of the scientific community, but would also be beneficial in enhancing the accurate communication of methodologies between immunotherapy programs.
In 2008, a core team of immunologists from Europe and the United States embarked on the establishment of a Minimal Information (MI) project for T cell assays and set the stage for an ongoing large-scale, field-spanning effort to generate a widely acceptable reporting framework to support scientific publications and enhance the utility of presented data. The project on Minimal Information About T Cell Assays (MIATA) was announced in October 2009 ; and a dedicated web site was launched in parallel with the listing of a first draft of guidelines, consisting of five modules based on the gained knowledge from harmonization efforts about factors that critically influence assay results, namely the requirement to provide information on the (1) sample, (2) the assay, (3) data acquisition, (4) the interpretation of data as well as (5) the lab environment . At the same time, a public consultation phase was initiated. Since then, based primarily on the input from the public consultation process and public workshops, the MIATA project has undergone a process of evolution and maturation, resulting in a streamlined set of guidelines for reporting on T cell assays. These guidelines can be found on the MIATA website (http://www.miataproject.org). Here, we elaborate on the ongoing collaborative approach taken to develop MIATA guidelines, which was found to be essential in order to achieve the desired high degree of quality and acceptability for T cell assays. The process also led to the identification of critical factors for success that might be applicable to similar projects in other areas of science.
Minimal information projects
The concept of Minimal Information projects was pioneered by Brazma et al. , for complex analytical assays that generate large data sets with several layers of information, to meet demands in the area of microarrays. Subsequently, the first successful MI project addressing microarray experiments was introduced . Meanwhile over 30 MI projects for different types of assays have been proposed, many of which are listed under the MIBBI portal, which provides assistance to investigators in identifying suitable MI guidelines for their work . Most MI projects are (1) primarily designed for high-throughput assays and (2) require detailed annotation of samples and related data sets to populate large databases and allow for data mining by third parties. The majority of current T cell assays do not fall under the category “high throughput”, and current data sharing refers mainly to comparing results from different groups working in similar areas, rather than exploring existing data sets generated by third parties. The introduction of new technologies in the immune assay field might change this situation [22, 23], and applicable MI projects including MIATA need to be adaptable and expandable for data mining, for example in an envisioned Human Immunity Project . It should be noted that a minimal information framework for publications might significantly differ from a framework for populating large databases including content and syntax .
What is MIATA?
MIATA is a framework for WHAT and HOW to report when publishing immune monitoring tests and results. The overall goal for MIATA is to provide the reader with the minimal information necessary to understand how reported data related to T cell assays were generated . This is accomplished by providing a structured basis for reporting the information that allows the full and objective evaluation of the presented data sets, as well as for the ability to integrate data sets across studies. MIATA focuses solely on the reporting content and structure for assays, not the assay content and structure.
What MIATA is not
MIATA is not an effort to impose specific standards on how T cell assays are performed, including assay standardization, validation, and setup of the appropriate laboratory environment. MIATA is without prejudice on how and when immune monitoring is performed. These issues need to be addressed by the individual investigators before a study is initiated. Consequently, conformity to the MIATA guidelines does not require the use of standardized assay protocols or a certain laboratory setup. Freedom of research and flexibility in the choice of assays and reagents used are of utmost importance and must not be limited at any time by any MI project.
Minimal information projects with overlap
Several MI projects with some degree of overlap with MIATA exist, for example Minimal Information About a Cellular Assay (MIACA) , Minimum Information for a Flow Cytometry Experiment (MIFlowCyt) , the Publication on Publishing Flow Cytometry Data  and Reporting recommendations for tumor Marker prognostic studies (REMARK) . Each of these projects is unique, and all are based on the expertise and input of highly experienced investigators. The REMARK criteria from the Statistics Subcommittee of the NCI-EORTC Working Group on Cancer Diagnostics have achieved a notable level of awareness in the community, supported by parallel publications in and endorsement by several international journals. Although REMARK has proven its value in several prognostic biomarker study publications , it does not specifically address aspects unique to T cell assays. Further, publications that claim conformity to other MI projects with overlap to MIATA are sparse at most. This might be partially due to the overwhelming amount of information required by other MI projects that were designed primarily to support data mining. These circumstances provide the foundations for the two-step approach introduced here with the initial focus on scientific publications, and later expandability for data mining. Such a process has the potential for wide acceptance and a successful adoption rate of MIATA, as it could for other MI projects.
Factors for success
Factors of success
Factors of success for MI projects
Technically precise (captures all essential information of the assay)
User-friendly (truly minimal and clearly understandable)
No existing concerns to prevent conforming
Accepted by peers
Added value to user/community
To begin, implementation of a new process (e.g., structured framework to report data from analytical assays such as MIATA) in a lab environment is time-consuming. It requires investigators to read, understand and approve the new framework internally, to teach staff members to apply the new process correctly, and to ensure proper execution. Consequently, any new reporting framework proposed to the field has to find a balance between being easy to understand and implement and requesting an amount of information sufficient to fulfill its purpose. Clearly, MI projects should be as “minimal” as possible. However, while minimizing the content of a reporting framework beyond a certain degree might make it easier to implement, it can also decrease its value significantly. Achieving the appropriate balance between not asking for too much or too little information has been and still is one of the most difficult tasks in the evolution of the MIATA project and is the focus of many discussions among the investigators involved. Clearly, the additional workload associated with implementation of MIATA should never outweigh its added value for individual investigators and the scientific community.
Even a user-friendly and technically precise reporting framework will not be of any value unless broadly accepted and adopted by scientists and journals. An investigator’s decision to accept or reject a proposed reporting framework will critically depend on the absence or existence of any concerns related to conforming to the guidelines. Various concerns were also raised after the first draft of MIATA (version 0) was published. It became clear that specific concerns were typically shared by groups of colleagues with a similar focus of their work (e.g., assay validation, research labs, labs performing correlative studies in larger clinical trials, applicants for funding schemes). Part of the MIATA approach has been to carefully consider and address each of these concerns en route to establishing a final set of reporting guidelines.
Finally, the adoption of MIATA will crucially depend on knowledge of the framework and its acceptance as standard practice by peers and stakeholders in the field. Despite the overwhelming redundancy in the nature of the assays used to evaluate immune responses across specialized fields such as cancer immunology, autoimmunity, transplantation, and infectious diseases, scientists in each of those fields have historically tended to operate as more or less isolated groups, developing closely related assays in a parallel, often redundant, fragmented, and non-integrated manner. Geographic fragmentation, facilitated by the establishment of specialty societies in the USA, Europe, parts of Asia, and Australia has further added to the non-integrated nature of T cell assay development. This fragmentation clearly needs to be addressed and overcome by integrating the expertise from as many directly involved peers as possible into the framework, independent of location, affiliation or background.
The past, presence, and future of MIATA
Many MI projects are authored by a significant number of scientists, and the process of reaching a consensus through discussions and seminars is a hallmark of most, if not all of them. On closer examination, the consensus-building process commonly appears to be confined to a group of experts, and it is not always clear at what level peers were involved.
The vision of the MIATA core team was to reach out to as many colleagues as possible in different areas of immunology and in laboratories involved in research and clinical trial monitoring. We believe that contributors to the process should include the scientists performing the assay, as well as principal investigators, regulatory authorities, and journal editors. Since its initial phase, the MIATA project has been working towards integrating suggestions and criticism from experts representing different areas of immunology, and from societies across geographic boundaries; accordingly, primary authors on this manuscript are active participants and members of a wide range of immunology associations. MIATA never was or shall be a closed community excluding individual colleagues or groups. On the contrary, the core team is pursuing a bottom-up approach and is reaching out to an increasing number of colleagues to integrate expertise into the framework from as many directly involved peers as possible and thereby make them an integral and important part of the MIATA project.
An independent project web site (http://www.miataproject.org)
An expanding network of supporting organizations
A public consultation period on the initial guideline draft (Version 0)
A public workshop and follow-up webinars
A public consultation period on the updated guidelines (Version 1)
A concluding workshop to consolidate the publication framework.
The project-related web site was found to be an invaluable tool for the transparent recording of MIATA’s progress. It proved to be an efficient tool for the ongoing consultation period during which each comment and responses to those comments were posted. The web site itself is run independently of any organization or other web site.
Public consultation period, workshop, and webinars
comments on the MIATA modules focusing on the specific content of each module and sub-module; and
comments on the overall focus and intent of MIATA, including logistics for its feasibility and applicability.
All comments were reviewed and considered in light of the module, sub-module, and general topic addressed, and the overall messages distilled and summarized in a discussion document for the first workshop held during the annual CIC/CRI meeting in Washington DC in March 2010. For this public workshop, expert panelists from different immunology fields (cancer, infectious diseases, autoimmunity), regulatory agencies, and journal editors were invited to discuss the initial guideline draft and comments received, and to update MIATA to version 1.
Scope of MIATA (framework for scientific publications vs. annotations for data base efforts)
Focused application for the MIATA publication framework (human immune monitoring versus T cell immune assays in general)
Striking a balance between minimal and sufficient information
The importance of freedom of research.
In effect, the focus of MIATA was re-defined and specified as a reporting framework for scientific publications of human immune monitoring results. After its successful implementation for the described purpose, MIATA should be adaptable in a step-wise manner to support annotations of immune monitoring data sets, for example in the context of a Human Immunity Project . It needs to be stressed that the consensus reached on these topics was found to be of the utmost importance for the increasing acceptance and continuation of MIATA.
Subsequently, the specific content of the guidelines was discussed by reviewing each single sub-module and related comments received. Due to the volume of information and impressive amount of feedback obtained, this process was extended over the remaining workshop and three separate webinars. A consensus was reached on the basis of professional discussions among panelists, and in consequence the MIATA guidelines were upgraded to Version 1 , parts of which differ considerably from the original draft.
Next steps and future perspectives
As for the initial guidelines, the updated MIATA modules have been posted on the project’s web site, and the field is once again invited to review and comment on its content. The first feedback already received indicates increasing agreement with the current framework. Most of the proposed changes and additions focus on single sub-modules.
In addition to soliciting further comments on the MIATA Version 1 guidelines, the current focus of the core team is to increase general awareness of the project in the field. In parallel, efforts are underway to promote the adoption of MIATA by scientists and journals. The MIATA experience has demonstrated that surprisingly little knowledge exists among immunologists about Minimal Information concepts and existing projects. Several recent articles addressing publication guidelines for optimized multicolor immunofluorescence panels might aid in raising further awareness . Keeping the high value of these projects in mind, it is important to disseminate information about them and at the same time initiate collaborations among all stakeholders and MI projects that overlap in any way with MIATA. This will help to avoid duplication of effort or even competition, while offering guidance to the community in their choice of the most suitable assays and implementation of the appropriate frameworks in their scientific work.
The current MIATA Version 1 is the result of a highly collaborative effort. It is essential for the success of this project to promote the further input of the community and to enhance the discussion and final consensus between scientists and scientific journals.
Funding for open access publication of this manuscript was provided by the Cancer Immunotherapy Consortium of the Cancer Research Institute (CIC/CRI) and the Cancer Immunotherapy Immunoguiding Program (CIP/CIMT), both being nonprofit organizations. They also offered generous support and patronage for the first MIATA workshop in Washington in March 2010. C.M.B and S.H.v.d.B were supported by the Wallace Coulter Foundation (Florida, USA) which also supported the organization of the MIATA workshop. The authors are indebted to all colleagues that contributed to the generation of the current version of MIATA at any stage of the project.
This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
- 2.Finke LH, Wentworth K, Blumenstein B, Rudolph NS, Levitsky H, Hoos A (2007) Lessons from randomized phase III studies with active cancer immunotherapies—outcomes from the 2006 meeting of the Cancer Vaccine Consortium (CVC). Vaccine 25(Suppl 2):B97–B109. doi: 10.1016/j.vaccine.2007.06.067 CrossRefPubMedGoogle Scholar
- 3.Schuster SJ, Neelapu SS, Gause BL, Muggia FM, Gockerman JP, Sotomayor EM, Winter JN, Flowers CR, Stergiou AM, Kwak LW, For the BiovaxID Phase III Study Investigators (2009) Idiotype vaccine therapy (biovaxid) in follicular lymphoma in first complete remission: phase III clinical trial results. J Clin Oncol 27:18s (Suppl; Abstr 2)Google Scholar
- 4.Schwartzentruber DJ, Lawson D, Richards J, Conry RM, Miller D, Triesman J, Gailani F, Riley LB, Vena D, Hwu P (2009) A phase III multi-institutional randomized study of immunization with the gp100: 209–217 (210 m) peptide followed by high-dose IL-2 compared with high-dose IL-2 alone in patients with metastatic melanoma. J Clin Oncol 27:18sCrossRefGoogle Scholar
- 5.Hodi FS, O’Day SJ, McDermott DF, Weber RW, Sosman JA, Haanen JB et al (2010) Improved survival with ipilimumab in patients with metastatic melanoma. N Engl J Med. doi: 10.1056/NEJMoa1003466
- 6.Kantoff PW, Higano CS, Shore ND, Berger ER, Small EJ, Penson DF, Redfern CH, Ferrari AC, Dreicer R, Sims RB, Xu Y, Frohlich MW, Schellhammer PF (2010) Sipuleucel-t immunotherapy for castration-resistant prostate cancer. N Engl J Med 363(5):411–422. doi: 10.1056/NEJMoa1001294 CrossRefPubMedGoogle Scholar
- 7.Kirkwood JM, Lee S, Moschos SJ, Albertini MR, Michalak JC, Sander C, Whiteside T, Butterfield LH, Weiner L (2009) Immunogenicity and antitumor effects of vaccination with peptide vaccine +/− granulocyte-monocyte colony-stimulating factor and/or IFN-alpha2b in advanced metastatic melanoma: Eastern Cooperative Oncology Group Phase II Trial E1696. Clin Cancer Res 15(4):1443–1451. doi: 10.1158/1078-0432.CCR-08-1231 CrossRefPubMedGoogle Scholar
- 11.Schloot NC, Meierhoff G, Karlsson Faresjo M, Ott P, Putnam A, Lehmann P, Gottlieb P, Roep BO, Peakman M, Tree T (2003) Comparison of cytokine elispot assay formats for the detection of islet antigen autoreactive t cells. Report of the third immunology of diabetes society t-cell workshop. J Autoimmun 21(4):365–376Google Scholar
- 12.Janetzki S, Panageas KS, Ben-Porat L, Boyer J, Britten CM, Clay TM, Kalos M, Maecker HT, Romero P, Yuan J, Kast WM, Hoos A (2008) Results and harmonization guidelines from two large-scale international Elispot proficiency panels conducted by the Cancer Vaccine Consortium (CVC/SVI). Cancer Immunol Immunother 57(3):303–315. doi: 10.1007/s00262-007-0380-6 CrossRefPubMedGoogle Scholar
- 13.Britten CM, Janetzki S, Ben-Porat L, Clay TM, Kalos M, Maecker H, Odunsi K, Pride M, Old L, Hoos A, Romero P (2009) Harmonization guidelines for HLA-peptide multimer assays derived from results of a large scale international proficiency panel of the Cancer Vaccine Consortium. Cancer Immunol Immunother 58(10):1701–1713. doi: 10.1007/s00262-009-0681-z CrossRefPubMedGoogle Scholar
- 15.Smith SG, Joosten SA, Verscheure V, Pathan AA, McShane H, Ottenhoff TH, Dockrell HM, Mascart F (2009) Identification of major factors influencing ELISpot-based monitoring of cellular responses to antigens from Mycobacterium tuberculosis. PLoS One 4(11):e7972. doi: 10.1371/journal.pone.0007972 CrossRefPubMedGoogle Scholar
- 16.Britten CM, Gouttefangeas C, Welters MJ, Pawelec G, Koch S, Ottensmeier C et al (2008) The CIMT-monitoring panel: a two-step approach to harmonize the enumeration of antigen-specific CD8+ T lymphocytes by structural and functional assays. Cancer Immunol Immunother 57(3):289–302. doi: 10.1007/s00262-007-0378-0 CrossRefPubMedGoogle Scholar
- 18.The MIATA project (2010)http://miataproject.org/. Accessed September 24
- 23.Ornatsky O, Bandura D, Baranov V, Nitz M, Winnik MA, Tanner S (2010) Highly multiparametric analysis by mass cytometry. J Immunol Methods. doi: 10.1016/j.jim.2010.07.002
- 27.MIACA (minimum information about a cellular assay) (2010) http://sourceforge.net/projects/miaca/. Accessed September 24
- 29.Alvarez DF, Helm K, Degregori J, Roederer M, Majka SM (2009) Publishing flow cytometry data. Am J Physiol Lung Cell Mol Physiol. doi: 10.1152/ajplung.00313.2009