Evaluation of the 1R6F certified reference cigarette for proficiency testing of mainstream smoke parameters

A goal of the cooperative agreement between the U.S. Food and Drug Administration and the Center for Tobacco Reference Products (CTRP) at the University of Kentucky was to produce a standard reference cigarette for researchers to further tobacco science. The 1R6F certified reference cigarette for analytical testing was developed to be typical of an American blend cigarette. The cigarettes have been used in proficiency testing (PT) schemes under the supervision of the CTRP since 2016 for a variety of parameters. Data were compared from 8 rounds of PT schemes submitted by participating laboratories, which include various international stakeholders comprised of academic institutions, federal research/regulatory laboratories, tobacco product manufacturers, and independent contract testing laboratories. Evaluation of the datasets for harmful and potentially harmful constituents (HPHCs) within the mainstream smoke of the 1R6F certified reference cigarette included tobacco-specific nitrosamines (TSNA), tar, nicotine, and carbon monoxide (TNCO), and Benzo[α]pyrene (BαP). Of the datasets evaluated, less than 5 % resulted in either a warning or alert, and less than 2 % resulted in an alert. Comparisons across multiple PT rounds revealed no significant differences in the consensus means of the tested parameters. Evaluation of data submitted by participating laboratories for multiple proficiency test rounds shows that the 1R6F certified reference cigarette is an acceptable material as a control reference for the analysis of HPHCs which is imperative since some stakeholders may be required to participate in, and receive satisfactory results, from PT for accreditation purposes.


Introduction
Proficiency testing (PT), which indicates the analytical performance of a laboratory for a particular method and analyte in comparison with other participating laboratories, has become a necessary part of third-party performance evaluation for analytical laboratories [1]. With the passage of legislation in the United States known as the Clinical Laboratory Improvement Act (CLIA) of 1967, and amended in 1988, clinical laboratories in the United States have been required to participate in PT for laboratory evaluation [2,3]. Since 2005, the standard governing for the general requirements for the competence of testing and calibration laboratories (ISO/IEC 17025) has required participation in PT [4]. PT cannot only identify laboratories that generate unsatisfactory results, but also identifies laboratories competent at properly performing analysis when compared to other laboratories [5]. PT providers have a significant ability to help participating laboratories achieve proficiency in routine analytical results [6]. Laboratory quality assurance program effectiveness can be evaluated by PT and improved by understanding the root causes of failures during a PT program [6]. PT can be used not only to monitor overall improvements in analysis by participants, but also to identify laboratory-to-laboratory or method-to-method differences [7]. A limitation of PT is the focus on the analytical and reporting 1 3 aspects of data, with no real consideration of the pre-and post-analytical handling of samples and analyses that may lead to random and systematic errors impacting the data [7]. In general, a PT scheme relies on analysis of identical samples by different laboratories for interlaboratory comparison following statistical analysis of the data provided by the laboratories to identify outliers or errors which the participating laboratories are expected to identify the cause of the errors identified [4].
PT data submitted by participating laboratories are statistically evaluated by determination of the Z-score. Participant performance in PT is based on the absolute value for Z-scores and is classified into three categories: |Z| ≤ 2 is a satisfactory result, 2 < |Z| < 3 is a warning result, and |Z| ≥ 3 is an alert result [1]. Laboratories that participate in PT and receive a warning or alert result are encouraged to conduct an internal investigation to identify the source of the error [6]. It has been shown that analytical methods, especially those that are not routinely employed by laboratories or follow protocols that are vague, cause the greatest impact on the rate of outliers and errors in data reported by participants of PT [4]. There is a correlation between higher incidences of unsatisfactory results for PT performance and the use of complicated processes for sample preparation and analysis more complicated the process is for sample preparation and analysis correlates with a higher incidence of unsatisfactory results during PT programs [6]. Review of errors from over 200,000 PT results from toxicology laboratories revealed 58 % of the unsatisfactory results were caused by laboratory mistakes and instrument malfunction, 32 % were caused by inaccurate transcription of test results to the proficiency test report forms, and the cause for 10 % could not be identified [6]. The most common laboratory mistake was specimen dilution and accounted for 20 % of the spurious values, while instrument malfunction accounted for 15 % of the spurious values [6]. Since the implementation of PT, improvements in technology have attributed to the better performance of participating laboratories as a result of ease of use, speed of operation, simpler sample handling, and automatic or less frequently required calibration [3]. A number of events can lead to a reduction in the number of unsatisfactory results, including, removal of laboratories which consistently produce unsatisfactory results, improvement in PT materials and report forms, familiarity with the program by repeat participants, identification of sources of errors and corrective actions, adoption of more accurate and reproducible methods, and improvement in technical education and performance [8].
Since 1968, the University of Kentucky has provided reference tobacco products as standards for non-clinical investigational purposes by tobacco manufacturers, contract and government laboratories, and academic institutions. Multiple reference cigarettes have been made available, with a new reference cigarette being produced as inventory of the previous reference cigarette is depleted. Comparison studies have been performed and published to ensure that the new reference cigarette is comparable to the previous version. For example, when the 3R4F was produced to replace the 2R4F cigarette, analysis of the mainstream smoke and in vitro and in vivo toxicology in 2011 revealed that there were no meaningful differences between the reference cigarettes [9]. The 3R4F reference cigarette provided by the University of Kentucky (UK) Center for Tobacco Reference Products (CTRP) has widely been used as a monitor or comparator cigarette for analysis of mainstream smoke constituents and in vitro and in vivo toxicology assays [10]. As the availability of the 3R4F reference cigarette diminished due to dwindling inventory, CTRP was awarded a Cooperative Agreement with the U.S. Food and Drug Administration's Center for Tobacco Products (FDA CTP) to develop a cigarette tobacco reference product in 2014. A goal of the cooperative agreement between the FDA CTP and UK CTRP was to produce a standard reference cigarette for researchers to further tobacco science as identified in the Family Smoking Prevention and Tobacco Control Act of 2009 that amended the Food, Drug, and Cosmetic Act. The 1R6F certified reference cigarette was produced in March 2015 as a reference material typical of an American blend cigarette. Approximately 50 million test pieces were produced to meet the need for quality control testing in the tobacco industry and to support research in both academic and industrial settings. The 50 million test pieces are expected to be available for multiple years as a consistent standard for tobacco product research. It is known that the levels of salient monitoring parameters in cigarettes vary due to inherent variability in tobacco as an agricultural crop. When several types of tobacco are blended to produce an American-type cigarette, this variability may increase. The 1R6F certified reference cigarette represents a suitable matrix for the quantitative analysis of salient parameters in blended cigarette products and is the first certified reference cigarette (Certificate of Analysis [11] available at ctrp.uky. edu). It has been shown that the 1R6F certified reference cigarette is a suitable substitute for the 3R4F cigarette based on analysis of the mainstream smoke constituents and in vivo and in vitro toxicity [10]. With the aim of enhancing the quality and traceability of measurements in analytical laboratories specializing in tobacco smoke chemistry, the CTRP was established utilizing funds from the CTP under funding through RFA-FD-14-001. It should be noted that the 1R6F was the first certified tobacco reference available to the tobacco research community with proven homogeneity and stability for use as PT material. Since the 1R6F certified reference cigarette was made available for non-clinical trial tobacco research, it has been used as test material in a number of analytical studies [12][13][14][15].
In February 2017, the CTRP program was accredited by American Association for Laboratory Accreditation (A2LA) in accordance with the recognized ISO/IEC 17043:2010 standard to be an Accredited Proficiency Test Provider. The A2LA certification process included the analysis of compounds and physical properties utilizing the 1R6F certified reference cigarette. A2LA is an independent, non-profit, internationally recognized, accredited body in the United States that provides a full range of comprehensive laboratory accreditation services. To maintain the accreditation status, laboratories that are ISO/IEC 17025 certified as Testing/Calibration Laboratories are required to participate in and achieve satisfactory results for analytes tested. Currently, the 1R6F certified reference cigarette is the only reference cigarette product covered under an A2LA scope of accreditation for use in PT schemes for analysis of multiple tobacco chemical and physical parameters. Presented here is a current assessment of the PT results that have been compiled by the CTRP. Comparison of data submitted by PT participants over multiple rounds of testing for several parameters within the mainstream smoke of the 1R6F certified reference cigarette reveals that the certified reference tobacco product is an acceptable material for proficiency testing.

Reference cigarette and PT at university of kentucky
A subset of the full production quantity of the 1R6F certified reference cigarette was selected for sole use in the PT program, based on low variability observed in the data collected during production of the entire batch of 50 million cigarettes. The PT calendar for analysis of analytes found within the mainstream smoke and physical characteristics of the 1R6F certified reference cigarette can be found at the CTRP website (https:// ctrp. uky. edu/ home). A protocol is published online for each PT round that identifies the particular parameters to be analyzed and references on the recommended analytical methods for the parameters. Evaluation of the datasets for harmful and potentially harmful constituents (HPHCs) within the mainstream smoke of the 1R6F certified reference cigarette included tar, total particulate matter (TPM), carbon monoxide (CO), nicotine, water, puff count, N-nitrosonornicotine (NNN), 4-(methylnitrosamino)-1-(3-pyridyl)-1-butanone (NNK), N-nitrosoanatabine (NAT), N-nitrosoanabasine (NAB), and Benzo[α]pyrene (BαP). Participants purchase a PT kit of 1R6F certified reference cigarette material to be tested on either a linear or a rotary smoking machine. The PT kit includes the 1R6F certified reference cigarette, protocol, Excel data submission template, and access to the study reports. In order to test on both types of machines, participants must purchase two PT kits. Regardless of the type of smoking machine, the testing can be performed under non-intense smoking regime (ISO 3308:2012), previously referred to as International Organization for Standardization (ISO) [16], and intense smoking regime (ISO 20778:2018) [17], previously referred to as the Health Canada Intense (HCI) [18], testing conditions, which have been incorporated with the ISO standard. To be consistent with the most current industry reporting standards, the testing conditions are referred to as non-intense and intense in the manuscript. Once a PT kit is selected and purchased, the 1R6F certified reference cigarette test material is sent to the participant for analysis. Participants are given access to download an Excel template for data submission and assigned a unique identifier. Participants are encouraged to submit data for each analyte in each PT round but are not required to do so. Once the data are submitted to CTRP, statistical analysis is performed, and an interim report is assembled and issued for participating laboratories to review their data submission for erroneously reported results, to make any needed corrections and to resubmit the corrected data prior to the release of the final report. Data resubmission is only allowed for participants that provide a valid scientific reason, such as incorrect units reported or incorrect calculations. After a period of time, comments and revised data, including the reason for revision, are compiled and analyzed for inclusion in the final report, which is used by individual laboratories to maintain accreditation if necessary.

Statistical methods
Once the data deadline passed for each round, the data are partitioned into 4 subsets: linear non-intense, linear intense, rotary non-intense and rotary intense. Each subset of data is reviewed for outliers using multiple outlier detection tests including Cochran's, Grubb's Single, and Mandel's h & k. Instead of removing outlying data points, robust means and standard deviations are calculated for each analyte using Algorithm A [19]. The standard deviation for proficiency assessment, pt , is calculated from the repeatability variance, the laboratory result is acceptable; when 2 < |Z| < 3 , the laboratory result is a warning; and when |Z| ≥ 3, the laboratory result is an alert. Graphical and tabular summaries of the outlier testing are presented in each report along with the critical values calculated by formulas enumerated by Wilrich [21]. After input from stakeholders, the proficiency reports provide raw and robust means and standard deviations, Z-scores and plots of the Z-scores by de-identified participants. All PT report analyses are conducted using internally developed SAS macros. This paper focuses on elements for which multiple rounds of testing had been completed and published for participants. The analytic and reporting process has been refined since the first round of testing. Most of the process changes involved adding data elements to the report, such as including raw means and standard deviations in the summary table, creating waterfall plots of Z-scores, and eliminating Z-scores when less than five laboratories report data for a particular analyte. Algorithm A is known to be unstable if more than 20 % of the data are outliers; hence, a minimum of five participants allows one participant to be an outlier. To ensure comparability, data from all proficiency rounds were re-analyzed using the latest version of the analysis macro in SAS. Consensus means and standard deviations are plotted. All analyses were completed with SAS 9.4.

Results
Since 2016, CTRP has offered PT for the analysis of the 1R6F certified reference cigarette. Analytes found within the mainstream smoke were collected using either a rotary or linear smoking machine using either the non-intense or intense smoking regime. Analytes, or parameters, within the mainstream smoke were divided into test elements based on chemical similarity for inclusion in PT schemes. Multiple test elements have been tested over the course of the PT calendar (Table 1). Tar, nicotine, and carbon monoxide are collectively known as TNCO and, in combination with water, total particulate matter (TPM) and puff count, represent the most commonly tested elements in the CTRP PT schemes (Elements 1 and 2). Tobacco-specific nitrosamines (TSNA) are a group of analytes known to be carcinogenic and, in combination with TPM and puff count, represent the second most commonly tested PT scheme (Elements 3-4). Benzo[α]pyrene (BαP), a carcinogenic polycyclic aromatic hydrocarbon, in combination with TPM and puff count, represents a PT scheme (Elements 5-6) that has been tested in combination with TNCO or TSNA. A Certificate of Analysis was released prior to the sale of the 1R6F proficiency test kits based on data collected from linear smoking machines. The latest Certificate of Analysis was released on October 12, 2018, and provides details on the characterization process, calculations and assumptions [11]. The x pt from each round was plotted by proficiency test along with reference line for the certified value and the confidence interval calculated from the certified uncertainty. The compiled data submitted for the five rounds for nicotine which is a part of the TNCO analyte group under the non-intense and intense smoking regime using both linear and rotary smoking machines are shown in Fig. 1. The proficiency rounds occurred over a period of approximately 5 years. Not only are the mean nicotine values consistent, but the bounds from the s pt are also stable across the rounds. Similarly, the complied data for NNN from the four rounds of the TSNA analyte group are displayed in Fig. 2. The mean NNN values are consistent across proficiency rounds. The bounds from the s pt for Round 2017C are noticeably larger than the other rounds for non-intense linear regime due largely to the small number of participants submitting data α. A similar trend is seen for the 3 rounds of testing of BαP displayed in Fig. 3.
When two analyte groups are analyzed within one PT round, a Cambridge filter pad is used for each analyte group which results in TPM and puff count being reported twice within that PT round. For clarification purposes, rounds with TPM reported for a second analyte group separately are denoted with a 'b' after the PT round designation. TPM is very consistent over the 12 sets of measurements for both nonintense and intense measures with linear and rotary smoking machines, though TPM results for the intense smoking regime on a rotary smoking machine reflect a loss of TPM due to the dead volume in the rotary smoking machine [22]. While under the intense regime, rotary smoking machines are unfailingly lower than the linear smoking machine values for each PT round, the mean TPM measurements across time for each smoking machine have little variation (Fig. 4). As noted earlier, all Certificate of Analysis certified values were calculated for cigarettes smoked using linear smoking machines.
For the proficiency reports, Z-scores are presented only if at least five laboratories report data for that parameter submitted. Warnings and alerts presented in the final report provide laboratories with feedback on their ability to properly perform the analysis of the parameters tested and provide documentation for accreditation reviews. Laboratories that receive a warning or alert are encouraged to investigate the root cause of the warning or alert to improve laboratory performance. Of the laboratories that participated in the TNCO rounds for nicotine and TPM measurements, there were only 14 instances of a warning or an alert for the datasets that were submitted which accounts for 3.7 % of all laboratories tested, with only 0.5 % of the data resulting in an alert. For the TSNA and BαP rounds, there were 12 instances of an alert or a warning which accounts for 4.3 % of all the laboratories tested. The rate of alerts is 2.9 % which is slightly higher than in the TNCO rounds, but most happened in the first PT round. While participants are not required to identify the accreditation status of the laboratory or to submit proof of accreditation, many participants have self-identified as being accredited. For each of the testing schemes evaluated, the percent of laboratories self-identifying as being accredited has increased, supporting the importance of proficiency testing for analytical laboratories.  over multiple rounds of testing for different parameters of interest were compared and evaluated to support the homogeneity and stability of the 1R6F certified reference cigarette. When the proficiency means for each parameter were compared across the different rounds, no visible trends or deviations were noted among the different proficiency rounds. Even the means for TPM, which are collected as part of each proficiency round, did not differ across all the rounds. Data collected for the Certificate of Analysis for the 1R6F certified reference cigarette were based on mainstream smoke utilizing a linear smoking machine and the non-intense and intense smoking regimes. Although the Certificate of Analysis is available at the CTRP website, it is not a part of the PT kits provided to participants neither is it used in calculations for scoring. In addition, the Certificate of Analysis was completed on a systematic sampling across the entire production of the 1R6F certified reference cigarette and therefore has a greater uncertainty.
Data analysis revealed only a few parameters had notable differences when comparing data collected utilizing a rotary smoking machine during PT testing and the Certificate of Analysis which was based solely on results from a linear smoking machine. Participants that performed the analysis and submitted data were evaluated based on the consensus values for each round of testing. The low percentages of warnings and alerts across the PT rounds show the 1R6F certified reference cigarette is a stable matrix for analytical testing of multiple parameters within mainstream smoke and can be utilized by laboratories to improve proficiency when analyzing these parameters.
In conclusion, data from the first five years of the PT studies on the 1R6F certified reference cigarette support the stability and homogeneity of the product across different types of smoke condensate parameters. The means across PT rounds provided fall within the certified ranges for all analytes reviewed which supports UK CTRP's decision to use consensus values for testing. Follow-up reviews of data collected from the proficiency testing program at CTRP will be performed as more datasets are submitted for the 1R6F certified reference cigarette and any other tobacco products that may be introduced, including certified reference smokeless tobacco products and cigars.  Funding Funding for this work was made possible, in part, by the U.S. Food and Drug Administration through grant UC2FD005049, UC2FD005671 and UC2FD006890. The views expressed in written materials or publications and by speakers and moderators do not necessarily reflect the official policies of the Department of Health and Human Services; nor does any mention of trade names, commercial practices, or organization imply endorsement by the United States Government. This research was supported by the Shared Resource Facilities of the University of Kentucky Markey Cancer Center (P30CA177558). Stacey Slone and C. Ruth McNees contributed to the study conception and wrote the initial draft of the manuscript. Formal analyses were completed by Stacey Slone and Brent Shelton, and data visualization was created by C. Ruth McNees and Stacey Slone. All authors commented on previous versions of the manuscript and read and approved the final version. Project administration and funding acquisition were completed by Ling Yuan and Orlando Chambers.
Availability of data and materials Proficiency test participant's identity is protected by confidentiality as required by ISO/IEC 17043. Data can be made available, but no information that can be used to identify laboratories can be shared.
Code availability Not applicable.

Declarations
Competing interests The authors declare no competing interests.

Conflicts of interest None declared.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/.