In-house testing prior to the collaborative trial
The specificity of the multiplex PCR method was first evaluated by theoretical in silico tests in order to predict potential false positive results (ENGL 2021). Unexpected similarities to other target sequences could not be identified. No negative PCR effects were observed during the primer check with the software tool OligoAnalyzer for possible formation of secondary structures (dimers and hairpins). The deltaG value for the dimer analysis was below − 10 kcal/mol, indicating a suitable primer design. The melting temperature of the formed hairpins showed in each case the recommended minimum temperature of 50 °C below the annealing temperature of the PCR.
Prior to the inter-laboratory validation, the specificity of the method was tested in-house. No amplification was obtained when testing DNA from non-target species according to Sect. 2.5.1a). For the species birch, an amplification signal was detected in the hazelnut system. However, to our knowledge birch has no relevance in terms of food. Furthermore, the variation of the response was tested when analysing materials of the target species from different origins and—if available—of different cultivars.
A sufficiently low variation of the response was observed for all detection systems ranging from 17 to 37% (data not shown). In table S2 (Supplementary Material), the results for walnut and hazelnut are shown as an example. Relative standard deviations of all values were 16.8% (walnut) and 36.9% (hazelnut), compared to 11.0% (walnut) and 36.4% (hazelnut) when using the singlecopy target sequence according Köppel (2010). Additionally, it has to be mentioned, that for hazelnut the detection method is only genus specific (Corylus spp.). The test of the quantitative response was carried out with commercially available references predominantly derived from Corylus maxima. Equally, nuts from Corylus avellana are detected by the method. The results of the tests with the available references hint to a response of C. avellana being about factor two to four stronger than the response of C. maxima.
In none of the four detection channels, crosstalk was observed. Thus, the acceptance criteria example of ENGL (2021) could be fulfilled for the parameter crosstalk.
The levels of the serial dilutions 10, 5, 2 and 1 mg/kg of each “nut” were detectable in six of six PCR replicates each. Additionally, the level of 5 mg/kg was still detectable in co-presence of high amounts of DNA of the other target species. Therefore, an asymmetric LOD of at least 5 mg/kg of each “nut” was estimated by in-house validation. The lowest spiked levels of incurred materials were detectable in eight of eight PCR reactions each: hazelnut and peanut: 0.9 mg/kg (rice cookie); walnut, cashew: 5 mg/kg (sauce powder).
Precision and trueness
Good precision data with relative standard deviations > 25% were obtained for all tested materials and spiked levels (detailed data not shown). However, bias, i.e. deviation of measured concentration in mg/kg from incurred level, was more than 50% for some materials and levels. This was the case, e.g., for the quantification of peanut, hazelnut and walnut in some sausage and sauce powder materials. The overall measurement uncertainty (see also data from collaborative trial) was estimated to be in the range of ± 50% (detailed data not shown).
PT samples hazelnut
All six samples were correctly classified as positive or negative. The recoveries of the five spiked potato powder materials spiked with different preparations of hazelnut were 59% (nut nougat and nut crocant), 64% (nut spread with cocoa), 68% (nut crocant), 108% (hazelnut roasted) and 223% (hazelnut unroasted).
Peanut: comparison roasted and unroasted
The influence of the roasting process on recovery was exemplarily tested for the peanut quantification. The results of quantification of 1000 mg peanut/kg in rice flour was 100 mg/kg for roasted peanut mix and 960 mg/kg for the unroasted peanut material corresponding to recovery rates of 10.0% and 96.0%, respectively.
Previous studies indicate that the considerable decrease of recovery is caused by the degradation of DNA during the process of roasting (see e.g., DLA 2019; Rossmanith 2011).
Collaborative trial validation
No deviations from the protocol were reported by none of the 12 participating laboratories. Three labs asked to repeat the analysis with a new set of samples for different reasons (use of unsuitable plastic ware, deviation from the protocol by mistake, suspicious results and amplification curves of the sensitivity control and two samples respectively).
Calibration standard DNA and matrix standard DNA
Table 3 summarises the results of the calibration series. Performance criteria for standard curves are described, e.g., in guidelines for GMO analysis (BVL 2016; ENGL 2015, 2021; Broeders et al. 2014). According to these guidelines, the slope of the standard curve should be between − 3.1 and − 3.6 and R2 should be > 0.98. For multiplex PCR used for qualitative analysis, a slope between − 2.9 and − 3.9 is considered to be acceptable (Broeders et al. 2014).
The results of lab No. 12 were considerably outside of these requirements for all PCR systems and therefore could not be taken into account, neither for qualitative nor quantitative evaluation. Furthermore, the results of two laboratories were excluded from the quantitative evaluation of the walnut PCR due to clearly deviating values from the standard curve for walnut (Table 3). In Table 3, for each PCR system the number of results (labs) fulfilling the standard curve criteria are given. In total, range of labs with satisfactory results was from 9 of 12 (walnut, slope and R2) to 11 of 12 (cashew, slope).
Sensitivity control of 0.64 mg “nut”/kg could be amplified in 110 out of 110 reactions for each PCR module (0% false negative rate; lab 12 was not considered, see above).
Qualitative evaluation of samples
The samples were decoded and assigned to the seven different materials. Per material and lab, six PCR results (three DNA extracts analysed in duplicates each) were obtained.
Table 4a–d show the results of the qualitative evaluation of the inter-laboratory validation. For each of the four spiked allergenic ingredients, the expected results were obtained, with some exceptions: for the rice cookie not incurred with hazelnut, in 15 of total 66 reactions, amplifications with Cq values from 34.4 to 38, mainly between 37 and 38 were obtained. No false-positive reaction resulted when comparing the Cq with the lowest standard (see Sect. 2.7) though. Similar results were reported for the walnut “blank” material. Unavoidable contaminations of the rice cookie material used as “blank standard”, are most probably the cause for these observations. Due to a calculated level of less than 0.1 mg/kg, the impact of the contamination on the qualitative evaluation is negligible.
The evaluation of the peanut results for the vegan cookie (5 mg/kg), compared with the mean Cq value of the lowest standard dilution, yielded 29 (of 66) false-negative results (= 44%). As previously mentioned, the use of roasted peanuts for spiking leads to significantly lower recoveries (see Sect. 3.1). Anyhow, for the veggie burger spiked with roasted peanuts at the level 10 mg/kg, the results were positive consistently, i.e. applying both methods for evaluation. Finally, for the rice cookie incurred with 5 mg/kg one false-negative reaction was obtained (= 1.5% of all reactions), when evaluated by comparison of Cq values.
Overall, the results demonstrate that a detectability of 5 mg peanut, hazelnut, walnut and cashew per kg [equivalent to approx. 1.3 mg (peanut), 0.7 (hazelnut), 0.7 mg (walnut) and 0.9 (cashew) protein per kg (BLS 3.02)] is still feasible in processed foods. If, as a”worst case”, in addition to being processed, the incurred food contains roasted instead of unroasted “nuts”, a LOD of 10 mg/kg can still be achieved.
Quantitative evaluation of samples
Table 5a–d summarise the statistical evaluation of all quantitative inter-laboratory validation results. Due to the lack of appropriate guidelines and performance criteria related to allergen analysis, guidelines for real-time PCR-based GMO analysis were used for the evaluation of the results (ENGL 2015, 2021). Due to the high proportion of false-negative results, data received for the vegan cookie spiked with roasted peanuts at the level 5 mg/kg were not taken into consideration for quantitative evaluation.
According to the guidelines for quantitative GMO analysis (ENGL 2015), the deviation of the mean value from the true value or an accepted reference value should not exceed 25%.
According to the results of the inter-laboratory validation study, this requirement could not be fulfilled in most cases for the different materials [peanut: criterion fulfilled in one of six materials, hazelnut (3/5), walnut (2/4) and cashew (2/6)].
The deviations from the true value (incurred level) ranged from − 4.7% for cashew in sausage to − 84.7% for walnut in vegan cookie (see Table 5a–d, “recovery”). It has to be noted that for the calibration of all results, a dilution series prepared from rice cookie material was used and the calculated recoveries are referring to %-deviations of the results in relation to the rice cookie material (= 100%) (Siegel et al. 2013). For none of the materials, the spiked “nuts”, production batch (year of production of incurred materials) and matrix were identical. The “nut” material spiked into the rice cookie standard was also used for rice cookie 2 and the sausage material. Identical materials were also spiked to sauce powder and rice cookie 1 as well as to vegan cookie and veggie burger. All rice cookie materials were made according to the same protocol (Siegel et al 2013). Especially for the materials spiked with roasted nuts, considerable deviations from the spiked level were obtained. They can mainly be explained by the loss of recovery during roasting (see Sect. 3.1).
Moreover, in the quantitative range, the relative repeatability standard deviation (Sr, rel) should not exceed 25% and the relative reproducibility standard deviation (SR, rel) should be below 35%. At the level of the quantification limit, a value below 50% is still acceptable (ENGL 2015, 2021). Except for SR, rel of 1.9 mg/kg hazelnut in rice cookies (25.5%) and of 5 mg/kg hazelnut in sauce powder (28.1%), the data of these two precision criteria were in the acceptable range for all four “nut” species, all spiked levels and all materials. This is even the case for the rice cookie material at the very low spiked level of 0.9 mg peanut/kg (see Table 5a–d, “Sr, rel” and “SR, rel”). However, it has to be pointed out, that the collaboration trial was designed in such way, that the extraction was performed by one laboratory and the analysis of unknown samples was limited to already prepared DNA solutions. A certain increase of SR, rel and Sr, rel could be expected, if the DNA extraction step was performed individually by each lab. In previous validation studies, DNA extraction had been included (Siegel et al. 2013; Waiblinger et al. 2014, 2017), thus experience to what extent the individual DNA preparation leads to additional uncertainty is already available. Within the collaboration trial, the performance of the PCR method had to be evaluated.
Finally, the relative measurement uncertainty was estimated by pythagorean addition of the relative deviation of the true value and the relative reproducibility standard deviation. Measurement uncertainties of less than 50% were calculated for peanut in two of six materials, for hazelnut in three of five, for walnut in three of four and cashew in four of six materials.