1 Introduction

The ultrafine particles (less than 100 nm) emitted by engines are suspected to be more toxic than large particles because they can be inhaled more deeply into the lungs [14]. Studies have suggested that surface area of the particles correlates better with the inflammatory response rather than mass [5, 6]. It is well known that the ultrafine particles contribute little to the total particulate matter (PM) mass but they are the dominant contributor to the particle number. The equipping of the engine with a diesel particulate filter (DPF) reduces PM and particle number (PN) by more than 90 and 99 %, respectively [79]. In the USA, DPFs have been used to meet PM standards for heavy duty engines as DPFs have effectively reduced PM mass, ultrafine particles, black carbon, and air toxics [10, 11]. In 2012, United Nation health agency re-classified diesel engine exhaust as “carcinogenic to humans” [12]. However, a recent study showed that the health impact of filtered diesel exhaust showed no evidence of cancer-causing effects, in contrast to unfiltered diesel exhaust [13].

Introduction of particle number standard in Euro VI ensures the use of DPF on diesel engines and provides better health benefits and air quality. The new measurement procedure for counting PN was defined based on the work of the Particle Measurement Programme (PMP) of United Nations Economic Commission for Europe (UNECE) [14]. The PN measurement requires the counting of only solid particles with diameters bigger than 23 nm. The measurement technique involves hot exhaust dilution, a volatile particle remover (VPR), further dilution to keep the volatile material from condensing and to lower the exhaust temperature to a particle number counter (PNC) with a cut off diameter of 23 nm. A VPR’s function is to desorb and evaporate volatile compounds which should minimize the contribution of nucleation mode particles and leaves solid particles to be counted by PNC. Although this PN measurement method has been proven robust in eradicating volatile particles, there are some limitation to the PN measurement system such as volatile fraction of the particles which is removed prior to particle count has also been proven to be responsible for some adverse health effects. Moreover, some engine technologies might emit solid particles <23 nm which are also difficult to be captured by DPF [1517].

In this study, measurement capability of PN instruments based on PMP method was investigated. Mamakos et al. [18] observed the PN variation to be between 30 and 70 % when PMP’s golden engine ran on cold/hot World Harmonized Transient Cycle (WHTC), World Harmonized Stationary Cycle (WHSC), and European Transient Cycle (ETC) test cycles. However, a worse variation in the range of 90–150 % was observed when the engine ran on European Stationary Cycle (ESC) test cycles. Giechaskiel et al. [19] showed that the intra-lab repeatability was ∼40 % and inter-lab variation was ∼25 %. Although this study was on light duty vehicles, results clearly showed the difficulty in quantifying particle numbers. The PN measurement repeatability in another study [25] was found to be impacted by emission levels. The repeatability changed from ∼5 % at emission levels >5E12 #/kW-h to 30 % for emission levels >3E11 #/kW-h. The between-laboratories variability was found to be around 30 % for emission levels >3E11 #/kW-h and >50 % for lower emission levels. In all of the particle number measurement studies, the variability due to the source (engine) and the test process was included.

This present study captured variation under the following conditions: one engine, three AVL Particle Counters (APC) units, multiple engine installations of the engine and APC units in multiple Constant Volume Sampler (CVS) test cells in one facility, multiple test cell operators, one APC unit operator, two types of emissions test cycles, and one test plan. It quantifies the variation between three APC units and demonstrates the measurement capability of the PN method. Moreover, variations due to just APC unit and just the test process were computed.

2 Experimental Details

2.1 Test Engine

A Cummins heavy duty 2007 ISX 14.8 L engine with a maximum rated power of 425 HP was used in this study. The engine out exhaust was treated by passing it through a Cummins diesel oxidation catalyst (DOC) and a Cummins diesel particulate filter (DPF) in order to meet the 2007 emission standards. Selected engine properties are provided in Table 1.

Table 1 Selected properties of the Cummins ISX engine

2.2 Test Cycles and Test Matrix

Euro VI regulations for particle number emissions from a heavy duty diesel engine required measurements over the WHSC and WHTC. However, in this study, measurements were made over the standard heavy duty transient Federal Testing Procedure (FTP) and the Ramped Modal Cycle (RMC) used in the USA since the measurements were to be taken as part of the normal baseline engine testing sequence. The baseline engine is an engine that is tested in the various CVS test cells at the Cummins Technical Center.

The ISX engine was tested nine times in 68 days at five CVS test cells. A typical heavy duty baseline engine test plan included regeneration of the DPF, followed by 5 hot FTPs (first set of hot FTPs), regeneration of DPF, 3 RMC, 5 hot FTPs (second set of hot FTPs), and concluded with the regeneration of the DPF.

2.3 APC Set Up

All three AVL Particle Counter units (APC 489, AVL List GmbH, Graz, Austria) were installed at the CVS location when the engine visited any of the test cells involved in this study. The schematic of test cell set up is shown in Fig. 1. Samples were taken near the dilution tunnel center line and at approximately 10 times tunnel diameters downstream of exhaust entrance into the tunnel. Sampling probes were facing upstream into gas flow with axis of sampling parallel to dilution tunnel. Although recommended by Regulation 49 [20], particle size pre-classifier such as a cyclone or a “hatted” sample probe was not used prior to the inlet of the APC chopper diluter as studies have shown negligible impact on total particle counts on using a pre-classifier [21]. In two test cells, chopper diluters of APC were installed horizontally to the axis of dilution tunnel whereas in other three test cells, chopper diluters were installed vertically to the axis of dilution tunnel due to space limitation. Three APC units used in this study comply with the requirements of EU Regulation no. 49. All units were calibrated prior to the study by the manufacturer, AVL. The APC includes a key subcomponent—a condensation particle counter (CPC, model 3790) manufactured by TSI Inc. (St. Paul, MN, USA). The CPCs were calibrated by TSI. Two of the APC units were calibrated at the same time at the AVL’s facility in Graz, Austria. The third APC was calibrated at different time and was calibrated by Southwest Research Institute (SwRI), and its CPC was calibrated by TSI.

Fig. 1
figure 1

The schematic of the test cell set up. The Chopper Diluter is the part of the APC unit

3 Results and Discussion

3.1 Impact of DPF Conditioning

A typical PN measurement from any given test cell followed the trend shown in Fig. 2. Clearly, the first set of hot FTPs had higher PN emissions than the second set of hot FTPs. The global average BSPN of 131 observations from the first set of hot FTP cycles was 58 % higher than the global average BSPN calculated from the second set of hot FTP cycles. The higher particle number emissions during the first set of hot FTP cycles were due to the fact that the DPF was regenerated (all the accumulated soot was removed) before the first set of hot FTP cycles whereas DPF had acquired certain amount of soot by running RMC test cycles before the second set of hot FTP cycles. The soot cake inside the DPF itself acted as filtering medium for the DPF and increased its efficiency to clean diesel exhaust and subsequently resulting in the repeatability of PN measurements. Other studies have seen similar trend where elevated particle numbers were observed after the regeneration event (empty DPF) with repeatability of measurement more than 35 % which decreased to less than 15 % at lower levels after soot cake was formed [22, 23]. Figure 2 also depicts that in a test plan used in this study, at least three RMC test cycles have to be run prior to the hot FTPs in order to achieve stabilized particle numbers at lower levels after a soot cake is formed. In this study, repeatability of BSPN measurements over RMC test cycles which occurred right after the regeneration event varied between 7.4 and 32.4 % where as in the case of FTP test cycles, repeatability was observed to be between 0.7 and 3.3 %. Yamada, 2013 [24] showed in his study that PN emissions decreased exponentially after regeneration for three WHTC test cycles and finally got stable after the 10th cycle. However, after 15th cycle, he saw an increase in PN emissions. In our study, 7–11 % increase in PN emissions were observed relative to the 1st hot FTP in the second set from the 8th hot FTP test cycle. Such differences can be attributed to differences in test conditions, such as the DPF volume relative to the engine displacement. In this study, BSPN measurements over the FTP were significantly below the EU standards (6E + 11 #/kW-h over WHTC) but in cases where PN is close to the particle number standards, the state of the DPF will play a key role in the certification of the engine which could depend upon test engine, size, and type of DPF, number of conditioning cycles prior to certification cycle, and non-SCR or SCR equipped [24]. Therefore, during the development phase of the engine, above factors should be evaluated and the number of conditioning cycles should be determined in order to ensure low and repeatable BSPN measurements.

Fig. 2
figure 2

A typical series of test cycles that were run from a test plan during any visit. This example is from test cell 215 where PN measurements were made from APC unit 1, 2, and 3

3.2 Variation Within Test Cells

The test plan included two sets of hot FTPs cycles. The first set of FTPs was excluded from the analysis presented in this study. This was due to the fact that the DPF regeneration cycle was run prior to the first set of FTPs and the PN emissions had not reached stability because of insufficient soot loading in the DPF. Therefore, the second set of FTPs was selected to characterize the repeatability and reproducibility of BSPN measurements during FTP test cycles.

The total variation in BSPN measurements is accounted by calculating the coefficient of variation (COVunits) between measurements made by three APC units (Eq. 1). Repeatability (Ra) of BSPN measurements within each test cell, when hot FTP cycles are run, is accounted by calculating the COV between each test in the second set of hot FTP cycles. The percentage difference is defined as the ratio of the difference between APC(i) and APC(j) and average of APC(i) and APC(j) as shown in Eq. 3 where the subscripts i and j stand for different APC units under consideration.

$$ {\mathrm{COV}}_{\mathrm{units}}^k=\frac{\upsigma_{\left(\mathrm{A}\mathrm{P}\mathrm{C}1,\;\mathrm{A}\mathrm{P}\mathrm{C}2,\;\mathrm{A}\mathrm{P}\mathrm{C}3\right)}}{\upmu_{\left(\mathrm{A}\mathrm{P}\mathrm{C}1,\;\mathrm{A}\mathrm{P}\mathrm{C}2,\;\mathrm{A}\mathrm{P}\mathrm{C}3\right)}}*100 $$
(1)
$$ {\mathrm{Ra}}^{\mathrm{cycle}}\left({\mathrm{APC}}_i\right)={\mathrm{COV}}_{(i)}^{\mathrm{cycle}}=\frac{\upsigma_{(i)}}{\upmu_{(i)}} $$
(2)
$$ \mathrm{Percentage}\;\mathrm{difference}\left({\mathrm{PD}}_{ij}\right)=\frac{{\mathrm{APC}}_{(i)}-{\mathrm{APC}}_{(j)}}{\upmu_{\left({\mathrm{APC}}_{(i)},\;{\mathrm{APC}}_{(j)}\right)}}*100 $$
(3)
$$ {\mathrm{Rb}}^{\mathrm{cycle}}\left({\mathrm{APC}}_i\right)=\mathrm{P}\mathrm{D}=\frac{\left|{\mathrm{visit}}_1{\mathrm{APC}}_{(i)}-{\mathrm{visit}}_2{\mathrm{APC}}_{(i)}\right|}{\upmu_{\left({\mathrm{visit}}_1\;{\mathrm{APC}}_{(i),}{\mathrm{visit}}_2{\mathrm{APC}}_{(i)}\right)}}*100 $$
(4)

where

COV kunits :

Coefficient of variation between BSPN using APC units for visit k into test cell (k = 1 or 2)

σ:

Standard deviation

μ:

Mean

Racycle(APC i ):

Repeatability of BSPN using APC i (i = unit 1 or unit 2 or unit 3) for (FTP or RMC) cycle

COV cycle(i) :

Coefficient of variation of BSPN using APC i during FTP or RMC test cycles

(PD ij ):

Percentage difference between two specific APC units

Rbcycle(APC i ):

Reproducibility of BSPN using APC i (i = unit 1 or unit 2 or unit 3)

Visit1 and visit2 :

First and second visit into the test cell, respectively

3.2.1 Coefficient of Variation

Figure 3 shows the COV between units for hot FTPs during each test cell visit. The COV is based on the average of consecutive FTPs in the second set. The range of COV was between 3.1 and 8.5 % with an average of 5.2 %. The COV was not calculated during the first visit of test cell 213 and 214 because APC unit 3 was not available due to technical difficulty. The COV between APC units defines variability from unit to unit and therefore, low COV is desired to minimize the variation due to units.

Fig. 3
figure 3

The average of PN measured from consecutive hot FTPs from the second set. The PN was measured from all available APC units during each visit of the test cells. The COV between APC units was calculated and presented on the secondary axis

3.2.2 Repeatability

The repeatability of BSPN measurements by all three APC units during RMC and FTP test cycles is presented in Table 2. The test engine visited four test cells twice and one test cell just once. The repeatability range for each visit is presented in Table 2. The repeatability of BSPN measurements over FTP test cycles for each APC unit was between 0.7 and 3.0 %. Also, Fig. 3 represents average particle number emissions in #/kW-h measured by each unit from the second set of hot FTP cycles for all nine visits. Clearly, a good agreement is observed between units irrespective of test cell location. Small error bars (1σ) indicate the good repeatability of BSPN measurements from each APC unit for the second set of hot FTP cycles. However, repeatability of BSPN measurements over RMC test cycles was between 9.9 and 34.4 % (18.8 ± 9.5 %) and 8.3–32.5 % (20.9 ± 8.8 %) for visit 1 and visit 2, respectively. The large range in repeatability during RMC test cycles was due to soot-free DPF as explained in the section above. The large range in repeatability during RMC test cycles was also investigated. It was found that the first RMC cycle sometimes had unusual spikes which was captured by all three APCs (Fig. 4) causing undesirable repeatable numbers. After neglecting the first RMC cycle, repeatability improved to 1.7–18.9 % (9.3 ± 5.7 %) and 4.4–26.5 % (10.3 ± 9.3 %) for visit 1 and visit 2, respectively.

Table 2 The ranges of repeatability of BSPN measurements from all three APC units when engine ran RMC and FTP test cycles during its two visits to each test cell except TC 212
Fig. 4
figure 4

Real-time traces of three consecutive RMC test cycles that were run on the test engine during its second visit to test cell 312

3.2.3 Reproducibility

The reproducibility of BSPN measurements by all three APC units during RMC and FTP test cycles for each test cell except test cell 212 is presented in Table 3. Given that the test engine visited each test cell twice, reproducibility of BSPN measurements based on two visits for each test cell ranged between 2 and 37.1 % over FTP test cycles. Figure 3 also reflects the reproducibility during FTP test cycles. The impact of data from test cell 212 on reproducibility is discussed in the next section. The range of reproducibility (4.8–54.7 %) for RMC test cycles was poorer as expected.

Table 3 The ranges of reproducibility of BSPN measurements from all three APC units over RMC and FTP test cycles

3.3 Overall Variations

In this section, overall variation was computed based on 9 FTP sets of repeat tests irrespective of test cells. The overall variation can be broken down into three categories: part-to-part, repeatability, and reproducibility. As only one engine was involved in this study, repeatability and reproducibility were determined and data from 53 test cycles were analyzed. The total number of BSPN observations was 149. The global (average) BSPN throughout the testing period of 68 days for hot FTP cycles was 1.3918E + 11 #/kW-h. The global BSPN was used to calculate variation in BSPN due to the test process and also due to the APC unit only.

3.3.1 Repeatability

The repeatability (RaBSPN) of the BSPN measurements was 3.5288E + 09 #/kW-h (2.5 % of global BSPN) for hot FTP test cycles. It is defined as the standard deviation (s) of repeat measurements within a series using the same APC unit. The repeatability was accounted for by calculating the root mean square (RMS) of the standard deviations from each series of repeat tests on each APC unit as shown in Eq. 5.

$$ {\mathrm{Ra}}_{\mathrm{BSPN}}=\sqrt{\frac{{\displaystyle {\sum}_{j=1}^m}{\displaystyle {\sum}_{i=1}^{n_u}}{{\mathrm{s}}_i}^2}{{\displaystyle {\sum}_{j=1}^m}{n}_u}} $$
(5)
n u :

is the number of APC units providing data on the mth set of repeat FTP cycles (usually 3)

m :

is the number of sets of repeat FTP cycles (9)

s i :

is the standard deviation of repeat FTP cycles in the mth set for the APC units.

Table 4 represents variation in repeatability due to APC and test process. The repeatability (RaAPC) of just APC unit was 9.6150E + 08 #/kW-h (0.69 % of global BSPN). This is an estimate of the standard deviation (s) of the short term variation (random error) of just the APC units. This standard deviation is estimated by calculating RMS value from the difference in BSPN readings between each pair of APC units (APC1-APC212), APC1-APC313), and APC2-APC323)) for each set FTP cycle of repeat tests. Assuming all instruments have the same random errors and then the random error of one instrument is equal to computed (RMS) value divided by the square root of 2 as shown in Eq. 6.

$$ {\mathrm{Ra}}_{\mathrm{APC}}=\sqrt{\frac{{\displaystyle {\sum}_{j=1}^m}{\left({\mathrm{s}}_{\varDelta_{12}}^2+{\mathrm{s}}_{\varDelta_{13}}^2+{\mathrm{s}}_{\varDelta_{23}}^2\right)}_j}{23\times 2}} $$
(6)
m :

is the number of sets of repeat FTP cycles (9); 23 is the total number of standard deviations

Table 4 Repeatability of BSPN measurements, APC units, and test process

3.3.2 Reproducibility

Reproducibility of BSPN (RbBSPN) measurements during hot FTP test cycles by using three APC units was 4.4155 #/kW-h (31.7 % of global BSPN) for hot FTP test cycles. It is quantified by calculating standard deviation of averages (μ) of repeat readings using three APC units with a single calibration as shown in Eq. 7. In a recent study by Mamakos et al. [18], repeatability and reproducibility was found to be between 30 and 70 %, when the Particle Measurement Program’s “golden” engine was run on cold WHTC, hot WHTC, WHSC, and the European Transient Cycle (ETC). Their repeatability and reproducibility worsened to 90–110 % and up to 150 %, respectively, when engine was run over ESC test cycle.

$$ {\mathrm{Rb}}_{\mathrm{BSPN}}=\mathrm{stdev}\left({\displaystyle \sum_{j=1}^m}{\displaystyle \sum_{i=1}^{n_u}}{\upmu}_i\right) $$
(7)

If data from TC 212 is voided as unrepresentative (Fig. 3), then the reproducibility of BSPN measurements improves to 14.4 %. One could suspect the background level in the tunnel to be higher in TC 212. Figure 5 represents real-time traces of the first hot FTP cycle from second set of hot FTP cycles collected when engine visited five CVS test cells. Clearly, it can be seen that at the low concentrations level, contribution of the background was minimal in all CVS test cells. Therefore, there are no reasons for considering data from TC 212 as an outlier.

Fig. 5
figure 5

Real-time traces of the first hot FTP cycle from second set of hot FTP cycles collected when test engine visited five CVS test cells

Assuming particle number source to be stable, reproducibility of BSPN measurements was affected only by variation in APC units and test process. Table 5 represents impact on reproducibility by APC and test process. The reproducibility (RbAPC) of just APC unit was 8.6086E + 09 #/kW-h (6.2 % of global BSPN). First, the averages of difference between each pair of APC units (APC1-APC212), APC1-APC313), and APC2-APC323)) for each set of repeat tests were computed and then RMS of those values was calculated. This value was then divided by the square root of 2 which represents reproducibility of single instrument with a single calibration.

Table 5 Reproducibility of BSPN measurements, APC units, and test process
$$ {\mathrm{Rb}}_{\mathrm{APC}}=\sqrt{\frac{{\displaystyle {\sum}_{j=1}^m}{\left({\upmu}_{\varDelta_{12}}^2+{\upmu}_{\varDelta_{13}}^2+{\upmu}_{\varDelta_{23}}^2\right)}_j}{23\times 2}} $$
(8)

m is the number of sets of repeat FTP cycles (9); 23 is the total number of standard deviations averages

If we consider test engine as a constant particle source (true value) and no variation due to test process, then uncertainties of particle number measurements could be due to random and systematic errors of APC units. Random errors or precision is defined as error in measurement that lead to measurable values being inconsistent when repeated measures of a constant attribute are taken. Random errors are unpredictable. In case of APC units, random errors could be in terms of noise and the drift. The noise of the APC unit is typically <1 %, and it originates from small variations in the flows, pressure, and temperatures. The drift of APC over a short period of time (1–2 h) is negligible (<1 %) because APC components (MFCs, temperature, and pressure sensors, etc.) drift more slowly than this time scale [25].

Systematic errors are predictable and typically constant and proportional to the true value. In the APC units, systematic errors can be due to calibration factor for volatile particle remover (VPR) particle concentration reduction factor (PCRF) and particle number counter (PNC) unit within the APC. Giechaskiel et al. [25] estimated this error to be between 9 and 18 %, depending upon the concentration levels measured by the PNC and the PCRF setting.

3.3.3 Total Variation

Total variation is defined as the sum of the variance of repeatability and reproducibility of BSPN measurements. Based on 149: BSPN measurements from three APC units, variation due to repeatability and reproducibility was found to be 0.6 and 99.4 % of total variation, respectively (Table 6).

Table 6 Variation in BSPN measurements due to repeatability and reproducibility

3.4 Variation Between Units

Over the period of this research study, significant variation was observed between three APC units. Out of 134 test cycles, in some cases, variation up to 20 % was noted. On average, percentage difference between APC unit 1 and 2, unit 1 and 3, and unit 2 and 3 were 4.9, −3.8, and −7.2 %, respectively. This suggests that during most of the test cycles, APC unit 1 and 3 were higher than unit 2 and followed each other more closely. In Fig. 6, the percentage difference between APC units was plotted against BSPN. The trend lines show that the agreement between APC 1 and 2 and APC 2 and 3 improved with increase in BSPN from 5.0E + 10 to 3.0E + 11 #/kW-h. In case of APC 1 and 3, no significant change in percentage difference was observed across the BSPN level encountered in this study. Figure 6 also suggests that the delta between units (APC 1 and 2, APC 2 and 3) decreased with increase in BSPN level whereas there was no significant change in delta between APC 1 AND 3 with increase in BSPN level. The higher variation of APC unit 2 is unclear. APC 1 is the oldest among all three APCs and is different model number than APC 2 and 3. Moreover, all three APCs were calibrated within 2 months to each other which leaves out the possibility of variation due to the difference in serial number and calibration dates. Even though APC units demonstrated good repeatability, these variations observed between units and the impact of particle number magnitude question the measurement capability of these instruments.

Fig. 6
figure 6

Percentage difference between APC units vs. BSPN (#/kW-h)

Recent study [26] has observed that the PNCs can drift 5–10 % every 3–6 months. AVL internal database of more than 40 PNCs reported that approximately 30–40 % of the PNCs that came for yearly calibration were drifted by more than 20 %. A possible explanation to the drift is the formation of ester due to the reaction of vehicle exhaust gasses and butanol and depositing on the saturator, affecting the partial pressure of butanol and, subsequently, the super-saturation ratios in the condenser. The VPRs can also drift up to 10 % in the dilution range 100–5000 over 1 year because of some blockage in the orifice or small leaks in the system. Overall, a 20 % drift of the APC unit is possible over the period of 1 year.

Current state-of-art of APC units cannot identify drift which raise concerns for the measurements conducted between two calibration checks. Therefore, frequent on-site checks of the VPR and PNC are important to calculate drift of APC with time. These checks can be performed by comparing measurement with a reference PNC in parallel with the PNC of APC unit. Use of PNC to measure upstream and downstream of the APC unit at a specific PCRF can show if the VPR of the APC has drifted or not.

4 Conclusions

The engine had very large variation in BSPN, which was observed on all three APC units. This was attributed to variation related to the conditioning of the DPF. Therefore, second set of FTPs was used to characterize repeatability and reproducibility of BSPN. Given this variation, it is expected that engine and after-treatment preconditioning will be a common practice when measuring BSPN. The range of COV between units for hot FTPs during each test cell visit was between 3.1 and 8.5 %. The repeatability of BSPN measurements over FTPs within test cell was between 0.7 and 3.0 %. However, RMCs cycles which were run right after regeneration event demonstrated the repeatability range between 8.3 and 34.4 %. This study provides the formulas for segregating variation due to test process and measurement system (APC). The global average BSPN throughout the testing period of 68 days for hot FTP cycle was 1.3918E + 11 #/kW-h well below the threshold for Euro VI, albeit with the caveat of a different test cycle being used. The BSPN measurement repeatability was 2.5 % of the global average whereas repeatability of just APC unit was 0.69 % of the global average. The BSPN measurement reproducibility of 31.7 % of the global average was observed in this study. The reproducibility of APC unit was found to be 6.2 % of the global BSPN. The repeatability and reproducibility were found to be 0.6 and 99.4 % of the total variation, respectively. Among 134 test cycles, during some cycles, significant variation up to 20 % was noted between two APC units. The agreement between units improved with increase in the BSPN level. Frequent on-site checks and validation are required to ensure reliable measurements for engine certification testing.