Background

Patients admitted with acute signs of respiratory failure are a challenge for emergency physicians and among the most common indications for admission to the emergency department (ED) among adults [1, 2].

Symptoms are caused by a variety of respiratory and circulatory diseases of which some of the most common are exacerbation in chronic lung disease, pneumonia, pulmonary embolism and congestive heart failure [1, 3]. A fast and correct diagnosis can be crucial as vital signs are often close to normal in those potentially life-threatening diagnoses with a substantial risk of ICU admission [3]. Moreover, these patients are among those with the highest over-all 30-day mortality [4].

Over the past decade point-of-care ultrasound (PoCUS) examination in the ED has gained its place in the armamentarium of diagnostic tools [5, 6].

PoCUS of the lungs has proven to be a helpful tool in the assessment of patients with suspected respiratory diseases and has proven to be superior to chest x-ray in the diagnostic assessment of various pulmonary diseases [7,8,9].

Combined PoCUS of the heart and lungs has significantly improved the diagnostic accuracy of various cardiopulmonary diseases (e.g., pneumonia, pulmonary embolism, or edema) and identified life threatening diseases missed at primary clinical examination [10,11,12].

Studies have demonstrated that novice sonographers can find pathology with accuracy comparable to expert sonographers and reference standards with only little training [13, 14].

The amount of diagnostic accuracy studies is vast, and we are short of studies of the PoCUS´ impact on patient related outcomes. Results from diagnostic accuracy studies are rarely directly translational to clinical practice, hence we need randomized pragmatic studies to investigate the potential impact of PoCUS in daily clinical practice.

We investigated the hypothesis that adding a cardiopulmonary PoCUS to standard clinical examination of patients admitted to the ED with acute signs of respiratory failure could increase the proportion of patients with a presumptive diagnosis in agreement with final diagnosis (previously described as `correct presumptive diagnosis´ in the published protocol) at four hours (4 h) after admission when compared to standard clinical examination alone [15].

Methods

Study design

This prospective pragmatic randomized semi-blinded, multicenter, superiority trial with a parallel group design and allocation ratio of 1:1 was undertaken in 10 of the 21 EDs in Danish community Hospitals. Three of these EDs were located at university hospitals and seven in secondary or tertiary hospitals. Patients were enrolled from October 9, 2015 to April 5, 2017.

Acute admissions to Danish EDs are to public hospitals and established by the general practitioner or by emergency call. Exceptions are patients suspected of having heart disease who are admitted directly to the cardiology department.

A presumptive diagnosis at admission and at 4 h as well as a treatment plan must ideally be journalized by the emergency physician (EP) within 4 h from the patient’s admission to the ED.

Inclusion and exclusion

We screened patients admitted to the ED for inclusion. Patients ≥18 years with a primary sign or symptom of respiratory failure of; cough, dyspnea, chest pain, respiratory frequency > 20 or peripheral oxygen saturation < 95% or any combination of these were included upon written informed consent. Exclusion criteria were inability to give written informed consent, PoCUS of the lungs or heart already performed by others than the investigator in relation to the primary examination, or inability to randomize or perform the PoCUS within 4 h from the patient’s admission to the ED.

Randomization and masking

Randomization numbers were created by a block randomization database (REDCap, OPEN) using permuted blocks of random numbers to ensure equal numbers of patients in both trial arms at each center [16]. The allocation sequence of randomization numbers was generated by a data manager from REDCap OPEN and the project manager, MR, using an online random number service. Randomization numbers were paired with the REDCap OPEN database developed for this trial by MR.

The investigators screened the patients for inclusion at alternating shifts including day and night shifts on all weekdays. Once a patient signed the informed consent, the investigator registered the patient in the trial database that randomly allocated the patient and assigned the patient a unique computerized randomization number. Patients were aware of their group assignment.

The investigator then noted the basic clinical values upon the patient’s admission and performed the PoCUS unaware of the primary presumptive diagnoses. All PoCUS results were entered in the project database.

PoCUS results from patients in the control group remained blinded to the EP, whereas once the investigator had received the primary presumptive diagnoses, he unblinded the intervention groups PoCUS results to the EP both orally and in writing on a paper record marked with the patients ID number and stored it in an accessible briefcase in the ED. The EP was then free to re-evaluate his presumptive diagnoses and treatment according to the ED’s clinical guidelines. The EP was instructed to leave all information of randomization and PoCUS results out of the medical records to ensure blinding of the medical record audit.

In the medical record the investigator only noted the projects identification number and localization of the paper records containing the PoCUS results.

Procedures

In all patients admitted to the ED the EP assessed the primary presumptive diagnoses using standard methods of diagnostic examination (e.g., clinical examination, blood samples, ECG, chest x-ray) as soon as possible after the patient’s arrival.

Standard diagnostic tests were available within 4 h. Supplementary imaging examinations such as computerized tomography (CT), ultrasonography and echocardiography performed by specialists were available if necessary.

The investigator performed the PoCUS blinded to the EP in both groups. Subsequently, the EP announced his primary presumptive diagnoses to the investigator who then unblinded the PoCUS findings in the intervention group.

The PoCUS was performed within 4 h from the patients´ admission to the ED and consisted of PoCUS of the heart and lungs. The PoCUS protocol was defined as follows:

The PoCUS of the lungs was a modification of the ultrasound protocol used by Laursen et al. originally modified from the principles of lung ultrasound by Volpicelli and Lichtenstein [12, 17, 18]. It was performed as follows: The anterior and lateral part of thorax was divided into a superior and inferior quadrant. Each quadrant represented a zone in which the probe was placed centrally to create a transverse picture of the costae and pleurae. We looked for pleural effusion, interstitial syndrome/pulmonary edema and pneumothorax.

The PoCUS cardiac ultrasound was performed according to the principles described in the international evidence based guideline [19]. The views used were the 4-chamber picture of the heart achieved either from a sub-xiphoid or an apical window. We looked for pericardial effusion, altered left ventricular ejection fraction and right ventricular overload.

There were no regulations in the choice of ultrasound machines or probes used for inclusion as long as the PoCUS was performed with an image quality deemed sufficient for evaluation by the operator.

All PoCUS examinations were performed by investigators who were either specialists or in specialist training and who received patients in the ED on a regular basis. All investigators used PoCUS on a daily basis in their clinical practice but had varying degrees of PoCUS experience. Prior to become an investigator they all received an educational program regarding data collection and PoCUS examination, which was composed by MR and consisted of an e-learning presentation with instructional videos [20]. Then, MR made a 4 h on site presentation comprising the project, the collection and registration of data and an introduction to the project’s PoCUS protocol. Hereafter, MR evaluated each investigator’s PoCUS skills and handling of the project database by hands on and by multiple choice questionnaire tests to ensure that the investigators were familiar with their investigator tasks prior to initiation of inclusion. During the period of inclusion, investigators could take daily contact to the project manager for questions.

The diagnostic criteria for the PoCUS examination are provided in Appended file 1.

Diagnostic examinations

The primary presumptive diagnoses were registered in the medical record. The EP was free to reassess the presumptive diagnoses, diagnostic tests, and treatment.

The 4 h presumptive diagnosis was the last registered clinical examination made by the EP within 4 h from the patients´ admission to the ED and was assessed by blinded audit of the medical record after the patients discharge from hospital. New diagnostic tests and treatments could be prescribed after the 4 h examination but would then be part of the final diagnosis.

The final diagnosis was assessed by blinded audit of the medical record and included electronic journal data (e.g. clinical, microbiological and biochemical data, and imaging results) and was performed after the patients´ discharge from hospital by two auditors, who independently of each other, set the final diagnosis. In case of discrepancy a third auditor set the final diagnosis. A predefined audit protocol with diagnostic criteria of the most common diagnoses was used (Appended file 2).

Outcomes

The primary outcome was to assess the percentage of patients with a presumptive diagnosis in agreement with final diagnosis at 4 h after the patient’s admission to the ED. The 4 h cut off was set because various Danish hospitals request that examination, presumptive diagnoses and plan for further treatment are assessed within 4 h from admission to the ED [21].

The secondary outcomes were as follows:

  1. 1.

    Diagnostic accuracy of the primary presumptive diagnoses made upon arrival.

  2. 2.

    The proportion of patients who upon arrival received a primary presumptive diagnosis in agreement with final diagnosis.

  3. 3.

    The proportion of patients who, within 4 h after admission to the ED, is given the appropriate treatment.

  4. 4.

    Time spent in the ED (hours).

  5. 5.

    Time spent in hospital (days).

  6. 6.

    The proportion of patients who were transferred from the ED to the intensive care unit.

  7. 7.

    The proportion of patients who were transferred from the ED to a hospital ward.

  8. 8.

    The proportion of patients who were discharged from the ED and directly to their home.

  9. 9.

    The proportion of patients who were readmitted to hospital within 30 days from discharge.

  10. 10.

    In-hospital mortality.

  11. 11.

    30-day mortality.

As a negative PoCUS can be found in various pulmonary diseases (e.g. chronic obstructive pulmonary disease (COPD), asthma) we could not compare PoCUS findings directly with final diagnoses and were omitted from the secondary endpoint `The proportion of PoCUS examinations with a correct diagnostic examination´ as written in the published protocol [15].

Unblinding

For safety and ethical reasons the PoCUS findings in the control group were unblinded to the EP in charge of the patient if the PoCUS raised suspicion of a life-threatening condition (e.g. pulmonary edema, pneumothorax, pericardial effusion, or heart failure) [15].

Statistical analysis

The sample size estimate was based on the results from a previous similar trial where about 65% of the patients had a presumptive diagnosis in agreement with final diagnosis at 4 h after admission to the ED when PoCUS was not used [11]. A clinically significant improvement of the diagnosis by using PoCUS in a multicenter trial was set to be 15%.

To detect a 15% increase in the number of presumptive diagnoses in agreement with final diagnoses, from 65% in the control group to 80% in the intervention group, with an 80% chance for detection, a level of significance of 5% and with an estimated drop out of 6% we had to include 288 patients, 144 patients in each group (intervention/control). Sample size calculations were made with the online database for clinical trials; Sealed Envelope [22].

We used the intention-to-treat method as main comparative analysis on all participants. Descriptive statistics were handled as follows: Categorical data by number and percentages of patients with 95% confidence intervals (CI), continuous data by number of patients (n), mean, standard deviation, median, minimum and maximum.

Missing data in the baseline characteristics were handled as simple imputation when represented as dichotomous data. Other missing data were evaluated to be missing at random and was handled by multiple imputations using the Markov Chain Monte-Carlo method including auxiliary variables in the model. We plotted the mean value of the imputed variables in the spot of missing data. The rule of three was used to find 95% CI in categories without events. Categorical endpoints were summarized by numbers and percentages with 95% confidence intervals [23].

We used the Chi [2] test and the Fischer exact test for comparison of proportions expressed as percentages and for the comparison of continuous endpoints we used the Student t test (means) and the Mann Whitney test (medians). A two-sided significance level of 5% was applied to all tests.

We used the diagnoses from blinded audit as reference test for determining the diagnostic accuracy of the presumptive diagnoses established at admission and at 4 h and their 95% confidence interval based at binominal distribution. To assess the interrater reliability of the final diagnoses we used the Cohens kappa coefficient. Data analyses were conducted using STATA Release V. 15.0 (Stata Corp) including professional statistical advice. To assess the time a patient spent in the ED or hospital and the mortality we withdrew data from the Danish National Patient Register (LPR) administered by the Danish Health Data Authority. No data monitoring committee oversaw the trial.

Role of the funding source

Funders of this trial had no role in the trial design, data collection, data analysis, data interpretation or writing of the report. The corresponding author had full access to all the trial data and had final responsibility for the decision to submit for publication.

Results

We included 220 patients from October 9, 2015 to April 5, 2017 in 10 hospitals counting three university hospitals and seven tertiary hospitals with a total of 21 investigators. The trial was ended before we reached the predefined sample size as investigators travelled to other departments and the inclusion stagnated. We initially excluded 9 patients: two patient contacts due to double inclusions (their latest inclusions were excluded), one patient who withdrew informed consent, one patient who had no randomization or PoCUS within 4 h from admission and seven patients due to loss to follow up.

For the intention to treat analyses 211 patients (96%) were randomized: intervention n = 106 (50%) and control n = 105 (50%). We then excluded 10 patients due to unblinding of the PoCUS results as they were unblinded for the treating physician due to signs of one of the predefined life threatening conditions within 4 h from admission in 6 patients (control n = 2, intervention n = 4) and as the investigator became treating physician within 6 h from admission in 4 patients (control group). We thereby ended with 201 patients (91%) in the per protocol analyses; intervention n = 102 (51%), control n = 99 (49%) (Trial profile Fig. 1).

Fig. 1
figure 1

Trial profile (appended). *The 7 randomized patients we lost due to loss of follow up are assessed to be missing at random and were excluded from all analyses as all outcomes were to be calculated by statistic approximation. They were lost as follows: 4 were caused by investigators in two hospitals who withdrew their consent to participation due to too busy working hours in the ED, in 2 the medical records were lost in the transition to a new electronic medical record system, and in 1 an unidentifiable personal ID number was written in the database. Abbreviations: ED (emergency department). PoCUS (Point-of-care ultrasound)

Full size image

There are no deviations of the PoCUS examination in either group due to adverse events or complications related to the trial and no patients had PoCUS performed in the ambulance prior to admission to the ED. For patient baseline characteristics see Table 1. For PoCUS findings see Appended file 3.

Table 1 Baseline characteristics of the intention to treat population
Full size table

Final diagnoses were assessed by the two auditors with an overall agreement of 93·18% (kappa 0·58) ranging from 81·04–100% (kappa − 0·02–1·0) within the specific diagnoses. A detailed description of the interrater reliability of the audit is provided in Appended file 4. The most common final diagnoses in the intervention and control group were; pneumonia (25%/ 32%), exacerbation of COPD (25%/27%) and systolic heart failure (16%/21%). The proportion of presumptive diagnoses in agreement with final diagnoses at 4 h in the intervention group is increased compared to the control group for the diagnoses of exacerbation of COPD (89%/75%), pulmonary edema (53%/33%) and para-pneumonic effusion (77%/38%) but with overlapping CIs. The proportion of correctly diagnosed pneumonia is equal among the groups (88%/88%) (Tables 2, 3).

Table 2 Final diagnoses and the proportion of presumptive diagnoses in agreement with final diagnoses at 4 h
Full size table
Table 3 Diagnostic accuracy of emergency physicians´ 4 h presumptive diagnoses compared to final diagnoses
Full size table

No difference was found in the proportion of patients who received a presumptive diagnosis in agreement with final diagnosis at arrival; intervention group, n = 65 (61·32%; 95% CI 51·59–70·23) versus control group, n = 65 (61·90%; 95% CI 52·12–70·80), (p = 0·93) nor at 4 h after admission to the ED; intervention group, n = 84 (79·25%; 70·32–86·02) versus control group, n = 81 (77·14%; 68·00–84·28) (p = 0·71). However, the proportion of patients who had an appropriate treatment prescribed at 4 h was significantly larger in the intervention group, n = 84 (79·25%; 95% CI 70·32–86·02) than in the control group n = 69 (65·71%; 95% CI 55·99–74·27) (p = 0·03) with an absolute increase of 13·5% (Table 4).

Table 4 Primary and secondary outcomes for the intention to treat population
Full size table

In the intervention group, a significantly increased proportion of patients had supplementary diagnostic tests performed within the first 4 h, and the most frequent diagnostic test was PoCUS of the lungs performed by a physician specialized in lung ultrasound; intervention n = 34 (32·1%; 23·3–41·8), control n = 10 (9·5%; 4·7–16·8) (p = 0·0001).

Moreover, a significantly increased proportion of patients in the intervention group spent less than 1 day in hospital, n = 42 (39·6%; 25·8–38·4) compared to the control group n = 25 (23·8%; 16·5–33·0) (p = 0·01) clarified by an absolute increase of 15·8%. Aside from that we found no significant changes in the proportion of time spent in the ED or in hospital, in the patients´ itinerary after discharge from the ED or in the proportion of readmissions and mortality (Table 4).

The diagnostic accuracy of the 4 h presumptive diagnoses compared to final diagnoses are listed in Table 2 and added contingency tables in Appended file 5. For `Other diagnoses´ than final diagnoses, see Appended file 6. Causes of death in patients who died within 30 days from admission are summarized in Appended file 7. For per protocol analysis, see Appended file 8.

No adverse events related to the PoCUS were observed.

Discussion

To our knowledge, no similar randomized pragmatic multicenter trials of adding PoCUS to standard clinical examination have been performed in adult ED patients admitted with signs of acute respiratory failure in which they have been looking at both the diagnostic accuracy, the proportion of appropriate treatment prescribed and the length of stay.

In this trial we found that adding PoCUS to standard clinical examination in adult patients with acute signs of respiratory failure led to an insignificant, absolute increase of 2·1% (p = 0·71) in the proportion of patients with presumptive diagnoses in agreement with final diagnoses at 4 h after admission to the ED. We observed a significant, absolute increase of 13·5% (p = 0·03) in the proportion of patients who had appropriate treatment prescribed within 4 h from admission and of patients who stayed less than 1 day in hospital (Table 4).

We find our patient population comparable to similar studies. The basic characteristics of our study groups were highly consistent regarding demographic and clinical characteristics, with only a slight imbalance in the distribution of gender and they had a comparable combination of vital signs and medical history upon admission [11, 24].

The insignificant increase of the presumptive diagnoses in agreement with final diagnoses at 4 h is surprising. The high 4 h diagnostic accuracy found in the control group was unexpected when compared to a similar single center trial, which reported an absolute increase of presumptive diagnoses in agreement with final diagnoses of 24·3% (from 63·7% to 88·0, 95% CI 15·0–33·1) and of appropriate treatment prescribed of 21·2% (from 56·7 to 78%, 95% CI 10·8–30·9) at 4 h after the patients admission (p < 0·0001). They added cardiopulmonary PoCUS to standard clinical assessment performed by a single physician specialized in PoCUS [11]. However, this does not explain the high 4 h diagnostic accuracy we found in both groups nor the significant increase of appropriate treatment prescribed that we found in the intervention group [11].

Instead, the high diagnostic accuracy at 4 h can be due to the augmented focus on getting more specialist doctors in front in the first critical hours from the patients´ admission to the ED, which may have increased the diagnostic accuracy in general.

The significant increase of 13.5% in the proportion of patients in the intervention group who, at 4 h from admission, had an appropriate treatment prescribed despite an insignificant difference in diagnostic accuracy was similar to the single center study by Laursen et al. who found an absolute increase of 21·2% (Table 4) [11].

Laursen et al. found that the diagnostic tests prescribed within the first 4 h in the intervention group had a higher proportion of tests in which the suspected diagnoses were confirmed, but that the proportion of diagnostic tests prescribed evened out when compared throughout the entire hospital stay [11]. These results indicate that the implementation of PoCUS leads to faster execution of diagnostic tests earlier in the hospital stay to confirm or reject a presumptive diagnosis and could be a contributory cause to the significant increase in the proportion of appropriate prescribed treatment at 4 h in the intervention group. Finally, combined with the significant increase in the proportion of intervention patients who spent less than 1 day in hospital, these results indicate that the PoCUS has a positive impact on several clinical outcome parameters.

Apart from the above mentioned, we found no difference in the patients´ itinerary, time spent in hospital or in the rate of readmissions or mortality as found by Laursen et al. 11 The length of time spent in the ED or in hospital is often fairly short hence, larger studies are needed to assess whether PoCUS has a benefit on these parameters.

In our study population we find a low number of severely ill patients (Table 1) and surprisingly few with pulmonary edema, pulmonary embolism and acute myocardial infarction despite a greater proportion of patients with systolic heart failure than found in similar trials [11, 24, 25]. We cannot rule out that a selection bias has taken place where severely ill patients have been deselected due to the more cumbersome process of including these patients in a study trial.

We may have reduced the number of final diagnoses of pulmonary edema (Table 2) as the investigators´ PoCUS findings of pulmonary edema did not overrule a negative chest x-ray in the diagnostic criteria despite that PoCUS of the lungs has shown higher diagnostic accuracy. And as we did not implement pro-brain natriuretic peptide (BNP) in the diagnostic criteria, we may have reduced the number of final diagnoses of cardiac pathology [9]. The proportion of patients with pulmonary embolism or AMI might be low due to the Danish emergency department setting that bypasses the ED and directs these patients straight to the department of cardiology or to outpatient clinics, and because the prevalence of cardiopulmonary diseases is known to vary across studies despite similar inclusion criteria (Table 2) [11, 24, 26]. Moreover, we limited the subjective bias from the blinded audit of the final diagnoses by predefined diagnostic criteria based on internationally accepted guidelines. However, the kappa agreement between the two auditors was only weak to moderate according to Cohen’s guidelines, probably because the auditors had to agree on all diagnoses to obtain agreement or because the kappa statistic values tend to underestimate the agreement in situations with high inter-observer agreement (Appended file 4). However, the inter-observer variability reflects daily clinical dilemmas where a final clear diagnosis is not always possible to establish. Hence, we find no reason to believe that the manner of evaluating the final diagnoses plays a significant role in the results as it has been tested in similar studies [11, 27, 28]. This study indicates that cardiopulmonary PoCUS in the hands of non-specialist sonographers can impact patient treatment, underlined by studies of steep learning curves which demonstrates that novice sonographers can find pathology comparable to gold standards [13, 14]. The pragmatic multicenter trial design increases the generalizability of the results.

Nevertheless, the study has limitations. Firstly, we never reached sample size (inclusion: 220 versus sample size: 288) which might have caused the lack of statistically significant results. Second, enrolment only took place when the investigators were present in the ED, which might have caused bias. Third, the broad inclusion criteria prompted the inclusion of a large number of patients with diseases others than cardiopulmonary. Compared to other studies we ended up with several patients who had pathology that the PoCUS has no diagnostic impact on which may have caused a reduction in the absolute effect of our outcome parameters [11]. Fourth, our diagnostic criteria may have been too strict as we refrained from applying the BTS criteria that indicates that pneumonia can be diagnosed if there is “no other explanation for the illness, which is treated as community acquired pneumonia (CAP) with antibiotics” (Appended file 2), and from the investigators positive PoCUS findings of pneumonia, despite that PoCUS of the lungs has proven to be superior to chest x-ray in diagnosing pneumonia. This is emphasized by a higher proportion of diagnoses of pneumonia at 4 h than at final diagnoses in both groups as well as a lack of increase in the diagnostic accuracy (Table 3) [8, 29]. Fifth, the general absence of a CT-thorax or complete echocardiography in the assessment of the final diagnoses is a weakness in the standard criteria of the final cardiopulmonary diagnoses and may have reduced the amount of significant findings. But, as it applies to all included patients we believe it to be of less importance.

Our study was not designed robust enough to enlighten the actual impact that PoCUS might have on acute as well as on hard clinical outcomes as patient morbidity, mortality and time spent in hospital. Larger studies are needed to specify the recommendations and the selection of patients whom would benefit the most from this diagnostic test as it has substantial healthcare and socioeconomic potential.

Conclusion

PoCUS in combination with standard clinical examination, performed in patients admitted to the ED with acute signs of respiratory failure, did not increase the proportion of patients who received a presumptive diagnosis in agreement with final diagnosis within 4 h from admission. However, PoCUS in combination with standard clinical examination was superior to standard clinical examination alone as it increased the proportion of patients, who had appropriate treatment prescribed within 4 h from admission and who spent less than 1 day in hospital.