Introduction

The reliability of an instrument, test, or exam relies on its accuracy compared to the 'gold standard' for diagnosing a condition or disease. It is not different when pain is assessed. Self-reported measures, the gold standard in pain measurement, allow for evaluations in patients without neurological impairments, conscious and awake individuals, or those with sufficient cognitive development to report their perceptions of pain through scales, questionnaires or interviews1,2,3,4.

On the other hand, assessments of nociception combined with the best pain control strategy (analgesia) have encouraged studies with instruments that can evaluate pain objectively5. In this perspective, the Analgesia Nociception Index (ANI, Physiodoloris™; Metrodoloris, France) is a non-invasive tool placed on the market in the last decade. ANI is based on the analysis of the respiratory fluctuations of heart rate6,7.

The pain/analgesia evaluation algorithm8 was idealised and used in patients submitted to different procedures under general anaesthesia9,10,11,12,13 to assess autonomic nervous system (ANS) activity and thus optimise analgesic drugs prescription14,15. ANI analyses the balance of nociception/antinociception through heart rate variability (HRV) on a scale from 0 (maximum of nociception/predominance of the sympathetic nervous system) to 100 (complete analgesia/predominance of the parasympathetic nervous system), making a distinction between appropriate and inappropriate antinociception in anesthetised adult patients16,17,18,19,20,21.

Nevertheless, ANI has also been used in conscious patients because of its understandable mechanism, easy reading, and non-invasive characteristic22,23. Therefore, an overview of the results obtained by comparisons with self-reported pain tools could help define its accuracy.

The level of evidence to support the application of the ANI technology in awake patients is still unclear. Given that there is no standardisation in clinical references in the literature, this systematic review and meta-analysis aimed to compare ANI and self-reported measures for diagnosing pain in conscious individuals undergoing medical procedures or painful stimuli.

Methods

Protocol and registration

This systematic review and meta-analysis conform to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) checklist24. A protocol was drafted and registered in the International Prospective Register of Systematic Reviews (PROSPERO, CRD42018114439).

Eligibility criteria

Observational studies in which ANI was compared with any subjective measures (numerical scales or questionnaires) to assess pain in awake or conscious individuals undergoing medical procedures or painful stimuli were included. No restrictions regarding the age of participants and the study's language or year of publication were imposed. Studies reporting assessments of individuals with cognitive or neurological impairment were excluded. So were letters to the editor, meeting abstracts, and qualitative studies.

The following PIRD25 acronym was applied:

Population: conscious individuals undergoing medical procedures or painful stimuli.

Index Test: ANI.

Reference test: self-reported measures of pain.

Diagnosis of interest: pain.

Information sources

Computerised searches across five electronic databases were conducted in October 2018. The databases used were PubMed (National Library of Medicine), Scopus (Elsevier), Web of Science (Clarivate Analytics), Ovid (Wolters Kluwer), and Embase (Elsevier). An update took place in March 2021. In addition, the reference lists of the included articles were also screened for references that might not have been retrieved during the computerised searches. Finally, searches for literature in Open Grey and Google Scholar were undertaken; the searches were limited to the first 300 most relevant hits26. Duplicate references were removed upon identification. The references were managed using EndNote software (Thomson Reuters, Toronto, Canada; https://www.myendnoteweb.com).

Search strategy

The search strategy in PubMed, Ovid, Embase, and Web of Science was analgesia nociception index OR analgesia-nociception index. A specific search strategy was tailored for Scopus: "analgesia nociception index" OR "analgesia-nociception index", and for Embase: 'analgesia nociception index'/exp OR 'analgesia nociception index'. Searches in Google Scholar and Open Grey were carried out with the "analgesia nociception index OR analgesia-nociception index" algorithm.

Study selection

Study selection was conducted in two phases. In Phase 1, two review authors (DAB and LRC) read the titles/abstracts independently. The references whose titles/abstracts met the eligibility criteria were included straight away. In Phase 2, the same authors evaluated the full references with titles/abstracts containing insufficient information for a final decision. The references whose full texts met the eligibility criteria were also included. In both phases, divergences between authors were resolved by discussion until a consensus was reached.

Data extraction and data items

Two review authors (DAB, LRC) performed data extraction independently. Disagreements were resolved through discussion. If disagreements persisted, a third review author (LGA) decided. When additional or missing information was needed, the authors of the articles were contacted. The primary data were extracted and are cited in Table 1.

Table 1 Summary of characteristics and results of the included studies.
Full size table

Risk of bias of individual studies

Two review authors (DAB, LRC) independently carried out the quality assessment using the University of Adelaide critical appraisal checklists for diagnostic test accuracy studies27 and analytical cross-sectional studies28. Any disagreement between the review authors over the risk of bias in individual studies was resolved by a third review author (LGA). For both tools, each item could be answered with yes (low risk of bias), no (high risk of bias), unclear (unclear risk of bias), or not applicable.

Diagnostic accuracy measures

Accuracy and correlation measures were the outcomes. Sensitivity, specificity, positive predictive value, negative predictive value, area under the curve (AUC), and receiver operating characteristics (ROC) were used to measure diagnostic accuracy. Correlation between the ANI index and the self-reported pain diagnosis measures included r-values, p values, and confidence intervals (CI).

Synthesis of results and subgroup analysis

Articles included that were methodologically homogeneous were incorporated into meta-analysis. Subgroup analyses were conducted considering the subjective pain measure used, medical procedures or electrical stimulus applied. Correlation analyses between ANI and subjective measures were conducted.

Statistical heterogeneity of the analyses was assessed using the I2 statistics. In the meta-analysis with an I2 higher than 40%, the random-effect model was used. In the meta-analysis with an I2 lower than 40%, the fixed-effect model was used29. Meta-analyses were conducted with the MedCalc statistical software version 19.2.6 (MedCalc Software bv, Ostend, Belgium; https://www.medcalc.org; 2020). Correlation coefficients (r values) and confidence intervals (CI) were provided.

Results

Study selection

Eight hundred thirty-two references were identified across the five electronic databases and the grey literature. Following the removal of 351 duplicate hits, 481 titles/abstracts were screened in Phase 1. The full texts of 34 references were retrieved, and the eligibility criteria were applied in Phase 2. Following the evaluation, 16 articles15,16,22,30,31,32,33,34,35,36,37,38,39,40,41,42, assessing 1.602 individuals, fulfilled the eligibility criteria and were included in this study (Fig. 1). The complete reference of the 19 articles in Phase 2 and the reasons for exclusion are presented (Supplementary Appendix A).

Figure 1
figure 1

Flow chart depicting the search process.

Full size image

Study characteristics

Among the 16 articles included, six reported accuracy and correlation measures16,22,37,38,41,42, seven only correlation tests15,32,33,35,36,39,40 and three reported accuracy measures exclusively30,31,34 (Table 1).

Three articles indicated the calculation of the sample size36,37,38. One was a pilot study conducted with French women during labor22, and another was carried out with individuals being treated for burn wounds34. Three articles were on conscious and healthy volunteers32,33,35, ten about aware patients after procedures under general anaesthesia16,30,31,36,37,38,39,40,41,42, and one in patients on sedo-analgesia (no premedication was administered before the procedure)15.

The self-reported subjective measures used to compare with ANI were the Visual Analogue Scale (VAS)22,35,38,42 and the Numerical Rating Scale (NRS)15,16,30,31,32,33,34,36,37,39,40,41. Three studies34,37,38 used objective measures other than ANI: cardiovascular depth of analgesia (CARDEAN)34, Surgical Plethysmographic Index (SPI)37, the pupillary light reflex (PLR)37, and the variation coefficient of pupillary diameter (VCPD)38.

Risk of bias in individual studies

The methodological quality evaluation is summarised in Tables 2 and 3 (Supplementary Appendices B and C). The domain judged as having the highest risk of bias in accuracy studies was blinding the index test results concerning the reference standard because these results had not been cited or the test had not been performed.

Three correlation studies presented a high risk of bias in identifying confounding factors and strategies to deal with them15,38,41, while one study34 exhibited a high risk of bias in four items.

Results of individual studies

ANI performed well to detect moderate to severe pain upon arrival in the Post-Anaesthesia Care Unit (PACU), which was improved with propofol-based (AUC = 0.93) in comparison with halogenated-based anaesthesiaUC = 0.82)30. Likewise, Boselli et al.31, demonstrated a high negative predictive value of ANI: ANI ≥ 50, predicting that 92% of patients had appropriate analgesia (NRS ≤ 3) upon arrival in PACU for orthopaedic surgery (AUC = 0.93, 95% CI 0.86–0.97) and otolaryngology surgery (AUC = 0.83, 95% CI 0.75–0.90). ANI measures correlated well with subjective NRS scores in the postoperative period after using volatile agents and opioid-based anaesthesia in another study39. In two other studies, the measure used in a similar scenario was VAS. Jeanne et al.42 evidenced no correlation between ANI and VAS scores (Spearman rank test, r2 = − 0.164, P=0.25) in total knee replacement orthopaedic surgery. Charier et al.38 also found a similar weak negative correlation (Pearson correlation, r = − 0.15; P=0.006) in surgeries whose general anaesthesia and postoperative analgesia protocols had been left to the anaesthetist's discretion.

ANI was strongly correlated with VAS in postpartum women (p < 0.0001), in particular before epidural analgesia22 and presented a weak negative correlation (r = − 0.15). Two studies reported that ANI did not reflect different states of moderate to severe pain measured after sevoflurane-based general anaesthesia in adults16,37, revealing low sensitivity and specificity to detect the difference between NRS 0 and NRS 6–10 (AUC = 0.43) in one study16. In patients who had undergone colonoscopy under sedo-analgesia, ANI correlated significantly with NRS (r = − 0.402)15.

One study did not obtain satisfactory results when correlating ANI with NRS in three different groups for pain intensity (group I: NRS ≤ 3, group II: NRS 4–6, group III: NRS ≥ 7) in patients who had undergone laparoscopic cholecystectomies under sevoflurane/remifentanil anaesthesia40, and no correlation was observed between the postoperative NRS score and the postoperative ANIm values in elective supratentorial tumour surgery36.

Papaioannou et al.34, demonstrated considerable sensitivity (67%) and specificity (70%) of ANI in predicting pain. Furthermore, the accuracy increased when associated with another measure (CARDEAN). Thus, ANI was fit to measure nociception in a group of conscious burnt patients under analgesic effects during wound care procedures.

One study showed no correlation between ANI minima and NRS in individuals submitted to unexpected electrical pain or expected electrical pain, indicating that ANI was neither a specific nor a robust measure to assess pain intensity compared with NRS in conscious men. There was no correlation between minima ANI and NRS when assessing painful stimuli (rs = − 0.01, P = 0.97)33. Issa et al.32, showed a weak correlation between ANI and NRS (Pearson, − 0.089; 95% CI − 0.192 to − 0.014; P = 0.045), suggesting that ANI was not specific for the assessment of pain intensity in alert volunteers. Yan et al.36, evaluated conscious, healthy volunteers with a cold pressor simulator, showing that the correlation between ANI and VAS was negative and weak (r = − 0.27 and P = 0.017).

Synthesis of results and subgroup analysis

Nine studies were incorporated into a meta-analysis. Two subgroup analyses of correlation between ANI and NRS were feasible: (1) data of studies assessing conscious individuals who had undergone medical procedures under general anaesthesia were pooled; (2) data of studies evaluating participants submitted to electrical stimulus were pooled.

In the first subgroup analysis of correlation, seven studies16,30,31,36,37,39,41 were incorporated. This subgroup demonstrated a weak negative correlation between ANI and NRS (r = − 0.0984, CI  − 0.397 to 0.220, I2 = 95.82%). The random-effect model was used (Fig. 2). The second subgroup analysis32,33 compared the ANI and the NRS in individuals who had been submitted to electrical stimulus and showed a weak negative correlation (r = − 0.089; CI = − 0.390 to 0.228, I2 = 0%). The fixed-effect model was used (Fig. 3).

Figure 2
figure 2

Subgroup correlation analysis between ANI and NRS in individuals submitted to medical procedures under general anaesthesia (r = − 0.0984, CI = − 0.397 to 0.220, I2 = 95.82%). The random-effect model was used. MedCalc Statistical Software version 19.2.6 (MedCalc Software bv, Ostend, Belgium; https://www.medcalc.org; 2020).

Full size image
Figure 3
figure 3

Subgroup correlation analysis between ANI and NRS in individuals submitted to electrical stimulus (r = − 0.089; CI = − 0.390 to 0.228, I2 = 0%). The fixed-effect model was used. MedCalc Statistical Software version 19.2.6 (MedCalc Software bv, Ostend, Belgium; https://www.medcalc.org; 2020).

Full size image

Discussion

This systematic review and meta-analysis demonstrated that nociception assessed through ANI had a weak and negative correlation with subjective self-reported measures of pain in conscious individuals, i.e., those undergoing medical procedures or submitted to painful experimental stimuli. However, good accuracy of the ANI as compared with NRS was observed in some studies30,31,41.

The current definition of pain by the International Association for the Study of Pain (IASP) (2020) is: "An unpleasant sensory and emotional experience associated with actual or potential tissue damage, or described in terms of such damage"43. For nociception, the concept is “The neural process of encoding noxious stimuli. Note: Consequences of encoding may be autonomic (e.g., elevated blood pressure) or behavioural (motor withdrawal reflex or more complex nocifensive behaviour). Pain sensation is not necessarily implied”44. These definitions strongly underline the influence of stress and emotions in modifying the correlation of nociception and pain assessment in awake individuals after surgical procedures or painful stimuli. The nature of pain is multifactorial45. Nociception depends on the trigger, and pain is clearly defined as a subjective experience46.

The satisfactory ANI's accuracy reported by Boselli et al.30 and Boselli et al.31 in the post-operative period should be interpreted with attention. Such findings may be clinically relevant because their results suggest that ANI can support practitioners to assess pain in the surgical setting and consequently allow a more reliable prescription of medications during and after surgical procedures. Thus, the use of ANI in the PACU may be potentially appropriate since inadequate management of pain in the postoperative period leads to undesirable results during the patient's recovery47.

On the other hand, the large variability in the results of correlation between ANI (objective measure) and NRS (subjective measure) in participants in the postoperative period of general anaesthesia proved by meta-analysis provides power to self-report measures of pain as the “gold standard” for deciding on analgesic complementation in conscious patients.

Our findings underline the influence of anaesthetic agents on ANI scores. However, there is no consensus on which anaesthetic agent would improve the correlation of ANI with subjective pain measures, possibly because the studies have compared different types of anaesthetics beside their relevant heterogeneity16,30,31,36,37,39,41. The anaesthetic agent and the drug consumption for pain control during the surgical procedure (remifentanil, fentanyl, sevoflurane, propofol, halogenated)36,37,38,39,40,41,42,47,48,49 or the technique (spinal, regional, or general anaesthesia)22,50 may influence the ANS regulation and alter the response of HRV to nociception.

Another vital point is whether the patients were conscious when answering about their pain. Factors such as the patient’s level of awareness and perception of the situation may also impact the final result of pain assessment39. Different surgical procedures36,37,39,42 and drugs' residual effect36 should also be taken into account in assessing pain. It is worth mentioning the negative correlation found in one included study, in which the patients exhibited spontaneous breathing during labour22.

As evidenced by studies included in the second subgroup analysis, the subjectivity of pain may impair the ANI assessment in individuals exposed to electrical stimuli32,33. The ANS is an essential regulator of heart rate. The transition in the time between two heartbeats is designated as HRV. It provides reliable information about the interaction of the sympathetic and parasympathetic nervous systems51. Parasympathetic activity is dominant in resting conditions, such as relaxation and sleep, whereas sympathetic activity increases heart rate and blood pressure in situations requiring energy expenditure. Their interaction is known as the sympathetic-vagal balance of the ANS. ANS balance and its conditions are reflected in HRV, which refers to short- and long-term heart rate variations due to several states, including emotional issues52,53,54,55,56. The sympathovagal balance is influenced by arousal, emotions, medications, and drugs used intraoperatively. Some of these factors, such as arousal and feelings, are more evident in conscious patients.

A weak correlation has made us reflect on the statistical analysis of the included studies whose authors had evaluated the agreement between two tests. Correlation analysis, the measure used by the majority, may be a powerless statistic. Therefore, in future research, accuracy measures or the Bland–Altman plot test should be used57.

According to a narrative review, measuring nociception in clinical settings is practically unfeasible. Still, it would be desirable for patients under general anaesthesia or unable to communicate to prevent acute postoperative pain. The authors conclude that no device has its usefulness justified in practice5. However, a recent systematised review described the validity of ANI for nociception assessment in anesthetised patients undergoing surgery and reported considerable changes in ANI values were found in response to nociceptive stimuli at different opioid concentrations and higher ANI values were noted during nociceptive stimuli58.

The studies included in our review used two Patient-Reported Outcome Measures (PROMS): VAS22,35,38,42 and NRS16,30,31,32,33,34,36,37,39,40,41. According to the FDA (Food and Drug Administration), a “PRO (Patient-Reported Outcome) is any report of the status of a patient's health condition that comes directly from the patient, without interpretation of the patient's response by a clinician or anyone else”. Therefore, the accuracy of ANI was assessed according to PROs59.

The present study has limitations. Although individuals in the included studies were conscious and reported their pain, the pain stimuli assessed were quite different. Women in labour, patients treated for burns, various elective surgeries, and patients who had received electrical stimuli may exhibit different responses to pain, taking into account the subjective pattern of pain and the influence of nociceptive stimuli. This study provided two subgroups of meta-analysis, but it is necessary to consider that, individually, some included studies30,31,41 demonstrated adequate accuracy and correlation of ANI with subjective measures of pain.

The strengths of this systematic review and meta-analysis include a comprehensive literature search through major electronic databases and grey literature, adherence to the PRISMA guidelines, and the inclusion of the highest number of studies published on this topic. Finally, data extraction, evaluation of outcomes, and the risk of bias assessment were all performed in duplicate. One limitation is the methodological heterogeneity among the included studies with different designs, precluding additional aggregated analyses. Among the included studies, only nine showed homogeneity. Due to differences regarding the setting where the studies had been conducted, a unique meta-analysis was unfeasible. Analyses were conducted in two subgroups; in one subgroup, data of only two studies were aggregated. According to the literature, quantitative analyses (even those with a few studies) represent a powerful tool to summarise data and increase sample size, allowing the researchers to obtain more reliable estimates. Nevertheless, the findings of those quantitative analyses should be interpreted with caution due to shortcomings of data that have been aggregated, such as studies' risk of bias, publication bias, and small-study effect60,61.

Conclusion

There was a weak correlation between the subjective pain scales and the Analgesia and Nociception Index, i.e., a part of pain self-report is explained by nociception assessed through ANI. Therefore, in the perioperative period, fully or partially conscious children or other individuals, who cannot self-report their pain, might benefit from using ANI during health procedures.