A randomized controlled trial demonstrates that a novel closed-loop propofol system performs better hypnosis control than manual administration
The purpose of this randomized control trial was to determine the performance of a novel rule-based adaptive closed-loop system for propofol administration using the bispectral index (BIS®) and to compare the system’s performance with manual administration. The effectiveness of the closed-loop system to maintain BIS close to a target of 45 was determined and compared with manual administration.
After Institutional Review Board approval and written consent, 40 patients undergoing major surgery in a tertiary university hospital were allocated to two groups using computer-generated block randomization. In the Closed-loop group (n = 20), closed-loop control was used to maintain anesthesia at a target BIS of 45, and in the Control group (n = 20), propofol was administered manually to maintain the same BIS target. To evaluate each technique’s performance in maintaining a steady level of hypnosis, the BIS values obtained during the surgical procedure were stratified into four clinical performance categories relative to the target BIS: ≤ 10%, 11-20%, 21-30%, or > 30% defined as excellent, good, poor, or inadequate control of hypnosis, respectively. The controller performance was compared using Varvel’s controller performance indices. Data were compared using Fisher’s exact test and the Mann-Whitney U test, P < 0.05 showing statistical significance.
In the Closed-loop group, four females and 16 males (aged 54 ± 20 yr; weight 79 ± 7 kg) underwent anesthesia lasting 143 ± 57 min. During 55%, 29%, 9%, and 7% of the total anesthesia time, the system showed excellent, good, poor, and inadequate control, respectively. In the Control group, five females and 15 males (aged 59 ± 16 yr; weight 75 ± 13 kg) underwent anesthesia lasting 157 ± 81 min. Excellent, good, poor, and inadequate control were noted during 33%, 33%, 15%, and 19% of the total anesthesia time, respectively. In the Closed-loop group, excellent control of anesthesia occurred significantly more often (P < 0.0001), and poor and inadequate control occurred less often than in the Control group (P < 0.01). The median performance error and the median absolute performance error were significantly lower in the Closed-loop group compared with the Control group (-1.1 ± 5.3% vs -10.7 ± 13.1%; P = 0.004 and 9.1 ± 1.9% vs 15.7 ± 7.4%; P < 0.0001, respectively).
The closed-loop system for propofol administration showed better clinical and control system performance than manual administration of propofol. (Clinical Trials gov. NCT 01019746).
Une étude randomisée contrôlée démontre qu’un système innovant d’administration de propofol en circuit fermé permet un meilleur contrôle de l’hypnose qu’une administration manuelle
L’objectif de cette étude randomisée contrôlée était de déterminer l’efficacité d’un système innovant d’administration de propofol en circuit fermé, adaptatif et basé sur des règles. Pour ce faire, nous avons utilisé l’index bispectral (BIS®). Notre objectif visait aussi à comparer la performance de ce système à une administration manuelle. L’efficacité du système en circuit fermé dans le maintien d’un BIS avoisinant une cible de 45 a été déterminée et comparée à celle d’une administration manuelle.
Après avoir obtenu l’approbation du Comité d’éthique de la recherche et le consentement écrit des patients, 40 patients devant subir une chirurgie majeure dans un hôpital universitaire de soins tertiaires ont été randomisés en deux groupes à l’aide d’une méthode de randomisation informatique par bloc. Dans le groupe Circuit fermé (n = 20), le contrôle du circuit fermé a été utilisé pour maintenir l’anesthésie à un BIS cible de 45, alors que dans le groupe témoin, le propofol a été administré manuellement afin de maintenir la même cible de BIS. Afin d’évaluer l’efficacité de chacune des techniques pour le maintien d’un niveau stable d’hypnose, les valeurs de BIS obtenues pendant l’intervention chirurgicale ont été stratifiées en quatre catégories de performance clinique associées au BIS cible, soit: ≤ 10 %, 11-20 %, 21-30 %, ou > 30 %, soit un contrôle excellent, bon, médiocre ou inadapté de l’hypnose, respectivement. La performance du contrôleur a été comparée à l’aide des indices de performance du contrôleur de Varvel. Les données ont été comparées à l’aide du test de Fisher-Student et de test de U de Mann-Whitney, P < 0,05 démontrant une signification statistique.
Dans le groupe en circuit fermé, quatre femmes et 16 hommes (âgés de 54 ± 20 ans; poids 79 ± 7 kg) ont subi une anesthésie durant 143 ± 57 min. Pendant 55 %, 29 %, 9 %, et 7 % du temps total de l’anesthésie, le système a démontré un contrôle excellent, bon, médiocre et inadéquat, respectivement. Dans le groupe témoin, cinq femmes et 15 hommes (âgés de 59 ± 16 ans; poids 75 ± 13 kg) ont subi une anesthésie durant 157 ± 81 min. Un contrôle excellent, bon, médiocre et inadéquat a été noté pendant 33 %, 33 %, 15 %, et 19 % du temps total de l’anesthésie, respectivement. Dans le groupe en circuit fermé, un contrôle excellent de l’anesthésie a été observé significativement plus souvent (P < 0,0001), et un contrôle médiocre et inadéquat a été observé moins souvent que dans le groupe témoin (P < 0,01). L’erreur de performance médiane et l’erreur de performance médiane absolue étaient significativement plus basses dans le groupe en circuit fermé que dans le groupe témoin (−1,1 ± 5,3 % vs −10,7 ± 13,1 %; P = 0,004 et 9,1 ± 1,9 % vs 15,7 ± 7,4 %; P < 0,0001, respectivement).
Le système d’administration de propofol en circuit fermé a démontré une meilleure performance clinique et un meilleur système de contrôle que l’administration manuelle de propofol. (NCT 01019746).
The use of closed-loop systems might improve the quality of drug administration.1 These automated drug delivery systems use a controlled input variable, e.g., anesthetic depth, blood pressure, or the degree of neuromuscular blockade, to regulate the output (drug delivery rate) and maintain the preset target of the variable. By frequent or continuous sampling of the control variable and more frequent change of drug delivery rates than with manually delivered anesthesia, greater stability of the control variable might be achievable.2 Automated drug administration systems can maintain efficiency throughout a surgical procedure that is similar to manual administration yet free of the drawback of fatigue.3
The effectiveness of a closed-loop system depends strongly on the reliability of the input variable, i.e., the physiological signal to be controlled.4 While the ideal variable to measure the effect of hypnotic drugs is unknown,5 parameters derived from the analysis of the electroencephalogram (EEG) have emerged as objective and reliable measures of the depth of hypnosis for closed-loop systems.2,6 The bispectral index (BIS) is derived from processing the phase and frequency relations of the component frequencies of the EEG.7 The BIS is a dimensionless number ranging from 0 (isoelectric activity) to 100 (consciousness). A value from 40 to 60 is considered as representing an adequate state of hypnosis.7 Previous studies have shown that the BIS is well suited as a control variable for closed-loop systems.1,2,8-11 Also, studies have shown that closed-loop systems using the BIS outperform manual control.3,5,6
Relying on a single input signal, the controller can be misled by artefacts that can occur on the EEG signals, and this poses an inherent safety risk to the patient.6 To be effective, a closed-loop system using the BIS should minimize these artefacts. Two indicators, the signal quality index (SQI) and electromyography (EMG), are provided on commercially available BIS equipment to assess the reliability of a BIS value. The SQI reflects the percentage of artefact-free EEG data entering the BIS system over the preceding minute.7 Artefacts contaminating raw EEG and affecting BIS are usually high frequency signals related either to the use of surgical instruments and/or to EMG activity. By displaying an EMG signal, both sources of artefacts can be observed.7
The present investigation was designed to introduce a novel rule-based adaptive closed-loop system for propofol administration and to compare its performance in maintaining the level of hypnosis close to the BIS target with the performance of propofol manually administered by an anesthesiologist during elective abdominal, thoracic, urologic, and orthopedic surgery.
In order to determine clinical performance as the primary outcome, BIS values obtained during the surgical procedure were stratified into four categories of the target BIS: ≤ 10%, 11-20%, 21-30%, or > 30% defined as excellent, good, poor, or inadequate control, respectively. It was hypothesised that patients who received propofol with the novel closed-loop system would have significantly longer periods of time with the BIS close to the target of 45 compared with patients who received propofol by manual administration.
As a secondary outcome objective, the performance of the specific system controller was determined and compared using Varvel’s controller performance indices. The tertiary outcome was to assess the emergence time from tracheal extubation in the two groups.
The question remains whether it is possible to develop a rule-based closed-loop system capable of providing hypnosis with a better performance, i.e., longer periods of BIS values around a target of 45, than with manual control.
The BIS Vista monitor (BIS Vista™) (Aspect Medical Systems, Inc., Newton, MA, USA) was used as the control variable, while a standard syringe pump, Graseby 3400, (Graseby Medical, Watford, UK) was the actuator. To close the loop, a notebook computer was used to implement the algorithm, provide the graphical interface, and control communication between the BIS Vista™ monitor and the syringe pump via RS-232 ports.
A given dose is calculated on the basis of the following algorithm:
Dose = DosePrecedent • Km • Kv • Kh • Ka • Kw with the coefficients defined as follows: Km = proportional to the difference of the actual BIS to the target BIS; Kv = proportional to the variability of the BIS over the last time interval; Kh = proportional to the difference of the mean BIS to the target BIS over the last time interval; Ka = proportional to the age of the patient; and Kw = proportional to the weight of patient.
The system is self-adaptive in that the dose calculation is a function of the previous dose and the adjustment factors are proportional to: 1) BIS error, i.e., the difference between the target and the actual BIS value; 2) BIS variation, i.e., the difference between the actual and the previous BIS; and 3) BIS trend, i.e., the difference between the target and the average BIS values on a longer time interval. Patient characteristics (age and weight) determine the minimum and maximum allowable doses of propofol infusion and the bolus doses. These allowable doses are also a function of the BIS trend. For instance, measured BIS values above the target for the preceding five minutes will induce an increase of the maximal dose. During periods of artefacts (invalid BIS), the algorithm calculates and administers the average doses infused during the last period of time, and time is proportional to the duration of artefacts.
In “manual” mode, the anesthesiologist needs to change the propofol infusion dose to maintain the BIS as close as possible to a desired target. In both modes, the system records the input variable from the BIS monitor as well as output data at ten-second intervals for subsequent analysis.
This trial received Institutional Ethics Committee approval (McGill University Health Centre, Montreal General Hospital, Montreal, QC, Canada), and written informed consent was obtained from all patients. The study was conducted from August 1, 2007 to May 12, 2008 and included patients scheduled for elective abdominal, thoracic, urologic, and spine or upper extremity orthopedic surgery. Inclusion criteria were patients aged 18 to 85 years scheduled for surgery lasting more than 30 min, and exclusion criteria were the patients’ inability to provide informed consent or allergy to propofol. Patients were interviewed during the preoperative evaluation; all met the inclusion criteria for the investigation, and no patients refused to participate. Consequently, the patients were allocated to two groups using a computer-generated block randomization. In the Closed-loop group, propofol was administered using the closed-loop control system to maintain anesthesia at a target BIS of 45, and in the Control group, propofol was administered manually by an experienced anesthesiologist using a syringe pump in order to maintain a BIS as close as possible to the target of 45. Two research fellows were responsible for generating the allocation sequence, the enrolment, and the group assignment of the study patients.
During the clinical trial, a blinded investigator determined when patients met tracheal extubation criteria.
Induction was provided in all patients using fentanyl 2.5 μg·kg−1 and propofol 1.5 μg·kg−1, and endotracheal intubation was facilitated using rocuronium 0.6 mg·kg−1. Intermittent positive pressure ventilation in a breathing gas of 50% oxygen in air was maintained throughout surgery for a partial pressure of end-tidal carbon dioxide (PETCO2) of 35-40 mmHg. Analgesia was provided using a bolus of fentanyl 1.5 μg·kg−1 five minutes prior to incision and repetitive boluses of fentanyl 1.5 μg·kg−1 during surgery until the final 45 min of surgery. In cases where an epidural catheter was installed prior to surgery, epidural analgesia was used solely for postoperative pain treatment—the epidural catheter was bolused at the end of surgery with 2% lidocaine 4-8 mL, followed by a continuous infusion of 0.1% bupivacaine with fentanyl 3 μg·mL−1 at an infusion rate of 6-12 mL·hr−1. Neuromuscular blockade was maintained using boluses of rocuronium 0.15 mg·kg−1 according to train-of-four monitoring at the adductor pollicis muscle.
After manual propofol induction, closed-loop control was used to maintain anesthesia in the Closed-loop group at a target BIS of 45. In the Control group, propofol was administered manually by an experienced anesthesiologist using a syringe pump in order to maintain the BIS as close to the target of 45 as possible.
In both groups, propofol infusion was discontinued when the last stapler closure occurred, and emergence time was determined when tracheal extubation occurred.
The clinical performance of the control of hypnosis was defined as the efficacy to maintain BIS as close to the target of 45 as possible, and the BIS values obtained during the surgical procedure were stratified into four categories of the target BIS: ≤ 10%, 11-20%, 21-30%, or > 30% defined as excellent, good, poor, or inadequate, respectively.
The percentage of time was calculated where BIS was not valid due to artefacts (as defined above), and propofol consumption and emergence time were also calculated. Emergence time was defined as the time from the end of propofol infusion until tracheal extubation. For the Control group, the number of manual dose changes was recorded.
A “simulation” mode was used in preliminary tests to determine the appropriate sample size for the clinical protocol. In this mode, the input control variable was provided by a random number generator. Every five seconds, the random number generator issued a value in the same range as the BIS during maintenance of anesthesia (the boundaries were chosen for the purpose of each simulation). A simple rule managed the generation process, namely, if two consecutive propofol doses were low, a number > 50 would be chosen. Inversely, the number to be chosen would be < 40 if two consecutive doses were high. In all other cases, the number would be between 40 and 50. Signal quality index and EMG would be decreased or increased, respectively, to simulate presence of artefacts. The algorithms behind dose calculation and propofol administration were the same as in “automatic” mode.
The sample size was calculated to show a difference in the percentage of time during which excellent control is achieved in the Closed-loop group vs the Control group. For the Closed-loop group, 66% was taken as the estimated percentage of excellent control based on the results of the simulation study. The aim of our investigation was to detect a 50% proportion of time difference between both groups, assuming a standard error of 5% in the Closed-loop group. An alpha-error of 0.05 and a power of 0.8 were chosen, resulting in a sample size of 17 patients for each separate sample.13 To improve statistical power and take potential drop out into account, two groups of 20 patients each were planned.
Data are presented as mean ± standard deviation (SD). Parameters between the two groups were compared using the Mann-Whitney U test for continuous data and the Fisher’s exact test for categorical data. P < 0.05 was considered statistically significant. Statistical tests were performed using XLSTAT (Addinsoft™, New York, NY, USA).
Closed-loop group (n = 20)
Control group (n = 20)
54 ± 20
59 ± 16
79 ± 7
75 ± 13
Types of surgery
Types of surgery
Types of dissections or reductions in the Closed-loop group
n = 20
Types of dissections or reductions in the Control group
n = 20
Colectomy for Crohn’s disease
n = 2
Colectomy for Crohn’s disease
n = 1
Colectomy for tumour
n = 7
Colectomy for tumour
n = 2
n = 1
n = 1
n = 1
Retroperitoneal lymph node
n = 1
n = 2
n = 1
n = 1
n = 3
n = 1
n = 2
n = 2
n = 1
n = 1
n = 1
n = 1
n = 2
n = 3
n = 3
Patients in the Closed-loop group underwent anesthesia for 143 ± 57 min and received propofol 120 ± 28 μg·kg−1·min−1, which was not significantly different from the Control group that underwent anesthesia for 157 ± 81 min and received propofol 115 ± 27 μg·kg−1·min−1. All patients received fentanyl 2.5 μg·kg−1 at induction. In the Closed-loop group, a mean total of fentanyl 4.8 ± 0.2 μg·kg−1 was administered during surgery vs a mean total of fentanyl 5.1 ± 0.4 μg·kg−1 in the Control group, which was not statistically different. The last dose of fentanyl was administered at 51 ± 5 min in the Closed-loop group and at 54 ± 6 min in the Control group.
Closed-loop group (n = 20)
Control group (n = 20)
−1.1 ± 5.3
−10.7 ± 13.1
9.1 ± 1.9
15.7 ± 7.4
8.5 ± 1.9
12.5 ± 8.5
0.0012 ± 0.0644
−0.1097 ± 0.2228
155 ± 148
40 ± 39
Emergence time was significantly shorter in the Closed-loop group at 9.0 ± 3.7 min vs 12 ± 3.3 min in the Control group (P = 0.03).
The current study demonstrates that the administration of propofol in closed-loop feedback control guided by BIS using an expert-based control system is feasible and provides adequate hypnosis. Compared with manual administration of propofol, this novel closed-loop control system provided significantly longer periods of excellent control of depth of hypnosis and significantly shorter periods of inadequate control. The performance indices of the system were significantly better than manual control. Emergence from anesthesia seemed faster when the automated system was used.
Our closed-loop system demonstrated better control to maintain a target BIS of 45 than manual administration of propofol. We defined four performance attributes depending on the zone where BIS was situated with respect to the target. We believe that excellent control and inadequatecontrol were the most clinically significant of these attributes. In fact, excellent control indicated that the BIS was within ± 10% of its set-point. In terms of BIS values, excellent control for a typical target of 45 occurs when the BIS range is 41-49, whereas inadequate control occurs when the BIS is > 58 or < 32, with an imminent high risk of either awakening or overly profound anesthesia.
Other research groups expressed the performance of the controller in terms of the percentage of time that the BIS was in a specific range. Liu et al.3 and Puri et al.5 calculated the percentage of time when the BIS was in the 40-60 range, which is equivalent to the percentage of time when the BIS was within ± 20% of the target. Their results showed 89 ± 9% and 87.3 ± 9%, respectively. Our results are very similar—in accordance with their method of calculation, we could state, in the present study, that the BIS stayed within 20% of the target during 84 ± 7% of the study period (sum of the periods of excellent and good control).
We assessed the precision of the system using Varvel’s performance indices,12 conceived originally to evaluate the predictive performance of computer-controlled infusion pumps but used widely to assess the performance of closed-loop systems.1-3,5,6,10 The median of the PE, also known as the bias or MDPE, is a signed value representing the direction (overshoot or undershoot) of the PE. We found a MDPE of −1.1%, indicating that the median BIS was slightly below the target—a median BIS of 44 in all patients indicates a good clinical performance of our system compared with an overshoot median BIS of 35 in the Control group. The size of the error or precision is indicated by the MDAPE. An MDAPE of 9.1% indicates that 50% of the measured BIS was within 9.1% of the target BIS. This result is equivalent to the percentage time of excellent control, i.e., the BIS was within 10% of the target for 55% of the time. The performance parameters for the present study compare favourably with those obtained in previous studies by Puri et al.5 (MDPE = 1%; MDAPE = 9.45%; wobble = 8.4%) and Liu et al.3 (MDPE = −3.32%; MDAPE = 9.94%; wobble = 8.10%). However, as yet, there are no defined limits of these performance parameters for control systems in the human body. If closed-loop systems are to be introduced into clinical practice, there must be a defined consensus regarding acceptable clinical limits for any anesthesia delivery system.
Liu et al. have demonstrated that MDPE, MDAPE, and wobble, if taken alone, may mislead the interpretation of the system evaluation. Hence, they calculated a global score that should be as low as the MDAPE, exhibit a low wobble, and show the BIS in the 40-60 range for a high percentage of time.3 In an attempt to provide a score that integrates performance parameters, we present a GPI that is inversely proportional to the MDAPE, wobble, and the percentage of time of inadequate control; thus, to indicate a better performance, the GPI should be high. The GPI is best used when comparing cases within each group or when comparing both groups. According to the GPI, the Closed-loop group performed significantly better than the Control group.
Our closed-loop system includes several algorithms that limit the influence of artefacts. As in previous studies,1,5,8 records where no BIS was generated because of poor signal quality (SQI < 15) were excluded from analysis. While the system was running, we defined an invalid BIS when the corresponding SQI was < 40 and the EMG was > 40, which constituted 2.25 ± 2.5% of the anesthesia time. There are no agreed on SQI and EMG threshold values characterizing the acceptability of the BIS. For their isoflurane closed-loop systems, Gentillini et al.14 and Locher et al.6 assumed a valid BIS for SQI > 20 and SQI > 30, respectively. Liu et al. accepted BIS values with SQI > 50.3 None of the previous studies of closed-loop applications using the BIS included the EMG signal as a means to indicate artefacts affecting the validity of the BIS. However, it is well documented that the presence of an EMG signal (displayed in the bar graph, range of 30-55) may be accompanied by a falsely high BIS.7,14 Electromyographic activity may result from the use of external devices (warming systems, surgical instruments, circulatory assist systems, cardiac pacing devices), from inadequate neuromuscular blockade,7,15 or simply from patient shivering. During defined periods of artefacts or in the absence of BIS measurements, model-based controllers often “open the loop” and rely solely on the pharmacokinetic model,1,14 or they do not modify the propofol target concentration until the BIS is valid yet again.3
The tracheae of the patients in the Closed-loop group were extubated in less than ten minutes after propofol infusion was discontinued, demonstrating a significant difference from the patients in the Control group—the anesthesiologist responsible for assessing the tracheal extubation criteria was blinded to group assignment. However, emergence is influenced not only by the hypnotic component of anesthesia but also by the analgesic component. In this study, we tried to control intraoperative opioid administration by limiting analgesia to fentanyl alone and by giving guidelines for dose and timing of its administration. In addition, even when following strict extubation criteria, assessment of readiness for extubation remains subjective and differences should not be over-estimated.
The influence of opioids on the BIS is controversial; whereas some studies have found an influence of opioids, such as remifentanil, on the BIS,16 this could not be confirmed in other studies.17 Mi et al.18 investigated the interaction between fentanyl and propofol on the BIS; despite increasing plasma concentrations of fentanyl, the BIS values and recovery times did not change in patients undergoing abdominal surgery with propofol-fentanyl anesthesia. Glass et al.15 criticized a study in which the performance of a closed-loop system1 of propofol was determined while analgesia was provided with a high dose of continuous remifentanil infusion resulting in minimal propofol changes. Glass et al. proposed to determine the performance of such a closed-loop system in situations where the dose of the analgesic was much lower. In the present study, we used a moderate dose of fentanyl (in our institution, a total mean dose of fentanyl 500 μg for spine surgery of 2.5 hr duration would be considered a sufficient moderate dose for a 75 kg patient), and two doses were given at definite time points. The total fentanyl dose as well as the interval between the last administered dose and emergence were very similar in the two groups. Opioid administration for closed-loop studies must be equivalent and non-biased for both groups while, at the same time, reflecting current practice. We believe these criteria were achieved in the present study.
The anesthesiologist for the Control group was not involved in the research of the closed-loop algorithms. However, a bias cannot be excluded, as there was the clear objective to focus on maintaining the BIS as close to the target as possible, and the number of propofol changes per hour in the Control group – significantly higher than clinical routine – hints to the intention of maintaining a stable BIS. Many similar studies have cited the Hawthorne effect as an indication that human performance exceeds routine performance when tested against automated system performance.19
In conclusion, we present a closed-loop system that includes a novel user-interface for automatic administration of propofol and performs better than conventional manual control.
The authors would like to thank all the members of the Department of Anesthesia at the Montreal General Hospital, in particular Dr. D. Bracco, for their collaboration during this study.
Conflicts of interest
- 1.Struys MM, De Smet T, Versichelen LF, Van De Velde S, Van den Broecke R, Mortier EP. Comparison of closed-loop controlled administration of propofol using bispectral index as the controlled variable versus “standard practice” controlled administration. Anesthesiology 2001; 95: 6-17.CrossRefPubMedGoogle Scholar
- 7.Kelly SD. Monitoring Level of Consciousness During Anesthesia and Sedation. Newton, MA, USA: Aspect Medical Systems, Inc.; 2003.Google Scholar
- 13.Machin D, Campbell M, Fayers P, Pinol A. Sample Size Tables for Clinical Studies. 2nd ed. Oxford: Blackwell Science Inc.; 1997.Google Scholar