A randomized controlled trial demonstrates that a novel closed-loop propofol system performs better hypnosis control than manual administration

  • Thomas M. Hemmerling
  • Samer Charabati
  • Cedrick Zaouter
  • Carmelo Minardi
  • Pierre A. Mathieu
Reports of Original Investigations



The purpose of this randomized control trial was to determine the performance of a novel rule-based adaptive closed-loop system for propofol administration using the bispectral index (BIS®) and to compare the system’s performance with manual administration. The effectiveness of the closed-loop system to maintain BIS close to a target of 45 was determined and compared with manual administration.


After Institutional Review Board approval and written consent, 40 patients undergoing major surgery in a tertiary university hospital were allocated to two groups using computer-generated block randomization. In the Closed-loop group (n = 20), closed-loop control was used to maintain anesthesia at a target BIS of 45, and in the Control group (n = 20), propofol was administered manually to maintain the same BIS target. To evaluate each technique’s performance in maintaining a steady level of hypnosis, the BIS values obtained during the surgical procedure were stratified into four clinical performance categories relative to the target BIS: ≤ 10%, 11-20%, 21-30%, or > 30% defined as excellent, good, poor, or inadequate control of hypnosis, respectively. The controller performance was compared using Varvel’s controller performance indices. Data were compared using Fisher’s exact test and the Mann-Whitney U test, P < 0.05 showing statistical significance.


In the Closed-loop group, four females and 16 males (aged 54 ± 20 yr; weight 79 ± 7 kg) underwent anesthesia lasting 143 ± 57 min. During 55%, 29%, 9%, and 7% of the total anesthesia time, the system showed excellent, good, poor, and inadequate control, respectively. In the Control group, five females and 15 males (aged 59 ± 16 yr; weight 75 ± 13 kg) underwent anesthesia lasting 157 ± 81 min. Excellent, good, poor, and inadequate control were noted during 33%, 33%, 15%, and 19% of the total anesthesia time, respectively. In the Closed-loop group, excellent control of anesthesia occurred significantly more often (P < 0.0001), and poor and inadequate control occurred less often than in the Control group (P < 0.01). The median performance error and the median absolute performance error were significantly lower in the Closed-loop group compared with the Control group (-1.1 ± 5.3% vs -10.7 ± 13.1%; P = 0.004 and 9.1 ± 1.9% vs 15.7 ± 7.4%; P < 0.0001, respectively).


The closed-loop system for propofol administration showed better clinical and control system performance than manual administration of propofol. (Clinical Trials gov. NCT 01019746).

Une étude randomisée contrôlée démontre qu’un système innovant d’administration de propofol en circuit fermé permet un meilleur contrôle de l’hypnose qu’une administration manuelle



L’objectif de cette étude randomisée contrôlée était de déterminer l’efficacité d’un système innovant d’administration de propofol en circuit fermé, adaptatif et basé sur des règles. Pour ce faire, nous avons utilisé l’index bispectral (BIS®). Notre objectif visait aussi à comparer la performance de ce système à une administration manuelle. L’efficacité du système en circuit fermé dans le maintien d’un BIS avoisinant une cible de 45 a été déterminée et comparée à celle d’une administration manuelle.


Après avoir obtenu l’approbation du Comité d’éthique de la recherche et le consentement écrit des patients, 40 patients devant subir une chirurgie majeure dans un hôpital universitaire de soins tertiaires ont été randomisés en deux groupes à l’aide d’une méthode de randomisation informatique par bloc. Dans le groupe Circuit fermé (n = 20), le contrôle du circuit fermé a été utilisé pour maintenir l’anesthésie à un BIS cible de 45, alors que dans le groupe témoin, le propofol a été administré manuellement afin de maintenir la même cible de BIS. Afin d’évaluer l’efficacité de chacune des techniques pour le maintien d’un niveau stable d’hypnose, les valeurs de BIS obtenues pendant l’intervention chirurgicale ont été stratifiées en quatre catégories de performance clinique associées au BIS cible, soit: ≤ 10 %, 11-20 %, 21-30 %, ou > 30 %, soit un contrôle excellent, bon, médiocre ou inadapté de l’hypnose, respectivement. La performance du contrôleur a été comparée à l’aide des indices de performance du contrôleur de Varvel. Les données ont été comparées à l’aide du test de Fisher-Student et de test de U de Mann-Whitney, P < 0,05 démontrant une signification statistique.


Dans le groupe en circuit fermé, quatre femmes et 16 hommes (âgés de 54 ± 20 ans; poids 79 ± 7 kg) ont subi une anesthésie durant 143 ± 57 min. Pendant 55 %, 29 %, 9 %, et 7 % du temps total de l’anesthésie, le système a démontré un contrôle excellent, bon, médiocre et inadéquat, respectivement. Dans le groupe témoin, cinq femmes et 15 hommes (âgés de 59 ± 16 ans; poids 75 ± 13 kg) ont subi une anesthésie durant 157 ± 81 min. Un contrôle excellent, bon, médiocre et inadéquat a été noté pendant 33 %, 33 %, 15 %, et 19 % du temps total de l’anesthésie, respectivement. Dans le groupe en circuit fermé, un contrôle excellent de l’anesthésie a été observé significativement plus souvent (P < 0,0001), et un contrôle médiocre et inadéquat a été observé moins souvent que dans le groupe témoin (P < 0,01). L’erreur de performance médiane et l’erreur de performance médiane absolue étaient significativement plus basses dans le groupe en circuit fermé que dans le groupe témoin (−1,1 ± 5,3 % vs −10,7 ± 13,1 %; P = 0,004 et 9,1 ± 1,9 % vs 15,7 ± 7,4 %; P < 0,0001, respectivement).


Le système d’administration de propofol en circuit fermé a démontré une meilleure performance clinique et un meilleur système de contrôle que l’administration manuelle de propofol. (NCT 01019746).

The use of closed-loop systems might improve the quality of drug administration.1 These automated drug delivery systems use a controlled input variable, e.g., anesthetic depth, blood pressure, or the degree of neuromuscular blockade, to regulate the output (drug delivery rate) and maintain the preset target of the variable. By frequent or continuous sampling of the control variable and more frequent change of drug delivery rates than with manually delivered anesthesia, greater stability of the control variable might be achievable.2 Automated drug administration systems can maintain efficiency throughout a surgical procedure that is similar to manual administration yet free of the drawback of fatigue.3

The effectiveness of a closed-loop system depends strongly on the reliability of the input variable, i.e., the physiological signal to be controlled.4 While the ideal variable to measure the effect of hypnotic drugs is unknown,5 parameters derived from the analysis of the electroencephalogram (EEG) have emerged as objective and reliable measures of the depth of hypnosis for closed-loop systems.2,6 The bispectral index (BIS) is derived from processing the phase and frequency relations of the component frequencies of the EEG.7 The BIS is a dimensionless number ranging from 0 (isoelectric activity) to 100 (consciousness). A value from 40 to 60 is considered as representing an adequate state of hypnosis.7 Previous studies have shown that the BIS is well suited as a control variable for closed-loop systems.1,2,8-11 Also, studies have shown that closed-loop systems using the BIS outperform manual control.3,5,6

Relying on a single input signal, the controller can be misled by artefacts that can occur on the EEG signals, and this poses an inherent safety risk to the patient.6 To be effective, a closed-loop system using the BIS should minimize these artefacts. Two indicators, the signal quality index (SQI) and electromyography (EMG), are provided on commercially available BIS equipment to assess the reliability of a BIS value. The SQI reflects the percentage of artefact-free EEG data entering the BIS system over the preceding minute.7 Artefacts contaminating raw EEG and affecting BIS are usually high frequency signals related either to the use of surgical instruments and/or to EMG activity. By displaying an EMG signal, both sources of artefacts can be observed.7

The present investigation was designed to introduce a novel rule-based adaptive closed-loop system for propofol administration and to compare its performance in maintaining the level of hypnosis close to the BIS target with the performance of propofol manually administered by an anesthesiologist during elective abdominal, thoracic, urologic, and orthopedic surgery.

In order to determine clinical performance as the primary outcome, BIS values obtained during the surgical procedure were stratified into four categories of the target BIS: ≤ 10%, 11-20%, 21-30%, or > 30% defined as excellent, good, poor, or inadequate control, respectively. It was hypothesised that patients who received propofol with the novel closed-loop system would have significantly longer periods of time with the BIS close to the target of 45 compared with patients who received propofol by manual administration.

As a secondary outcome objective, the performance of the specific system controller was determined and compared using Varvel’s controller performance indices. The tertiary outcome was to assess the emergence time from tracheal extubation in the two groups.

The question remains whether it is possible to develop a rule-based closed-loop system capable of providing hypnosis with a better performance, i.e., longer periods of BIS values around a target of 45, than with manual control.


System specifications

The BIS Vista monitor (BIS Vista™) (Aspect Medical Systems, Inc., Newton, MA, USA) was used as the control variable, while a standard syringe pump, Graseby 3400, (Graseby Medical, Watford, UK) was the actuator. To close the loop, a notebook computer was used to implement the algorithm, provide the graphical interface, and control communication between the BIS Vista™ monitor and the syringe pump via RS-232 ports.

The user must insert the target BIS and the age and weight of the patient, and the system automatically controls the rate of propofol infusion in “automatic” mode. The target BIS can be changed according to clinical need during the course of the surgical procedure. The system acquires an update of the BIS, SQI, and EMG every five seconds and calculates a moving average of valid BIS every 20 sec. A valid BIS measurement is assumed when the SQI is > 40% and the EMG is < 40 dB. It is important to note that an empty EMG bar on the BIS monitor corresponds to an EMG level < 30 dB. If the resultant BIS average is 30 to 60, the algorithm calculates another average of valid BIS such that the resultant BIS is an average taken at each 40 sec interval, and a dose would be calculated at that time interval. If the first BIS average (20 sec) is 20 to 30, a minimal dose is administered, if < 20, the infusion stops, and if > 60, an automatic bolus is given (Fig. 1).
Fig. 1

Main algorithm of the system. Bispectral index (BIS) values are acquired every five seconds, and they are averaged every 20 sec. A valid BIS is assumed when the signal quality index (SQI) > 40% and the electromyographic index (EMG) < 40. The BIS error represents the difference between the BIS value and the target value. The BIS variation is the difference between the present and the previous value

A given dose is calculated on the basis of the following algorithm:

Dose = DosePrecedent • Km • Kv • Kh • Ka • Kw with the coefficients defined as follows: Km = proportional to the difference of the actual BIS to the target BIS; Kv = proportional to the variability of the BIS over the last time interval; Kh = proportional to the difference of the mean BIS to the target BIS over the last time interval; Ka = proportional to the age of the patient; and Kw = proportional to the weight of patient.

The system is self-adaptive in that the dose calculation is a function of the previous dose and the adjustment factors are proportional to: 1) BIS error, i.e., the difference between the target and the actual BIS value; 2) BIS variation, i.e., the difference between the actual and the previous BIS; and 3) BIS trend, i.e., the difference between the target and the average BIS values on a longer time interval. Patient characteristics (age and weight) determine the minimum and maximum allowable doses of propofol infusion and the bolus doses. These allowable doses are also a function of the BIS trend. For instance, measured BIS values above the target for the preceding five minutes will induce an increase of the maximal dose. During periods of artefacts (invalid BIS), the algorithm calculates and administers the average doses infused during the last period of time, and time is proportional to the duration of artefacts.

In “manual” mode, the anesthesiologist needs to change the propofol infusion dose to maintain the BIS as close as possible to a desired target. In both modes, the system records the input variable from the BIS monitor as well as output data at ten-second intervals for subsequent analysis.

The graphical user interface is designed using LabVIEW (National Instruments, Austin, TX, USA). It contains colour-coded graphic and numeric elements, push buttons, and graphs (Fig. 2). As recommended by the BIS manufacturer, the SQI and EMG are displayed along with the BIS, and they turn red when they fall outside their accepted boundaries, indicating artefacts. The BIS blinks during that period. The interface requires patient characteristics, which can be modified during start-up. It also displays the infusion dose, the corresponding rate, the average dose, and emergency bolus information.
Fig. 2

A user interface is developed with combined graphical-numerical elements. Three zones are created: On the left, system-relevant patient data are displayed (patient number, age, weight), followed by the pharmacological data of propofol; In the middle, the bispectral index (BIS) is displayed with different colour zones according to the degree of alert messaging for the anesthesiologist (red indicates a risk of awareness or awakening). In the lower middle zone, the user defines the desired target—infusion rates are displayed in both, mL·h−1 or μg·kg−1·min−1 in the form of “speedometers”; On the right, trends for the BIS and infusion rates are displayed. In the lower right, different bolus modes can be pre-defined (Color figure online)

Clinical protocol

This trial received Institutional Ethics Committee approval (McGill University Health Centre, Montreal General Hospital, Montreal, QC, Canada), and written informed consent was obtained from all patients. The study was conducted from August 1, 2007 to May 12, 2008 and included patients scheduled for elective abdominal, thoracic, urologic, and spine or upper extremity orthopedic surgery. Inclusion criteria were patients aged 18 to 85 years scheduled for surgery lasting more than 30 min, and exclusion criteria were the patients’ inability to provide informed consent or allergy to propofol. Patients were interviewed during the preoperative evaluation; all met the inclusion criteria for the investigation, and no patients refused to participate. Consequently, the patients were allocated to two groups using a computer-generated block randomization. In the Closed-loop group, propofol was administered using the closed-loop control system to maintain anesthesia at a target BIS of 45, and in the Control group, propofol was administered manually by an experienced anesthesiologist using a syringe pump in order to maintain a BIS as close as possible to the target of 45. Two research fellows were responsible for generating the allocation sequence, the enrolment, and the group assignment of the study patients.

During the clinical trial, a blinded investigator determined when patients met tracheal extubation criteria.

Induction was provided in all patients using fentanyl 2.5 μg·kg−1 and propofol 1.5 μg·kg−1, and endotracheal intubation was facilitated using rocuronium 0.6 mg·kg−1. Intermittent positive pressure ventilation in a breathing gas of 50% oxygen in air was maintained throughout surgery for a partial pressure of end-tidal carbon dioxide (PETCO2) of 35-40 mmHg. Analgesia was provided using a bolus of fentanyl 1.5 μg·kg−1 five minutes prior to incision and repetitive boluses of fentanyl 1.5 μg·kg−1 during surgery until the final 45 min of surgery. In cases where an epidural catheter was installed prior to surgery, epidural analgesia was used solely for postoperative pain treatment—the epidural catheter was bolused at the end of surgery with 2% lidocaine 4-8 mL, followed by a continuous infusion of 0.1% bupivacaine with fentanyl 3 μg·mL−1 at an infusion rate of 6-12 mL·hr−1. Neuromuscular blockade was maintained using boluses of rocuronium 0.15 mg·kg−1 according to train-of-four monitoring at the adductor pollicis muscle.

After manual propofol induction, closed-loop control was used to maintain anesthesia in the Closed-loop group at a target BIS of 45. In the Control group, propofol was administered manually by an experienced anesthesiologist using a syringe pump in order to maintain the BIS as close to the target of 45 as possible.

In both groups, propofol infusion was discontinued when the last stapler closure occurred, and emergence time was determined when tracheal extubation occurred.

Performance analysis

The clinical performance of the control of hypnosis was defined as the efficacy to maintain BIS as close to the target of 45 as possible, and the BIS values obtained during the surgical procedure were stratified into four categories of the target BIS: ≤ 10%, 11-20%, 21-30%, or > 30% defined as excellent, good, poor, or inadequate, respectively.

The precision of the system was assessed using the performance indices proposed by Varvel et al.12 Performance error (PE) was calculated as the difference between the actual and the target values. Bias or median performance error (MDPE) described whether the measured values were either above or below the target values and thus represented the direction (undershoot or overshoot) of the PE. Inaccuracy or median absolute performance error (MDAPE) described the size of the errors. Wobble measured the intra-individual variability in PE. Divergence reflected the evolution of the controller’s performance through time (worsening or improvement), i.e., it is the slope obtained from linear regression of the subject’s absolute PE against time and is expressed in units of percentage divergence per minute. A positive slope indicates a gradually widening gap between the measured and targeted values, whereas a negative value shows that the measured value tends to converge to the target values. In addition, we calculated another parameter, the global performance index (GPI), to integrate performance indices and thus obtain an overview of the global performance. Good performance—and hence a high value of GPI—is characterized by a high percentage of BIS within ± 10% of the target and low MDAPE, wobble, and percentage of inadequate control. Formulas are as follows:
$$ PE = {\frac{{BIS_{measured} - BIS_{target} }}{{BIS_{target} }}} \times 100 $$
$$ MDPE_{\text{i}} = Median\left\{ {PE_{ij,}\, j = 1, \ldots ,N_{i} } \right\} $$
$$ MDAPE_{\text{i}} = Median\left\{ {PE|_{ij,}\, j = 1, \ldots ,N_{i} } \right\} $$
$$ Wobble_{\text{i}} = Median\left\{ {PE_{ij,} - MDPE_{i} |,\, j = 1, \ldots ,N_{i} } \right\} $$
$$ GPI = {\frac{{\left( {\% {\text{Time}}\,{\text{BIS}}\, \pm 10\% \,{\text{Tartget}}} \right)^{2}}}{{MDAPE + Wobble + \% {\text{Time BIS}} \pm 30\% {\text{ Target}}}}} $$

The percentage of time was calculated where BIS was not valid due to artefacts (as defined above), and propofol consumption and emergence time were also calculated. Emergence time was defined as the time from the end of propofol infusion until tracheal extubation. For the Control group, the number of manual dose changes was recorded.

A “simulation” mode was used in preliminary tests to determine the appropriate sample size for the clinical protocol. In this mode, the input control variable was provided by a random number generator. Every five seconds, the random number generator issued a value in the same range as the BIS during maintenance of anesthesia (the boundaries were chosen for the purpose of each simulation). A simple rule managed the generation process, namely, if two consecutive propofol doses were low, a number > 50 would be chosen. Inversely, the number to be chosen would be < 40 if two consecutive doses were high. In all other cases, the number would be between 40 and 50. Signal quality index and EMG would be decreased or increased, respectively, to simulate presence of artefacts. The algorithms behind dose calculation and propofol administration were the same as in “automatic” mode.

Ten cases were run in simulation mode to verify correct functioning of the closed-loop system prior to its employment. The (simulated) subjects (aged 55 ± 8 yr; weight 77 ± 9 kg) received a mean infusion dose of propofol of 143 ± 13 μg·kg−1·min−1 for 196 ± 97 min. Performances indices yielded the following: MDPE = −0.83 ± 1.15%; MDAPE = 7.5 ± 1.15%; wobble = 7.5 ± 1.15%; and divergence = −0.0039 ± 0.0097 %·min−1. The BIS was ≤ 10% of the target (excellent control) during 67 ± 5% of the time and > 30% of the target (inadequate control) for 4 ± 2% of the time. An example of a 25-min simulation is shown in Fig. 3. The absence of BIS at min 15 indicated the presence of artefacts. The corresponding dose at this period was the average dose of the preceding five minutes.
Fig. 3

Trends of bispectral index (BIS), propofol dose, and the average dose over 25 min of a simulated case. At min 15, the BIS values were interrupted for three minutes. The dose administered at this period is the average dose of the preceding five minutes

The sample size was calculated to show a difference in the percentage of time during which excellent control is achieved in the Closed-loop group vs the Control group. For the Closed-loop group, 66% was taken as the estimated percentage of excellent control based on the results of the simulation study. The aim of our investigation was to detect a 50% proportion of time difference between both groups, assuming a standard error of 5% in the Closed-loop group. An alpha-error of 0.05 and a power of 0.8 were chosen, resulting in a sample size of 17 patients for each separate sample.13 To improve statistical power and take potential drop out into account, two groups of 20 patients each were planned.

Data are presented as mean ± standard deviation (SD). Parameters between the two groups were compared using the Mann-Whitney U test for continuous data and the Fisher’s exact test for categorical data. P < 0.05 was considered statistically significant. Statistical tests were performed using XLSTAT (Addinsoft™, New York, NY, USA).


Forty-two patients were interviewed and successively enrolled. Two of the patients were excluded due to equipment failure prior to the surgery, leaving 40 patients for randomization. In the Closed-loop group, the 16 men and four women were aged 54 ± 20 yr and weighed 79 ± 7 kg, which was not significantly different from the Control group with 15 men and five women who were aged 59 ± 16 yr and weighed 75 ± 13 kg (Table 1). The types of surgery were similar between the two groups (Table 2).
Table 1

Patient characteristics


Closed-loop group (n = 20)

Control group (n = 20)

Age (yr)

54 ± 20

59 ± 16

Weight (kg)

79 ± 7

75 ± 13

Sex (F/M)






Data are expressed as mean ± standard deviation (SD) or number; ASA = American Society of Anesthesiologists

Table 2

Types of surgery

Types of surgery

Types of dissections or reductions in the Closed-loop group

n = 20

Types of dissections or reductions in the Control group

n = 20

Abdominal surgery

Colectomy for Crohn’s disease

n = 2

Colectomy for Crohn’s disease

n = 1


Colectomy for tumour

n = 7

Colectomy for tumour

n = 2


Right hemicolectomy

n = 1


Sigmoid resection

n = 1

Anterior resection

n = 1

Thoracic surgery

Retroperitoneal lymph node

n = 1

Wedge resection

n = 2



n = 1


n = 1

Urologic surgery


n = 3


n = 1



n = 2



n = 2



n = 1

Orthopedic surgery

Radial fracture

n = 1

Elbow sarcomectomy

n = 1


Maxillary fracture

n = 1

Radial fracture

n = 2


Spinal fusion

n = 3

Spinal fusion

n = 3

Patients in the Closed-loop group underwent anesthesia for 143 ± 57 min and received propofol 120 ± 28 μg·kg−1·min−1, which was not significantly different from the Control group that underwent anesthesia for 157 ± 81 min and received propofol 115 ± 27 μg·kg−1·min−1. All patients received fentanyl 2.5 μg·kg−1 at induction. In the Closed-loop group, a mean total of fentanyl 4.8 ± 0.2 μg·kg−1 was administered during surgery vs a mean total of fentanyl 5.1 ± 0.4 μg·kg−1 in the Control group, which was not statistically different. The last dose of fentanyl was administered at 51 ± 5 min in the Closed-loop group and at 54 ± 6 min in the Control group.

Our system maintained anesthesia closer to a pre-defined target than manual administration of propofol. In the Closed-loop group, excellent control of anesthesia occurred significantly more often (P < 0.0001) and inadequate control occurred less often than in the Control group (P = 0.001) (Fig. 4). The period of artefacts was similar in both groups: Closed-loop group 2.25 ± 2.49 % vs Control group 2.48 ± 2.14 % of the anesthesia duration (P = 0.525).
Fig. 4

The performance of the controller as a percentage of the total anesthesia time. The performance is defined as excellent, good, poor, or inadequate, when the BIS is ≤ 10%, 11-20%, 21-30%, or > 30% of the target BIS, respectively; *P < 0.05

The controller indices showed the superiority of the Closed-loop group in providing a BIS target closer to 45 for a longer period of time (Table 3).
Table 3

Controller performance


Closed-loop group ( = 20)

Control group (n = 20)

P value*

MDPE (%)

−1.1 ± 5.3

−10.7 ± 13.1



9.1 ± 1.9

15.7 ± 7.4

< 0.0001

Wobble (%)

8.5 ± 1.9

12.5 ± 8.5


Divergence (%·min−1)

0.0012 ± 0.0644

−0.1097 ± 0.2228



155 ± 148

40 ± 39

< 0.0001

Data are expressed as mean ± standard deviation (SD); * P < 0.05, Mann-Whitney U test. MDPE = median performance error; MDAPE = median absolute performance error; GPI = global performance index

Emergence time was significantly shorter in the Closed-loop group at 9.0 ± 3.7 min vs 12 ± 3.3 min in the Control group (P = 0.03).

The BIS values and the propofol infusion doses from the end of induction to discontinuation of propofol infusion are presented for both groups in Figs 5 and 6. Fig. 7 presents the typical best-case (highest GPI) and worst-case (lowest GPI) performances for both groups.
Fig. 5

Bispectral index (BIS) values during the maintenance of anesthesia. (A) All individual data are shown (recorded every ten seconds and averaged every minute for the figure). (B) Median BIS values (thick line) are presented with 10th and 90th percentile (thin lines)

Fig. 6

Propofol doses during the maintenance of anesthesia. All individual data are shown (recorded every ten seconds and averaged every minute for the figure). (B) Median BIS values (thick line) are presented with 25th and 75th percentile (thin lines)

Fig. 7

Bispectral index (BIS) for best and worst cases (recorded every ten seconds). (A) Best patient in the Closed-loop group (global performance index (GPI) = 694). The male patient underwent a total colectomy and received an average dose of propofol of 165 ± 57 μg·kg−1·min−1. The BIS was within 10% of the target for 79% of the time; median performance error (MDPE) = 2.2; median absolute performance error (MDAPE) = 4.4; Wobble = 4.4. (B) Best patient in the Control group (GPI = 148). A male patient underwent open partial nephrectomy and received an average dose of propofol of 144 ± 41 μg·kg−1·min−1. The BIS was within 10% of the target for 54% of the time; MDPE = −2.2; MDAPE = 8.8; Wobble = 8.8. (C) Worst patient in the Closed-loop group (GPI = 71). The male patient underwent sigmoidectomy and received an average dose of propofol of 116 ± 63 μg·kg−1·min−1. The BIS was within 10% of the target for 49% of the time; MDPE = −6.6; MDAPE = 11.1; Wobble = 8.8. (D) Worst patient in the Control group (GPI = 4). A female patient underwent lumbar posterior discectomy and received an average dose of propofol of 76 ± 23 μg·kg−1·min−1. The BIS was within 10% of the target for 16% of the time; MDPE = −20; MDAPE = 22.2; Wobble = 8.8


The current study demonstrates that the administration of propofol in closed-loop feedback control guided by BIS using an expert-based control system is feasible and provides adequate hypnosis. Compared with manual administration of propofol, this novel closed-loop control system provided significantly longer periods of excellent control of depth of hypnosis and significantly shorter periods of inadequate control. The performance indices of the system were significantly better than manual control. Emergence from anesthesia seemed faster when the automated system was used.

Our closed-loop system demonstrated better control to maintain a target BIS of 45 than manual administration of propofol. We defined four performance attributes depending on the zone where BIS was situated with respect to the target. We believe that excellent control and inadequatecontrol were the most clinically significant of these attributes. In fact, excellent control indicated that the BIS was within ± 10% of its set-point. In terms of BIS values, excellent control for a typical target of 45 occurs when the BIS range is 41-49, whereas inadequate control occurs when the BIS is > 58 or < 32, with an imminent high risk of either awakening or overly profound anesthesia.

Other research groups expressed the performance of the controller in terms of the percentage of time that the BIS was in a specific range. Liu et al.3 and Puri et al.5 calculated the percentage of time when the BIS was in the 40-60 range, which is equivalent to the percentage of time when the BIS was within ± 20% of the target. Their results showed 89 ± 9% and 87.3 ± 9%, respectively. Our results are very similar—in accordance with their method of calculation, we could state, in the present study, that the BIS stayed within 20% of the target during 84 ± 7% of the study period (sum of the periods of excellent and good control).

We assessed the precision of the system using Varvel’s performance indices,12 conceived originally to evaluate the predictive performance of computer-controlled infusion pumps but used widely to assess the performance of closed-loop systems.1-3,5,6,10 The median of the PE, also known as the bias or MDPE, is a signed value representing the direction (overshoot or undershoot) of the PE. We found a MDPE of −1.1%, indicating that the median BIS was slightly below the target—a median BIS of 44 in all patients indicates a good clinical performance of our system compared with an overshoot median BIS of 35 in the Control group. The size of the error or precision is indicated by the MDAPE. An MDAPE of 9.1% indicates that 50% of the measured BIS was within 9.1% of the target BIS. This result is equivalent to the percentage time of excellent control, i.e., the BIS was within 10% of the target for 55% of the time. The performance parameters for the present study compare favourably with those obtained in previous studies by Puri et al.5 (MDPE = 1%; MDAPE = 9.45%; wobble = 8.4%) and Liu et al.3 (MDPE = −3.32%; MDAPE = 9.94%; wobble = 8.10%). However, as yet, there are no defined limits of these performance parameters for control systems in the human body. If closed-loop systems are to be introduced into clinical practice, there must be a defined consensus regarding acceptable clinical limits for any anesthesia delivery system.

Liu et al. have demonstrated that MDPE, MDAPE, and wobble, if taken alone, may mislead the interpretation of the system evaluation. Hence, they calculated a global score that should be as low as the MDAPE, exhibit a low wobble, and show the BIS in the 40-60 range for a high percentage of time.3 In an attempt to provide a score that integrates performance parameters, we present a GPI that is inversely proportional to the MDAPE, wobble, and the percentage of time of inadequate control; thus, to indicate a better performance, the GPI should be high. The GPI is best used when comparing cases within each group or when comparing both groups. According to the GPI, the Closed-loop group performed significantly better than the Control group.

Our closed-loop system includes several algorithms that limit the influence of artefacts. As in previous studies,1,5,8 records where no BIS was generated because of poor signal quality (SQI < 15) were excluded from analysis. While the system was running, we defined an invalid BIS when the corresponding SQI was < 40 and the EMG was > 40, which constituted 2.25 ± 2.5% of the anesthesia time. There are no agreed on SQI and EMG threshold values characterizing the acceptability of the BIS. For their isoflurane closed-loop systems, Gentillini et al.14 and Locher et al.6 assumed a valid BIS for SQI > 20 and SQI > 30, respectively. Liu et al. accepted BIS values with SQI > 50.3 None of the previous studies of closed-loop applications using the BIS included the EMG signal as a means to indicate artefacts affecting the validity of the BIS. However, it is well documented that the presence of an EMG signal (displayed in the bar graph, range of 30-55) may be accompanied by a falsely high BIS.7,14 Electromyographic activity may result from the use of external devices (warming systems, surgical instruments, circulatory assist systems, cardiac pacing devices), from inadequate neuromuscular blockade,7,15 or simply from patient shivering. During defined periods of artefacts or in the absence of BIS measurements, model-based controllers often “open the loop” and rely solely on the pharmacokinetic model,1,14 or they do not modify the propofol target concentration until the BIS is valid yet again.3

The tracheae of the patients in the Closed-loop group were extubated in less than ten minutes after propofol infusion was discontinued, demonstrating a significant difference from the patients in the Control group—the anesthesiologist responsible for assessing the tracheal extubation criteria was blinded to group assignment. However, emergence is influenced not only by the hypnotic component of anesthesia but also by the analgesic component. In this study, we tried to control intraoperative opioid administration by limiting analgesia to fentanyl alone and by giving guidelines for dose and timing of its administration. In addition, even when following strict extubation criteria, assessment of readiness for extubation remains subjective and differences should not be over-estimated.

The influence of opioids on the BIS is controversial; whereas some studies have found an influence of opioids, such as remifentanil, on the BIS,16 this could not be confirmed in other studies.17 Mi et al.18 investigated the interaction between fentanyl and propofol on the BIS; despite increasing plasma concentrations of fentanyl, the BIS values and recovery times did not change in patients undergoing abdominal surgery with propofol-fentanyl anesthesia. Glass et al.15 criticized a study in which the performance of a closed-loop system1 of propofol was determined while analgesia was provided with a high dose of continuous remifentanil infusion resulting in minimal propofol changes. Glass et al. proposed to determine the performance of such a closed-loop system in situations where the dose of the analgesic was much lower. In the present study, we used a moderate dose of fentanyl (in our institution, a total mean dose of fentanyl 500 μg for spine surgery of 2.5 hr duration would be considered a sufficient moderate dose for a 75 kg patient), and two doses were given at definite time points. The total fentanyl dose as well as the interval between the last administered dose and emergence were very similar in the two groups. Opioid administration for closed-loop studies must be equivalent and non-biased for both groups while, at the same time, reflecting current practice. We believe these criteria were achieved in the present study.

The anesthesiologist for the Control group was not involved in the research of the closed-loop algorithms. However, a bias cannot be excluded, as there was the clear objective to focus on maintaining the BIS as close to the target as possible, and the number of propofol changes per hour in the Control group – significantly higher than clinical routine – hints to the intention of maintaining a stable BIS. Many similar studies have cited the Hawthorne effect as an indication that human performance exceeds routine performance when tested against automated system performance.19

In conclusion, we present a closed-loop system that includes a novel user-interface for automatic administration of propofol and performs better than conventional manual control.



The authors would like to thank all the members of the Department of Anesthesia at the Montreal General Hospital, in particular Dr. D. Bracco, for their collaboration during this study.

Conflicts of interest

None declared.


  1. 1.
    Struys MM, De Smet T, Versichelen LF, Van De Velde S, Van den Broecke R, Mortier EP. Comparison of closed-loop controlled administration of propofol using bispectral index as the controlled variable versus “standard practice” controlled administration. Anesthesiology 2001; 95: 6-17.CrossRefPubMedGoogle Scholar
  2. 2.
    Absalom AR, Sutcliffe N, Kenny GN. Closed-loop control of anesthesia using bispectral index: performance assessment in patients undergoing major orthopedic surgery under combined general and regional anesthesia. Anesthesiology 2002; 96: 67-73.CrossRefPubMedGoogle Scholar
  3. 3.
    Liu N, Chazot T, Genty A, et al. Titration of propofol for anesthetic induction and maintenance guided by the bispectral index: closed-loop versus manual control: a prospective, randomized, multicenter study. Anesthesiology 2006; 104: 686-95.CrossRefPubMedGoogle Scholar
  4. 4.
    Struys MM, Mortier EP, De Smet T. Closed loops in anaesthesia. Best Pract Res Clin Anaesthesiol 2006; 20: 211-20.CrossRefPubMedGoogle Scholar
  5. 5.
    Puri GD, Kumar B, Aveek J. Closed-loop anaesthesia delivery system (CLADS) using bispectral index: a performance assessment study. Anaesth Intensive Care 2007; 35: 357-62.PubMedGoogle Scholar
  6. 6.
    Locher S, Stadler KS, Boehlen T, et al. A new closed-loop control system for isoflurane using bispectral index outperforms manual control. Anesthesiology 2004; 101: 591-602.CrossRefPubMedGoogle Scholar
  7. 7.
    Kelly SD. Monitoring Level of Consciousness During Anesthesia and Sedation. Newton, MA, USA: Aspect Medical Systems, Inc.; 2003.Google Scholar
  8. 8.
    Mortier E, Struys M, De Smet T, Versichelen L, Rolly G. Closed-loop controlled administration of propofol using bispectral analysis. Anaesthesia 1998; 53: 749-54.CrossRefPubMedGoogle Scholar
  9. 9.
    Morley A, Derrick J, Mainland P, Lee BB, Short TG. Closed loop control of anaesthesia: an assessment of the bispectral index as the target of control. Anaesthesia 2000; 55: 953-9.CrossRefPubMedGoogle Scholar
  10. 10.
    Leslie K, Absalom A, Kenny GN. Closed loop control of sedation for colonoscopy using the bispectral index. Anaesthesia 2002; 57: 693-7.CrossRefPubMedGoogle Scholar
  11. 11.
    Liu N, Chazot T, Trillat B, et al. Feasibility of closed-loop titration of propofol guided by the bispectral index for general anaesthesia induction: a prospective randomized study. Eur J Anaesthesiol 2006; 23: 465-9.CrossRefPubMedGoogle Scholar
  12. 12.
    Varvel JR, Donoho DL, Shafer SL. Measuring the predictive performance of computer-controlled infusion pumps. J Pharmacokinet Biopharm 1992; 20: 63-94.CrossRefPubMedGoogle Scholar
  13. 13.
    Machin D, Campbell M, Fayers P, Pinol A. Sample Size Tables for Clinical Studies. 2nd ed. Oxford: Blackwell Science Inc.; 1997.Google Scholar
  14. 14.
    Gentilini A, Rossoni-Gerosa M, Frei CW, et al. Modeling and closed-loop control of hypnosis by means of bispectral index (BIS) with isoflurane. IEEE Trans Biomed Eng 2001; 48: 874-89.CrossRefPubMedGoogle Scholar
  15. 15.
    Glass PS, Rampil IJ. Automated anesthesia: fact or fantasy? Anesthesiology 2001; 95: 1-2.CrossRefPubMedGoogle Scholar
  16. 16.
    Strachan AN, Edwards ND. Randomized placebo-controlled trial to assess the effect of remifentanil and propofol on bispectral index and sedation. Br J Anaesth 2000; 84: 489-90.PubMedGoogle Scholar
  17. 17.
    Guignard B, Menigaux C, Dupont X, Fletcher D, Chauvin M. The effect of remifentanil on the bispectral index change and hemodynamic responses after orotracheal intubation. Anesth Analg 2000; 90: 161-7.CrossRefPubMedGoogle Scholar
  18. 18.
    Mi W, Sakai T, Kudo T, Kudo M, Matsuki A. The interaction between fentanyl and propofol during emergence from anesthesia: monitoring with the EEG-bispectral index. J Clin Anesth 2003; 15: 103-7.CrossRefPubMedGoogle Scholar
  19. 19.
    Franke RH, Kaul J. The Hawthorne experiments: first statistical interpretation. Am Sociol Rev 1978; 43: 623-43.CrossRefGoogle Scholar

Copyright information

© Canadian Anesthesiologists’ Society 2010

Authors and Affiliations

  • Thomas M. Hemmerling
    • 1
    • 2
    • 4
  • Samer Charabati
    • 4
  • Cedrick Zaouter
    • 3
  • Carmelo Minardi
    • 3
  • Pierre A. Mathieu
    • 4
  1. 1.Department of AnaesthesiologyMontreal General HospitalMontrealCanada
  2. 2.Department of AnaesthesiologyMcGill UniversityMontrealCanada
  3. 3.Intelligent Technology in Anesthesia Group (ITAG)McGill UniversityMontrealCanada
  4. 4.Institute of Biomedical EngineeringUniversity of MontrealMontrealCanada

Personalised recommendations