Introduction

Changes in techniques and patient populations make it more challenging than ever to manage anesthesia in a fast, simple and safe way. A wide spectrum of pharmacological actions (analgesia, hypnosis, and suppression of somatic and autonomic responses to noxious stimuli) are needed to control the general anesthetic state [1]. During general anesthesia, opiates are classically applied to manage the nociception–antinoception balance, and short acting hypnotics are widely used to titrate the hypnotic component of anesthesia.

The ultimate goal when administering a particular dose of an anesthetic or analgesic drug is to obtain the desired clinical effect, for which a specific therapeutic concentration of the drug at the site of action (=the receptor) is required. At the same moment, the clinician wants to avoid side effects in order to reach the highest standards of care.

In the world of control engineering, dealing with the behavior of dynamic systems, closed-loop control can be defined as the management of single- or multiple-output variables of a system following a specific target value, whereby a controller adapts the system's inputs to reach and maintain a desired effect on the output. The goal of a closed-loop controller is to calculate solutions for an accurate corrective action from the controller that result in system stability, that is, the system will hold the set point and not oscillate around it [2]. Extrapolated to the world of anesthesia, this means that any action to maintain a specific pharmacological effect can be called a closed-loop control. Even a manual titration of drug infusion by a clinician is a closed-loop action as the clinician continuously monitors and adapts his/her actions. However, the clinician serves as the “human controller” in the loop, and as a consequence the control actions are intermittent and irregular in time [3•]. Computer-based closed-loop administration requires various system components : (1) a controlled variable representative for the targeted therapeutic effect; (2) a clinically relevant set-point or target value for this variable; (3) a control actuator, which is, in this case, the infusion pump driving the drug; (4) a system, in this case a patient; (5) an accurate, stable control algorithm [4]. Various closed-loop systems exist to control various steps in the dose-response relationship. When a system is able to control a specific set dose or drug concentration, it can be called a pharmacokinetic closed-loop controller. When a specific therapeutic effect is targeted, the control system is defined as a pharmacodynamic closed-loop system.

Pharmacokinetic Closed-Loop Systems in Anesthesia

Expired concentration of various drugs can be measured continuously. For the inhaled anesthetics such as desflurane, sevoflurane, and isoflurane, this can be done clinically using the spectrometric gas analyzers available in most of the anesthesia monitors. Using these inhaled anesthetic concentrations, a closed-loop system can be applied targeting a specific inspired or end-tidal concentration. Over the last decades, various experimental control systems were developed [5, 6]. More recently, a commercial closed-circuit anesthesia ventilator (Zeus®, Dräger Medical, Lübeck, Germany) was released. This machine is able to target the end-tidal concentrations of inhaled anesthetics and to control the fresh gas flow using closed-loop technology [7].

Recently, experimental devices measuring exhaled concentration from intravenously given propofol have been tested using proton transfer mass spectrometry and headspace solid-phase microextraction coupled with gas chromatography-mass spectrometry (HS-SPME-GC-MS) [8] or ion mobility spectrometry coupled to a multicapillary column for pre-separation (MCC-IMS). Grossherr et al. [9] used gas chromatography mass spectrometry to measure exhaled propofol. Until today, propofol exhaled concentration measures are still experimental and no closed-loop systems have been developed using this concentration as the controlled variable.

Pharmacokinetic-Dynamic Closed-Loop Systems

The Controlled Variable

The accuracy of closed-loop controlled drug administration strongly depends on the robustness and reliability of the controlled variable. Various drug actions can be measured directly using physiological measures. Examples are heart rate, respiration, blood pressure, and neuromuscular blockade. Various research groups have used direct measures to steer the feedback from controlled administration of cardiovascular drugs, anesthetics and muscular blocking agents. Some of these systems, such as the IVAC Titrator (Carefusion, San Diego, CA, USA) controlling nitroprusside using blood pressure as controlled variable, had been commercially available in the past but were discontinued. In contrast to these direct measures, surrogate measures are required to observe the hypnotic component of anesthesia or the balance between nociception and antinociception. However, they have to be interpreted with caution as a full correlation with all levels of drug effect might be missing.

Various surrogate measures have been studied to observe hypnotic drug effects. Both the spontaneous and evoked electro-encephalogram (EEG) have been proven to accurately measure cerebral hypnotic drug effects and be good candidate controlled variables closed-loop of hypnosis. Early closed-loop systems used computerized EEG derivatives like spectral edge frequency (SEF) and median frequency (MEF) [10]. More recently, the bispectral index (BIS©, Covidien, Boulder, CO, USA) has been used as controlled variable in multiple studies. BIS has been designed using multivariate statistical analysis, to combine multiple EEG features, including higher-order spectra and phase correlations between spectra into a more accurate indicator. Aside from BIS, State and Response Entropy (M-entropy, GE Healthcare, Helsinki, Finland), two spectral entropy parameters based on the irregularity in the EEG have been used recently to measure hypnotic drug effect during closed-loop administration [11–13•]. One research group has tested auditory evoked potentials, more specific the mid-latency auditory evoked potential (MLAEP) as controlled variable for closed-loop control of propofol administration [14, 15]. One of the major challenges when using a surrogate measure is the delay in the system, which adds complexity to the controller. All currently available indices have different time lags to react to a change in the level of anesthesia. Pilge and coworkers compared the time lag in three commercially available computerized EEG systems by using an artificial EEG signal and found time variable delays between 14 and 155 s [16].

Closed-loop administration of analgesics have been challenging because controlling the balance between nociception and antinociception is a difficult task. Liu et al. [13•] have used EEG to co-administer propofol and opioids, however, the inclusion of a real “analgesia index” in closed-loop is still lacking. Using the difference between response (RE) and state (SE) entropy derived from the EEG as a measure of frontal electromyographic (FEMG) activity, Mathews and coworkers found that remifentanil may be delivered using an algorithm that maintains the difference between RE and SE between the upper and lower boundary condition [17], however, this has not been incorporated in a closed-loop system. More recently, the same authors found that the Composite Variability Index (CVI), based on the variability in BIS and FEMG activity, might be useful to predict movement during anesthesia, which can be controlled by administering analgesics [18••].

The Target Value

The target value or set-point is the value set by the clinician and will be approached as closely as possible during the maintenance of anesthesia. A clinically adequate individual target is essential for the accuracy of the closed-loop system. Two types of set-points can be used: (1) set-points that are based on population mean data, or (2) individual data measured at the start or just before the control period. The latter type could be expected to more closely correspond to the clinical needs during the course of a surgical procedure [19].

Control Methods

Multiple control methods have been used to guide closed-loop anesthesia. Although on–off control was used in the early days, severe oscillation due to the complexity of drug behavior have limited this approach [20–24]. Proportional-Integral-Derivative (PID) control has also been used in various anesthesia-related closed-loop applications.

A PID controller is based on a straightforward mathematical derivative of the observed error, and can be written as:

$$ {\text{dR}}/{\text{dt }} = \, K_{\text{P}} * d\left( {\text{err}} \right)/{\text{dt }} + {\text{ err}}/K_{\text{I}} + \, K_{\text{D}} * d^{2} \left( {\text{err}} \right)/d^{2} t $$
(formula 1)

with err, being the error between the target and the observed value, causing a response R in the actuator. The constants K P, K I, K D are tuned by calculations from models of the system, by computer simulations, or derived from trials using tuning rules [25]. PID controllers have been applied under well-controlled situation after fine-tuning of the constants [26–29]. However, the use of a general PID controller to control the complex dose–response relationship when administering drugs (with R being the administration rate), could be slow in establishing control and may cause oscillations.

The complexity of the dose-response relationship can be decreased by implementing knowledge of the pharmacokinetic-dynamic behavior of the drug. Incorporating pharmacokinetic-dynamic models will enable a controller to use a specific plasma or effect-site concentration as R (see formula 1) instead of a dosing rate. The use of plasma or effect-site targeted drug administration is well understood and will lower the order of complexity of the resulting system [30]. Nowadays, the use of modern powerful microprocessors may allow better control through the incorporation of more sophisticated models describing the dose–response relationship, or by reverting to other control algorithms like MPC or fuzzy logic [31, 32].

Closed-loop control might benefit from adaptive fine-tuning. Various theoretical approaches can be used to adapt the control parameters toward the behavior characteristics of a specific individual. Examples are state estimation, mixed-effects pharmacokinetic or dynamic modeling using Bayesian estimation [33•, 34], Kalman filtering [35], fuzzy logic [31, 36] or other engineering techniques such as neural network applications [37] and reinforced learning [38, 39]. Bayesian optimization, as proposed by Sheiner and coworkers [40], individualizes the pharmacodynamic relationship by combining individual information with the knowledge of an a priori probability density function containing the statistical properties of the parameter to be estimated [41]. The Bayesian method starts from a standard, population-based response model providing the prior distribution of parameter values. These values are adjusted to reflect the patient’s own parameters over time, based on the observed response of the individual patient under varying circumstances [42]. The Kalman filter will apply a recursive method to calculate numbers for a given dose-response relationship for the specific patient, for example to individualize the constant for plasma-effect site equilibration [43, 44]. Fuzzy logic control is based on fuzzy set theory as proposed by Zadeh [45] in the sixties. An approach of model adaptation based on fuzzy logic was proposed by Kern and coworkers [46]. Recently, Moore and Doufas used reinforcement learning to control propofol closed-loop administration in a simulated environment. Reinforcement learning (RL) is an intelligent control method with an excellent record of success in difficult robotic control tasks. This method is based on a mathematically structured framework for goal-directed decision-making and is suitable for biological applications that are characterized by an inherent time delay between control actions and effects [38, 39].

Examples of Closed-Loop Drug Administration

Human-operated drug administration can be considered as a form of closed-loop control. Various patient-controlled drug administration systems are available to deliver individualized dosages of analgesics and hypnotics. This technology, called patient controlled analgesia (PCA) or patient-controlled sedation (PCS) systems, offers the possibility to set a continuous background infusion and to allow patients to administer themselves additional top-up dosages. PCA without background infusion is also used. Post-operative PCA usage has been described for analgesics such as morphine, piritramide, fantanyl, tramadol, and others [47–51]. In a systematic review, Walder and colleagues showed that the some evidence exists that in the postoperative pain setting, PCA with opioids, compared with conventional opioid treatment, improves analgesia and decreases the risk of pulmonary complications, and that patients prefer this option [52]. The development of advanced drug delivery devices that offer lock-out times and total amount of drug delivery per time is required. Strict hospital guidelines are required avoid drug overdose, causing potentially life-threatening side effects such as respiratory depression [53]. Recently, the feasibility of the use of intravenously delivered remifentanil during labor by PCA, under strict observation, has been demonstrated [54–59]. Large randomized controlled trials are required to prove that this technique can become an alternative for epidural analgesia during labor [60•]. PCS might offer comfort and anxiolysis during therapeutic procedures such as endoscopy. Various experimental devices for propofol PCS administration have been designed in the past [61–63], some of these even adding TCI technology into the system to optimize drug delivery [64–67]. Doufas and colleagues tested an automatic response test to optimize propofol administration for conscious sedation and showed that failure to respond to automated responsiveness monitoring precedes potentially serious adverse effects of sedation such as loss of responsiveness, and that the monitor was not susceptible to false-positive responses [68–70•]. An enlarged commercial version of this device, called Sedasys® (Ethicon Endo-Surgery, Cincinnati, OH, USA) has been tested in two studies. The system incorporates the automated responsiveness monitoring and in-built capnography and pulse oximetry. If responses to stimuli are inadequate, the increase in infusion rate is limited; whereas if apnea or hemoglobin oxygen desaturation is detected, then the infusion is stopped and additional oxygen administered. After a successful feasibility study [71], the system was then used in a large randomized study of sedation during upper gastrointestinal endoscopy and colonoscopy, and found to be associated with a reduced incidence of adverse events compared with standard care (5.8 vs 8.7 % respectively) [72•].

When feeding a continuously measured drug effect back to the drug delivery device, fully automated drug delivery will be enabled. The clinician has only to set a specific target value to be reached and maintained. Early peri-operative closed-loop technology focussed on the administration of cardiovascular drugs and neuromuscular blocking agents. For example, Kenny and coworkers [73] successfully evaluated closed-loop control of arterial pressure using a mixture of trimetaphan camsylate and sodium nitroprusside during controlled hypotensive anesthesia for local resections of intraocular melanoma. In the 1980s–1990s, various researchers tested the accuracy of closed-loop controlled administration of atracurium [36, 74] and vecuronium [75]. Due to the commercialisation of the reversal drug suggamadex, interest in closed-loop administration of muscle relaxants has declined.

Early developed closed-loop system used hemodynamic alterations to guide hypnotic drug delivery, due to a lack of availability of reliable cerebral drug effect monitors, such as EEG [21, 22, 31]. The commercialisation of more accurate cerebral effect measures enabled the development of EEG based closed-loop delivery of hypnotic-anesthetic drugs. Schwilden and Schüttler pioneered closed-loop administration of methohexital [76], propofol [77] and even alfentanil [78] using the EEG median frequency as controlled variable and an adaptive controller based on pharmacological principles, whereby adaptation in the pharmacokinetic part was applied. Kenny and coworkers developed a proportional-integral (PI) based closed-loop system for propofol administration using a mid-latency evoked potential derived index (AEPindex) as the controlled variable. The input variable was the predicted plasma concentration of propofol. As explained earlier in this chapter, concepts of TCI were applied to decrease the complexity of the PI controller. Accurate control was observed in most of the patients [14].

Most of the recent developed hypnotic closed-loop systems are guided by the EEG-derived bispectral index or BIS. Sakai and colleagues used an early version of the BIS and concluded that their closed-loop system provided intraoperative hemodynamic stability and a prompt recovery from sedative–hypnotic effects of propofol [79]. Absalom and Kenny proposed propofol closed-loop delivery using BIS and PID control of a plasma controlled TCI system and revealed acceptable control during major orthopedic surgery [30] and during sedation [80]. Although these researchers improved the performance of their control system by implementing more advanced effect-site targeted TCI, they also concluded that the PID controller might still face some stability problems. A similar BIS-guided propofol closed-loop system using a control system described as a proportional-derivative (PD) control to steer a specific effect-site concentration was developed by Liu and coworkers. Their system was tested during anesthesia and resulted in lower propofol consumption, longer induction time but with better hemodynamic stability, less excessive anesthetic levels (BIS < 40), similar hemodynamic stability and faster recovery [81, 82]. More recently, Liu tested a more advanced version of their BIS-guided system, now claiming full PID control, for closed-loop co-administration of both propofol and remifentanil. On top of the PID controller, the authors describe a rule-based algorithm that determines when to change the propofol or remifentanil targets. This system showed a better overall performance compared to manual administration in a multicenter study [83•]. A similar approach was used with an alternative EEG-derived index, spectral entropy [13•]. Recently, Liu fully explained his control system in a response to a letter to the editor of Anesthesiology. In this letter, Looke criticized the approach by Liu and coworkers by stating that the system should not be described as a PID controller, but as an empirically derived expert system controller. This author stressed on the importance of using a multidisciplinary team approach, including both medical and control engineering professionals when developing closed-loop systems, and stressed also on the application of simulation studies before entering into clinical practice. In their reply, Liu and coworkers revealed in great detail the structure of their algorithm and PID properties are certainly recognized. They also debated on the utility of simulation studies during closed-loop development [84•].

As said previously, automated drug delivery could benefit from the implementation of principles of pharmacokinetics and dynamics in the control algorithm. A BIS-guided, patient-individualized, model-based adaptive control system was developed and tested by Struys and De Smet during sedation and general anesthesia [85, 86]. The controller is based on a pharmacodynamic model represented by a sigmoidal E max model. Initially, the initial patient-specific pharmacodynamic profile is calculated automatically during induction by correlating all predicted effect-site concentrations with the corresponding BIS value. During closed-loop control, the controller minimizes the difference between measured and desired effect by using the pharmacodynamic model [86]. The authors compared this closed-loop controlled administration of propofol versus standard practice controlled administration and concluded that closed-loop control was clinically acceptable. In an accompanying editorial, Glass and Rampil [87] questioned whether the controller could become clinically acceptable outside the study population, because all subsequent adjustments were based on a static pharmacodynamic curve and only BIS of 50 was targeted in combination with continuous infusion of propofol or spinal anesthesia. As it might be considered unethical to stress the controller under extreme conditions outside the ranges of good clinical practice, a simulation study was undertaken and proved that even under extreme conditions, the model-based controller exhibited no behaviour problems and performed better than a previously published PID controlled closed-loop system [88]. As the original controller assumed a drug-free patient and used a fixed pharmacodynamic curve individualized during induction, De Smet and Struys included Bayesian optimization (as explained earlier in this article) into the original model-based controller to overcome these shortcomings (Fig. 1). They estimated the optimal modelling weights for this Bayesian-based BIS guided closed-loop system for propofol administration in a large simulation study, hereby stating that this system was safe enough to be introduced into clinical testing [34]. This accuracy and clinical feasibility was tested in a clinical study guiding propofol administration during anesthesia for ambulatory gynaecological procedures. They proved that the system performed better that BIS-guided, effect compartment controlled propofol administration titrated by the anaesthesiologist [33•].

Fig. 1
figure 1

Flow chart of the closed-loop system. The straight lines represent the closed-loop control system. At each time the required effect-site concentration is calculated by the controller. This value is sent to an additional algorithm taking the safety limits into account. The result of these calculations is the required effect-site concentration sent to the TCI algorithm, which steers a pump injecting propofol to the patient. The measured BIS is used as the input of the closed-loop controller. The dotted lines represent the Bayesian sigmoid E max model estimator. The estimator receives a priori information from the population sigmoid E max model, the optimal Bayesian variances for control and the patient measured BIS values (from De Smet T, Struys MM, Greenwald S, Mortier EP, Shafer SL: “Estimation of optimal modeling weights for a Bayesian-based closed-loop system for propofol administration using the bispectral index as a controlled variable: a simulation study. Anesth Analg 2007; 105:1629–1638, with permission)

Full size image

Closed-loop systems have been developed for a BIS-guided isoflurane administration using a controller with a cascade structure, originally described by Gentilini et al. [89, 90]. Recently, Moore and Doufas designed a closed-loop system using an intelligent system technique called “reinforcement learning”, for achieving optimal control in systems characterized by noise, nonlinearity, inherent time delays, and uncertainty [38, 39]. This system has not been tested in clinical practice.

Most of the above mentioned controllers are without a predictive forecasting horizon. Recently, Ionescu et al. described a model-based predictive control strategy, which is more advanced and might form the basis for future innovative engineering. Specifically, these authors were able to show that adaptive model update of the patient’s pharmaco-dynamic profile is possible, allowing detection of significant variable time-delays in the patient’s response to drug infusion and presence of artefacts in ICU [91].

Conclusion

Closed-loop drug delivery in anesthesia is certainly feasible. Unfortunately, most of the described closed-loop technologies have been used under well-controlled research conditions. The challenge is now to prove fully the safety and utility for its adaption into clinical practice [92]. Finally, clinicians will have to determine whether or not, adaptive, intelligent computer systems with dual, interacting, closed-loop systems will facilitate better control and improve outcome [3•].