Automatic Regulation of Hemodynamic Variables in Acute Heart Failure by a Multiple Adaptive Predictive Controller Based on Neural Networks

Abstract

Automated drug-delivery systems that can tolerate various responses to therapeutic agents have been required to control hemodynamic variables with heart failure. This study is intended to evaluate the control performance of a multiple adaptive predictive control based on neural networks (MAPC_NN) to regulate the unexpected responses to therapeutic agents of cardiac output (CO) and mean arterial pressure (MAP) in cases of heart failure. The NN components in the MAPC_NN learned nonlinear responses of CO and MAP determined by hemodynamics of dogs with heart failure. The MAPC_NN performed ideal control against unexpected (1) drug interactions, (2) acute disturbances, and (3) time-variant responses of hemodynamics [average errors between setpoints (+35 ml kg⁻¹ min⁻¹ in CO and ±0 mmHg in MAP) and observed responses; 6.4, 3.7, and 4.2 ml kg⁻¹ min⁻¹ in CO and 1.6, 1.4, and 2.7 mmHg (10.5, 20.8, and 15.3 mmHg without a vasodilator) in MAP] during 120-min closed-loop control. The MAPC_NN could also regulate the hemodynamics in actual heart failure of a dog. Robust regulation of hemodynamics by the MAPC_NN was attributable to the ability of on-line adaptation to adopt various responses and predictive control using the NN. Results demonstrate the feasibility of applying the MAPC_NN using a simple NN to clinical situations.

INTRODUCTION

Hemodynamic variables in a critical care patient with heart failure must be monitored during and after cardiac surgery. The hemodynamic conditions must then be regulated using infusion of several drugs. In particular, cardiac output (CO) and mean arterial pressure (MAP) are primary target variables because increased CO is required with suppressing myocardial oxygen consumption and MAP must be kept at the lowest level that can adequately maintain coronary circulation with decreasing systemic vascular resistance (SVR).14 Combined infusion of an inotropic agent such as dobutamine (DBT) or dopamine and a vasodilator such as sodium nitroprusside (SNP) or nitroglycerin has proven effective for patients with heart failure to regulate hemodynamics.12,13,25 Inotropic agents increase the force and velocity of cardiac muscle contraction and result in enhancing CO. Vasodilators reduce SVR and result in the decrease of the afterload of the heart, which thereby decreases MAP and increases stroke volume secondarily.3,14

Multivariable automated drug-delivery systems have been developed to help busy physicians or anesthesiologists use several drugs in many critical tasks to regulate the various hemodynamics that occur during heart failure.11,21,22,24,28,32,33 In simulation and animal studies undertaken during early system development, adaptive controllers demonstrated the feasibility of application of a multivariable drug-delivery system to simultaneously control CO and MAP using a combination of an inotropic agent and a vasodilator.11,24,28 In subsequent stages, multiple models and adaptive predictive controllers have been developed to adequately adjust the hemodynamic parameters in the presence of nonlinear physiological responses5,32 and drug interactions.21,22,33 Fuzzy controls have also shown optimal performance in regulation of multivariable hemodynamics with heart failure.6,7,31 However, patients with heart failure may have nonlinear and time-variant responses under unexpected variability to drugs and drug interactions with disturbances. Under such unknown conditions, the model-based controllers might require the preparation of numerous linear models to describe the various heart-failure patient responses to drugs,21,22,33 and the fuzzy controllers might require many rules based on the expert knowledge and heuristics of physicians or anesthesiologists for drug therapy of heart failure.6

The controllers that can adjust adaptively to unexpected responses with heart failure are required in actual clinical situations. Because neural networks (NN) might be one of the simple tools used for nonlinear and time-variant system responses in the presence of the unknown response variability and interactions with exogenous perturbation,15,27 the application of the NN to drug-delivery systems has been desired.8,17 To our knowledge, the multivariable controller using the NN for hemodynamic variables with heart failure has not been tested whereas the NN controllers have engendered the MAP controls such as post-operative hypertension1 and during acute hypotension9. Therefore, this study is intended to develop a multiple adaptive predictive control based on NN (MAPC_NN) for hemodynamic variables and to evaluate its control performance under unexpected responses to drugs during CO and MAP regulation using SNP and DBT in heart failure. To be assured of a control algorithm before animal experiments, the MAPC_NN was tested under largely unexpected (1) drug interactions, (2) acute disturbances, and (3) time-variant hemodynamic responses which can be freely operated. Finally, the performance of the MAPC_NN was tested by actual hemodynamics of a canine left heart failure.

MATERIALS AND METHODS

This section fundamentally includes three parts: (i) ‘Modeling of Pharmacological Response’, which explains the method for the development of a computer response model that allows testing of the MAPC_NN, (ii) ‘Development of Controller’, and (iii) ‘Evaluation of Controller’, which is the actual test of the MAPC_NN. In addition, the animal study was divided into two parts. First, to model the pharmacological response in (i), animal experiments were performed using dogs with heart failure. Five of eleven dogs were used for every drug infusion. Then, for additional validation of the developed controller after computer simulations, an animal experiment was performed using one dog with heart failure.

Modeling of Pharmacological Response

To produce the response models to therapeutic agents in acute heart failure, the following animal study, which conformed to the Guide for the Care and Use of Laboratory Animals published by the US National Institutes of Health, was performed. Microsphere embolization of the left main coronary artery induced acute ischemic heart failure in dogs (n = 5, 26–32 kg) that were anesthetized with pentobarbital sodium and ventilated artificially. A double-lumen catheter was introduced into the right femoral vein for administration of pharmaceutical agents using a computer-controllable infusion pump (CFV-3200; Nihon Kohden, Tokyo, Japan). An in-line electromagnetic flow probe (MFV-2100; Nihon Kohden) was used to measure CO; MAP was measured through a fluid-filled catheter and a pressure transducer (DX-200; Nihon Kohden). The CO and MAP were digitized at a 10-Hz sampling rate through a 12-bit digital-to-analog converter connected to a laboratory computer.

The total number of all animals used for modeling was 11. The orders of drug infusions were the following four: (i) 3, 6, and 9 μg kg⁻¹ min⁻¹ in DBT and 1, 2, and 4 μg kg⁻¹ min⁻¹ in SNP (n = 2 of 11 animals); (ii) 3 and 6 μg kg⁻¹ min⁻¹ in DBT (n = 3); (iii) 9 μg kg⁻¹ min⁻¹ in DBT (n = 3); and (iv) 1, 2, and 4 μg kg⁻¹ min⁻¹ in SNP (n = 3). Specifically, five of all animals were used for each drug infusion rate. The hemodynamic variables [n = 9 except for the two animals in the above case (iii), the 9 μg kg⁻¹ min⁻¹ in DBT (n = 3), because of no measurements] were changed to −60.2 ml kg⁻¹ min⁻¹ in CO (p < 0.01, paired t-test) and +4.9 mmHg in MAP (not significant) before (CO, mean ± S.E.M = 132.0 ± 11.2 ml kg⁻¹ min⁻¹; and MAP, 92.8 ± 3.2 mmHg) and immediately after (CO, mean ± S.E.M = 71.8 ± 6.2 ml kg⁻¹ min⁻¹; and MAP, 97.7 ± 4.3 mmHg) the heart failure; for hemodynamics immediately before each drug infusion, see Fig. 1A.). The period between two trials of drug infusions was 10 min in each animal. To prevent the washout process of a catheter in the above case (i), both the drug infusions of DBT and SNP (n = 2), a double-lumen catheter was used.

FIGURE 1.

Single-drug dose responses in canine left heart failure (n = 5). A. Step responses of the change of cardiac output (ΔCO, left) and mean arterial blood pressure (ΔMAP, right) from the baseline after induced acute heart failure during 10-min infusion of (a) dobutamine (DBT) at 3, 6, and 9 μg kg⁻¹ min⁻¹ and (b) sodium nitroprusside (SNP) at 1, 2, and 4 μg kg⁻¹ min⁻¹. Data digitized at 10 Hz were averaged every 30 s. Data are mean ± S.E.M. B. Unit impulse responses of ΔCO (left) and ΔMAP (right) calculated from data of (a) DBT infusion at 3, 6, and 9 μg kg⁻¹ min⁻¹ and (b) SNP infusion at 1, 2, and 4 μg kg⁻¹ min⁻¹ in dogs. CO response of SNP at 1 μg kg⁻¹ min⁻¹ was eliminated because of impossible fit to step response.

In this study, component models comprising a first order dynamic system cascaded with a nonlinear sigmoidal function were used to model the responses of CO and MAP with heart failure. After pharmacodynamics for the evaluation of controllers was represented by a linear first-order transfer function,18 a sigmoidal function was applied to the linear model to express the nonlinear characteristic with a component model approximating the positive step response.4,30 The procedure to produce the model responses can be described as follows.

First, simple models for responses to therapeutic agents were produced from experimental data in canine left-heart failure. Figure 1A shows the step responses of CO and MAP changed (ΔCO and ΔMAP) from baseline values immediately before each drug infusion after inducing the acute heart failure during 10-min (a) DBT and (b) SNP infusions. The step responses of ΔCO and ΔMAP during infusion of DBT at 3, 6, and 9 μg kg⁻¹ min⁻¹ or SNP at 1, 2, and 4 μg kg⁻¹ min⁻¹ were averaged every 30 s. Then, each single-input single-output response [Δŷ(t)] of the four step responses (DBT-CO, DBT-MAP, SNP-CO, and SNP-MAP loops as input–output relationships) was approximated to the linear first-order delay system with a pure time delay in the continuous-time domain, as

$$ \Delta \mathop y\limits^ \wedge (t) = K\left[ {1 - {\hbox{exp}}\left( { - \frac{{t - L}} {{T_{\rm c} }}} \right)} \right] $$

(1)

where K is a proportional gain, L is a pure time delay, and T _c is a time constant. If t < L then Δŷ(t) = 0. The fitted parameters to the averaged step responses (n = 5) in the single infusion of DBT or SNP were acquired by least squares method and used for calculation of the following unit impulse response (i.e., the response to the infusion of 1 μg kg⁻¹ min⁻¹ of a given drug).

Second, the linear-model response [Δy ^*(t)] was calculated by the convolution integral in the discrete-time domain:

$$ \Delta y^* (t) = \sum\limits_{\tau = 0}^{N_{\rm m} } {g(\tau )\Delta Tu(t - \tau )} $$

(2)

where

$$ g(\tau ) = \frac{{K_{\rm u} }} {{T_{\rm c} }}\exp \left( { - \frac{{\tau - L}} {{T_{\rm c} }}} \right) $$

.

In those equations, u(t) is the drug infusion rate, ΔT is the sampling interval, and N _m is the finite number of terms in the model for the unit impulse response. The unit impulse response is g(t), as calculated from the derived values of the step response of Eq. (1). The proportional gain of a unit impulse response is K _u; T _c and L are the same values as Eq. (1). For the simulation study, ΔT and N _m were set, respectively, to 30 s and 20. According to this method, the unit impulse responses of ΔCO and ΔMAP during infusions of DBT at 3, 6, and 9 μg kg⁻¹ min⁻¹ and SNP at 1, 2, and 4 μg kg⁻¹ min⁻¹ were calculated from the fitting parameters to the average step responses (Fig. 1B, the CO response to SNP at 1 μg kg⁻¹ min⁻¹ was excepted because of the impossible fit to the step response). Although some differences existed among unit impulse responses at those infusion rates, as shown in Fig. 1B, in the present study, median values (i.e. 6 μg kg⁻¹ min⁻¹ in DBT and 2 μg kg⁻¹ min⁻¹ in SNP) were used for the following simulations because they characterized the effects of drugs on hemodynamics well. Numerical data in K _u, T _c, L, and g(t) used for the simulation study are shown in Table 1 (left).

TABLE 1. Model parameters in linear-fitting and nonlinear-fitting functions.

Third, to express the nonlinear response to a single drug infusion, the Δy ^*(t) in Eq. (2) as the linear model response was modified through a sigmoidal function:30

$$ \Delta y'(t) = p_1 \tanh \left( {\frac{{p_2 \Delta y^* (t)}} {2}} \right) $$

(3)

in which p ₁ is the parameter of the response range, which shows the difference between the maximum and minimum values of Δy′(t) as the nonlinear model response, and p ₂ is the parameter of the coefficient of gain. Parameters p ₁ and p ₂ were determined by nonlinear least-squares method for the simulation study [Table 1 (right) and Fig. 2A]. Figure 2A contains the average values in ΔCO and ΔMAP responses of the final 30 s during 10-min infusions of DBT at 3, 6, and 9 μg kg⁻¹ min⁻¹ and SNP at 1, 2, and 4 μg kg⁻¹ min⁻¹ (Fig. 1A). The ΔCO and ΔMAP responses tested in simulations were over the ranges of the averaged responses in actual hemodynamics of dogs with heart failure.

FIGURE 2.

A. Nonlinear responses of ΔCO and ΔMAP in infusion of DBT or SNP. Circles (•) show experimental data for dogs with heart failure; solid lines are curves fitted to the averaged data. Dashed lines show the limits of the tested range [one-third (a ₁ = a ₂ = b ₁ = b ₂ = 1/3) to three-times (a ₁ = a ₂ = b ₁ = b ₂ = 3) responses compared with the averaged data (a ₁ = a ₂ = b ₁ = b ₂ = 1) in Eq. (4)] in simulations. B. Multi-input multi-output model responses for ΔCO and ΔMAP. System inputs are the infusion rates of DBT and SNP; outputs are the ΔCO and ΔMAP. The outputs were determined by the first-order dynamic systems cascaded with nonlinear sigmoidal functions. The proportional gains of patient sensitivities to drugs and drug interactions are shown as a ₁, a ₂, b ₁, and b ₂.

Finally, the model responses containing patient sensitivities to therapeutic agents and drug interactions are expressed as

$$ \left\{ \begin{aligned}{} & \Delta CO_{\bmod } (t) = a_1 \Delta CO'_1 (t) + a_2 \Delta CO'_2 (t) \\ & \Delta MAP_{\bmod } (t) = b_1 \Delta MAP'_1 (t) + b_2 \Delta MAP'_2 (t) \\ \end{aligned} \right. $$

(4)

, where ΔCO_mod(t) is the model response containing sensitivities to drugs and drug interactions of DBT and SNP; ΔCO₁′(t) and ΔCO₂′(t), respectively, indicate the nonlinear responses to single infusions of DBT and SNP; and a ₁ and a ₂, respectively, represent the proportional gain of patient sensitivity to DBT and SNP. The model response containing sensitivities to drugs and drug interactions of SNP and DBT is ΔMAP_mod(t). The respective nonlinear responses to single infusions of SNP and DBT are ΔMAP₁′(t) and ΔMAP₂′(t). The respective proportional gains of patient sensitivity to SNP and DBT are denoted as b ₁ and b ₂ (see Fig. 2B). In particular, a ₂ and b ₂ can be defined as the strength of the drug interaction when the two treatments for DBT-CO and SNP-MAP loops are performed.

Development of Controller

Control Design

Figure 3 portrays a block diagram of a MAPC_NN system for adaptation to various patient responses with heart failure. Treating the multiple loops in the therapy for heart failure separately allows setting of a clear goal of NN learning for the various patient responses during closed-loop control. On the other hand, in the completely separated controllers, the total control performance will be late because one controller performs the drug therapy after detection of the drug interaction disturbance induced by performing an action taken by the other controller. Therefore, the MAPC_NN in this study includes two module controllers for the DBT-CO loop considering the effects of SNP on CO and the SNP-MAP loop considering the effects of DBT on MAP.

FIGURE 3.

A block diagram showing multiple adaptive predictive control using neural networks (MAPC_NN) to regulate CO and MAP. The r is a target value, and e(t) is the error between the target value and observed value. The value e(t + i) represents the error between the target value and output predicted by the NN. Thick lines show the learning loop in the NN; dotted lines show the prediction loop using the NN.

Figure 4A depicts one of the two NN structures in MAPC_NN tested for the simulation study. The MAPC_NN is a control system in which the NN recursively learns patient characteristics using their observed responses to drug infusions only once every 30 s during closed-loop control (Learning Loop). It subsequently determines the future outputs using the learned NN (Prediction Loop). A multilayer feed-forward NN with two hidden layers (Δy _NN) emulated the nonlinear responses in ΔCO_mod and ΔMAP_mod. The Δy _NN is predicted through NN as

$$ \Delta y_{{\rm{NN}}} (t) = f\left[ {\Delta y_{{\hbox{mod}}} (t - 1),\,\Delta y_{{\hbox{mod}}} (t - 2),\,u_1 (t - 1),\, \ldots ,\,u_1 (t - 6),\,u_2 (t - 1),\, \ldots ,\,u_2 (t - 6)} \right] $$

(5)

where Δy _mod(t−1) and Δy _mod(t−2) are model responses of past ΔCO or ΔMAP; u ₁(t−1),..., and u ₁(t−6) or u ₂(t−1),... and u ₂(t−6) represent the past 3-min infusion rates of DBT or SNP. Here, to determine the length of the history of model response and infusion rates as inputs to the NN, the accuracy of NN learning was tested under various lengths of components (for a detailed protocol, refer to the following paragraph “Learning of Initial Weights in NN”). Figure 4B shows the average values (final 500 points) in the absolute error between the ΔCO_mod and ΔCO_NN responses (left) or ΔMAP_mod and ΔMAP_NN responses (right) from the trained NN [100,000 times, a ₁ = a ₂ = b ₁ = b ₂ = 1 in Eq. (4)] under various input–output components to the NN input. The numbers of the input units (m) to a single NN component and of the units in the first (n ₁) and second (n ₂) hidden layers of the NN were set to the same as that of components to the NN input. For example, if the inputs to the NN were 14 (the past 3 min in both the two drug inputs and the past 1 min in the model response), then m = n ₁ = n ₂ = 14. Learning rates of the two NN in both the CO and MAP controls were set to K _n1 = K _n2 = 0.2. The starting weights of NN were given at random every trial. In the past history of infusion rates, the error between the NN and model responses in the past 2-min infusion rates (eight components in Fig. 4B: four in DBT and four in SNP) showed adequate accuracy, although that in the past 1-min infusion rates (four components) was not demonstrably accurate in both the CO and MAP responses. Here, it was predicted that the pure time delays to drug inputs were different among patients and that the NN was required to express the characteristics of the transient response to a drug input adequately considering the effect of the other drug input. Therefore, to adjust the NN response to the changes of hemodynamics with noise and disturbances during the real-time control, the infusion rates of the past 3 min in both drugs (total 12 components: 6 components in each drug) were, on the safe side, selected for both the CO and MAP controls in the present study. In the past history of model response, the accuracy of NN learning was the almost equal among the cases of the past 1, 1.5, and 2 min (2, 3, and 4 components in Fig. 4B) in both the CO and MAP responses, whereas that in the past 30 s (1 component) showed inadequate accuracy. Accordingly, the length of the past history of model response was determined as 1 min (2 components) in both the CO and MAP controls.

FIGURE 4.

A. A single component of a four-layer feed-forward NN with two hidden layers in MAPC_NN to emulate the characteristics of a patient. The number of units in each hidden layer of the NN was set to 14 (numerically equal to the input units). A hyperbolic tangent function was used as the output of each unit. B. Average values (final 500 points) in the absolute error between the ΔCO_mod and ΔCO_NN responses (left) or ΔMAP_mod and ΔMAP_NN responses (right) from the trained NN [100,000 times, a ₁ = a ₂ = b ₁ = b ₂ = 1 in Eq. (4)] under various lengths of input–output components to NN input. C. The absolute error between the model response and the predicted response by NN in (a) ΔCO (left) or (b) ΔMAP (right).

In the simulation study, the number of input units to a single NN component was m = 14; the numbers of units in the first and second hidden layers of the NN were n ₁ = n ₂ = 14, being equal to the input units. The weights in the single NN were 435: 196 in the input to first hidden layer, 196 in the first to second hidden layer, 14 in the second hidden layer to output layer, and 29 for biases (Fig. 4A). The two NN for the controls of DBT-CO and SNP-MAP loops had identical structures.

In the learning loop, a single NN was trained by the output of ΔCO_mod or ΔMAP_mod to the random inputs of DBT and SNP using the backpropagation algorithm in an on-line mode, showing that the error function is calculated after presentation of an input. The prediction loop in the MAPC_NN determines the optimal DBT or SNP infusion rate that minimizes the cost function using the updated NN through the learning loop every 30 s.

$$ \left\{ \begin{aligned}{} & J_1 (t) = q_1 \left[ {u_1 (t) - u_1 (t - 1)} \right]^2 + \sum\limits_{i = 1}^{N_{p1} } {\left[ {r_1 (t + i) - \Delta {\hbox{CO}}_{{\rm{NN}}} (t + i)} \right]^2 } \\ & J_2 (t) = q_2 \left[ {u_2 (t) - u_2 (t - 1)} \right]^2 + \sum\limits_{i = 1}^{N_{p2} } {\left[ {r_2 (t + i) - \Delta {\hbox{MAP}}_{{\rm{NN}}} (t + i)} \right]^2 } \\ \end{aligned} \right. $$

(6)

Therein, the cost function [J ₁(t), J ₂(t)] comprises the weight of input change (q ₁, q ₂), the prediction range (N _p1, N _p2), and the setpoint (r ₁, r ₂) in the controllers for the DBT-CO loop or the SNP-MAP loop. The physiological responses predicted by the NN are indicated as ΔCO_NN and ΔMAP_NN. The controller based on the NN for each loop predicts future outputs using past inputs of infusion rates of DBT and SNP. The optimization of the infusion rates [u ₁(t), u ₂(t)] was performed using a Nelder–Mead Simplex algorithm.16,27

Determination of Control Parameters

Learning of Initial Weights in NN

The two NN, respectively, learned ΔCO_mod response in the DBT-CO loop and ΔMAP_mod response in the SNP-MAP loop to determine the initial weights in the NN for the MAPC_NN. The starting weights in the NN before learning the model response were given at random between −1 and 1. Subsequently, the infusion rates of DBT and SNP were given at random between −4 and 6 μg kg⁻¹ min⁻¹. Here, both the plus and minus signs as drug inputs (artificial infusions) to the NN were used because the learning of NN was inferred to be more effective than that under a plus sign alone as the drug input to the NN in the trial and error and previous studies.9,27 Learning of the NN for ΔCO_mod or ΔMAP_mod responses was repeated 100,000 times. The ΔCO_mod was divided by 200, and ΔMAP_mod was divided by 100 for normalization during NN learning.

Figure 4C shows the absolute error between the ΔCO_mod and ΔCO_NN responses (left) or ΔMAP_mod and ΔMAP_NN responses (right) from the trained NN in the average patient sensitivity [a ₁ = a ₂ = b ₁ = b ₂ = 1 in Eq. (4)]. Learning rates of the two NN for the following simulation study were set to K _n1 = K _n2 = 0.2 showing a suitable number by trial and error in both the ΔCO_mod and ΔMAP_mod. Learning results of the NN, respectively, showed errors of 2.5 ml kg⁻¹ min⁻¹ in the DBT-CO loop and 1.5 mmHg in the SNP-MAP loop.

Controller Tuning

Optimal values of the range of prediction (N _p1, N _p2) and the weight of input (q ₁, q ₂) in the cost function (6) of the controller were explored using the model patient responses, ΔCO_mod and ΔMAP_mod, to determine the initial controller parameters. The prediction range was set to N _p1 = N _p2 = 4, 8, or 12, and the weight of input was set to q ₁ = q ₂ = 0.01, 0.1, or 1. The learning rates of ΔCO_mod and ΔMAP_mod were fixed at K _n1 = K _n2 = 0.2. To regulate the control speed and stability simultaneously, the setpoint of ΔCO was guided by the linear function r ₁ = 35t/600 ml kg⁻¹ min⁻¹ during the 10 min following the start of the closed-loop control. Thereafter, it was maintained at r ₁ = 35 ml kg⁻¹ min⁻¹. The setpoint of ΔMAP was set to r ₂ = ±0 mmHg. The duration of the closed-loop control was set to 40 min.

Figure 5 shows simulation results of the MAPC_NN using average responses of ΔCO_mod and ΔMAP_mod [a ₁, a ₂, b ₁, and b ₂ in Eq. (4) were set to unity]. The controller suppressed a control speed instead of facilitating stable control with the increase of the weight of input (q ₁ = q ₂ = 0.01→1) at each range of prediction (N _p1 = N _p2 = 4, 8, or 12). On the other hand, when the weight of the input is small (q ₁ = q ₂ = 0.01), the controller performed a slightly aggressive control with the decreased range of prediction (N _p1 = N _p2 = 12→4). The relationship at the large weight of input (q ₁ = q ₂ = 1) was opposite because of the strong effects of the input weight at the small range of prediction (N _p1 = N _p2 = 4). For subsequent simulations, the parameters (N _p1, N _p2, q ₁, q ₂) in the MAPC_NN were set to N _p1 = N _p2 = 12 and q ₁ = q ₂ = 0.01 considering the settling time which reflects control speed and stability. In this case, the settling time within ±3 ml kg⁻¹ min⁻¹ in ΔCO_mod was 660 s; its time within ±2 mmHg in ΔMAP_mod was 990 s. The average absolute value of error between a setpoint and model response in ΔCO_mod or ΔMAP_mod over the entire control period (average error) for 40 min was 2.8 ml kg⁻¹ min⁻¹ in ΔCO or 0.5 mmHg in ΔMAP.

FIGURE 5.

Simulation results of the MAPC_NN under average responses of ΔCO_mod and ΔMAP_mod [a ₁ = a ₂ = b ₁ = b ₂ = 1]. The input weight was changed to q ₁ = q ₂ = 0.01, 0.1 or 1 fixing the range of prediction at N _p1 = N _p2 = (A) 4, (B) 8, or (C) 12. The NN learning rate was set to K _n1 = K _n2 = 0.2 under all conditions.

Evaluation of Controller

Simulations

To evaluate the control performance, a simulation study in MAPC_NN, which expressed the repeatability and freely operated physiological parameters such as nonlinearity and interaction,22,30 was performed under unexpected changes of patient responses to therapeutic agents with acute disturbances using the model response based on experimental data of canine heart failure. Increased CO to more than 95 ml kg⁻¹ min⁻¹, while keeping MAP within the normal range (80–100 mmHg) is desirable to treat acute heart failure.6,7 Therefore, the control objectives in this study were to increase the low CO (mean ± S.E.M. = 67.5 ± 3.4 ml kg⁻¹ min⁻¹, Fig. 1A) at the setpoint of +35 ml kg⁻¹ min⁻¹ (ΔCO = +35t/600 within 10 min after the start of closed-loop control) using DBT infusion and to simultaneously maintain the normal MAP (mean ± S.E.M. = 96.7 ± 2.0 mmHg, Fig. 1A) at the setpoint (ΔMAP = ±0 mmHg) using SNP infusion when hypertension is induced by treatment for DBT-CO loop in acute heart failure. The infusion rates were basically bounded as 0 ≤ u ₁(t) ≤ 10 in DBT and 0 ≤ u ₂(t) ≤ 6 in SNP to avoid an overdose or drug toxicity.5,22 Hemodynamic control was simulated under the following cases: drug interactions between DBT and SNP, acute disturbances, and time-variant changes of physiological parameters.

Drug Interactions

Grasping and estimating the acts of drug interactions are difficult for hemodynamic control using multiple drugs.6,30 To examine the controller’s robustness for wide ranges of patients’ sensitivity to drugs and drug interactions, parameters (a ₁, a ₂, b ₁, b ₂) in Eq. (4) were changed to 2, 1/3, or 3 every 40 min for 120-min control (Fig. 6A). Hemodynamic responses were unknown to the NN because the NN learned only the average model responses (a ₁ = a ₂ = b ₁ = b ₂ = 1). The controller was also tested under various combinations for sensitivities to drugs and drug interactions. The model parameters, a ₁, a ₂, b ₁, and b ₂ in Eq. (4), were set to one of 1/3 (Low), 1 (Mid.), and 3 (High), and all combinations were tested with or without the limitations of drug infusion rates. The control duration was 40 min.

FIGURE 6.

Simulation results of MAPC_NN with unexpected patient sensitivities and drug interactions. A in the top graph displays changes of parameters (a ₁, a ₂, b ₁, b ₂) in Eq. (4). B(a) and C(a) in the graph display setpoints, ΔCO_mod and ΔMAP_mod (solid lines), and predicted outputs by NN (dashed lines). B(b) and C(b) are the time courses of weights changed from the baseline at the starting time in the NN for controllers in the DBT-CO and SNP-MAP loops: weights between input and first hidden layers (top), first and second hidden layers (middle), and second hidden and output layers (bottom). D. Infusion rates of DBT (solid line) and SNP (dashed line).

To know the limitation of the control performance, the MAPC_NN was tested under very low (a ₁ = a ₂ = b ₁ = b ₂ = 1/5 or 1/10) and high (a ₁ = a ₂ = b ₁ = b ₂ = 5 or 10) sensitivities to drugs and drug interactions. The limitations of infusion rates of drugs were eliminated to emphasize the control performance. The control duration was 40 min.

Acute Disturbances

Bolus infusion of drugs during and after cardiac surgery often introduces severe disturbances to a controller. Hemorrhage, patient-position changes, and changes in anesthesia levels will modulate patient response characteristics.10,20 To examine the controller’s performance during acute disturbances, the MAPC_NN against acute hypertension was simulated using the ΔCO_mod and ΔMAP_mod responses for 120 min. Exogenous perturbations were added to the patient responses ranging within ±10 ml kg⁻¹ min⁻¹ in ΔCO_mod and +20 mmHg in ΔMAP_mod. To mimic physiological variation, random noises within ±3 ml kg⁻¹ min⁻¹ in ΔCO_mod and ±2 mmHg in ΔMAP_mod were also added [Fig. 9A(a) and B(a)].

To implicate the limitation of the control performance, the MAPC_NN was tested under very severe situations with random noise and exogenous perturbations over physiological responses: (i) very huge amplitudes of random noise, (ii) very large and (iii) acute disturbances. First, (i) three levels of huge amplitudes of random noise (level 1: ±5 ml kg⁻¹ min⁻¹ in ΔCO_mod and ±5 mmHg in ΔMAP_mod, level 2: ±25 ml kg⁻¹ min⁻¹ and ±25 mmHg, and level 3: ±50 ml kg⁻¹ min⁻¹ and ±50 mmHg) were added to model responses (a ₁ = a ₂ = b ₁ = b ₂ = 1) during drug treatment for 40 min [Figs. 10A(a) and B(a)]. The random noises among the three levels had the same pattern except for the amplitude. Next, (ii) the MAPC_NN was evaluated under very huge disturbances (level 1: −20 ml kg⁻¹ min⁻¹ in ΔCO_mod and +20 mmHg in ΔMAP_mod, level 2: −50 ml kg⁻¹ min⁻¹ and +50 mmHg, and level 3: −100 ml kg⁻¹ min⁻¹ and +100 mmHg; a ₁ = a ₂ = b ₁ = b ₂ = 1; Fig. 11A). Finally, (iii) the MAPC_NN was tested under very acute disturbances with model responses in high sensitivities to drugs and drug interactions (a ₁ = a ₂ = b ₁ = b ₂ = 3). At 20 min after the start of control for 80 min, the acute disturbances of three levels [10 (level 1), 5 (level 2), and 0 (level 3) min to reaching the offset values of −50 ml kg⁻¹ min⁻¹ in ΔCO_mod and +50 mmHg in ΔMAP_mod after the start of the disturbances] were added (Fig. 11B). In those simulations, the limitations of infusion rates of drugs were eliminated to elucidate the limitations of the control performance.

Time-variant Changes

Patient responses to therapeutic agents have a nonlinear and time-variant nature. If the delay of a plant, which reflects the infusion rate of drugs through the catheter, internal patient circulation and perfusion delay, and the drug-recirculation characteristics of a patient,2,26,30 is not known accurately or changes during drug infusions, infusion delays will engender an unstable condition. Therefore, the controller robustness was tested under the change of the infusion delay in this study. The pure time delays [L in the unit impulse response of Eq. (2)] of patient responses to therapeutic agents were varied from 30 to 90 s in CO responses and 60 to 120 s in MAP responses to DBT and SNP infusions during the closed loop control [Fig. 12A(a)]. The parameters (a ₁, a ₂, b ₁, b ₂) in Eq. (4) were varied from 1/3 to 3 to examine the controller performance under time-variant and wide ranges of patient sensitivities to drugs and drug interactions [Fig. 12A(b)]. The exogenous perturbations were added to the time-variant patient responses ranging within ±10 ml kg⁻¹ min⁻¹ in ΔCO_mod and +20 mmHg in ΔMAP_mod. Random noises within ±3 ml kg⁻¹ min⁻¹ and ±2 mmHg were added to ΔCO_mod and ΔMAP_mod responses [Fig. 12B(a) and C(a)]. The control duration was 120 min.

Animal Study

To evaluate the control performance in MAPC_NN under the unknown physiological responses such as nonlinearity and drug interaction, the simultaneous control of CO and MAP was performed using a dog with acute heart failure. Acute ischemic heart failure in an anesthetized and ventilated dog (23 kg) was induced by microsphere embolization of the left main coronary artery. A double-lumen catheter was introduced into the right femoral vein for administration of drugs using an infusion pump (CFV-3200; Nihon Kohden, Tokyo, Japan). CO was measured by an electromagnetic flow probe (MFV-2100; Nihon Kohden), and MAP was measured through a pressure transducer (DX-200; Nihon Kohden) at a 10-Hz sampling rate through a 12-bit digital-to-analog converter.

The control objective was to increase the low CO (62.4 ml kg⁻¹ min⁻¹) at the setpoint (+40%) using DBT infusion and to maintain the MAP (73.9 mmHg) at the setpoint (ΔMAP = ±0 mmHg) using SNP infusion in acute heart failure. The closed-loop control duration was 60 min. The infusion rates were bounded as 0 ≤ u ₁(t) ≤ 10 in DBT and 0 ≤ u ₂(t) ≤ 6 in SNP.

RESULTS

Simulations

Drug Interactions

Figure 6 shows simulation results of closed-loop control by MAPC_NN (N _p1 = N _p2 = 12, q ₁ = q ₂ = 0.01, and K _n1 = K _n2 = 0.2) under unknown patient sensitivities to drugs and drug interactions. The ΔCO in the DBT-CO loop converged on the setpoint (+35 ml kg⁻¹ min⁻¹) within approximately 15 min, according to the guided setpoint during the first 40-min control period, regardless of unanticipated patient sensitivities and drug interactions (a ₁ = a ₂ = b ₁ = b ₂ = 2). The control for the ΔMAP in the SNP-MAP loop minimally suppressed the hypertension (+4.2 vs. +15.0 mmHg with or without SNP infusion) induced by the DBT infusion. At 40 and 80 min of the closed-loop control, patient sensitivities and drug interactions were widely changed (a ₁ = a ₂ = b ₁ = b ₂ = 2 to 1/3 and 1/3 to 3). The ΔCO converged on the setpoint robustly, whereas the ΔCO showed transient and large changes (+35.0 showing the setpoint to +14.9 and to +99.3 ml kg⁻¹ min⁻¹ at approximately 50 and 85 min). Although ΔMAP was decreased acutely by the change of patient sensitivity and drug interaction (±0 to −20.7 mmHg at approximately 85 min), it returned robustly to a normal level. The average errors between setpoints and observed responses were 6.4 ml kg⁻¹ min⁻¹ in CO and 1.6 mmHg (10.5 mmHg without SNP infusion) in MAP during 120-min closed-loop control. Weights and biases of two NN were adjusted to optimal values [Fig. 6B(b) and C(b)] when the unexpected changes occurred. The infusion rates of DBT and SNP were adjusted smoothly to optimal levels corresponding to the unknown patient responses to drugs.

Table 2 shows average errors between setpoints and model responses in ΔCO and ΔMAP during closed-loop control by MAPC_NN (N _p1 = N _p2 = 12, q ₁ = q ₂ = 0.01, and K _n1 = K _n2 = 0.2) under the various sensitivities to drugs and drug interactions. The control performance was overall accurate and a tendency excited for dependence on the sensitivity of CO to DBT (i.e., parameter a ₁); the control performance was increased as the sensitivity of CO to DBT increased (a ₁ = 1/3→3). However, in some cases, great hypertension was observed during control, as displayed in Fig. 7A; although the CO control was smoothly completed under such cases, the great hypertension was induced because of the high interaction of DBT infusion used for CO treatment (i.e. b ₂ = 3). The maximum values of the hypertension in cases 1 (a ₁ = ‘Low’, a ₂ = ‘Low’, b ₁ = ‘Low’, and b ₂ = ‘High’), 2 (a ₁ = ‘Low’, a ₂ = ‘Low’, b ₁ = ‘Mid.’, and b ₂ = ‘High’), and 3 (a ₁ = ‘Low’, a ₂ = ‘Mid.’, b ₁ = ‘Low’, and b ₂ = ‘High’) were 60.3, 39.4, and 33.8 mmHg, respectively. Here, ‘Low’ = 1/3, ‘Mid.’ = 1, and ‘High’ = 3. In case 2, the infusion rates of DBT were slightly disturbed and the estimation error between the NN and ΔCO responses oscillated. However, the ΔCO in case 2 was unaffected by the oscillation of DBT infusion because of the low sensitivity of CO to DBT (a ₁ = ‘Low’). Infusion rates of SNP in all cases were saturated at around 8 min because of the limitation of drug infusion. After the saturation, the infusion rate of SNP in case 1 was decreased at around 12 min.

TABLE 2. Average errors between setpoints and model responses in CO and MAP under various sensitvities to drugs and drug interactions.

FIGURE 7.

Simulation results of MAPC_NN under cases of great hypertension (A) with and (B) without the limitation of drug infusion rates. Parameters (a ₁, a ₂, b ₁, b ₂) in Eq. (4) are a ₁ = ‘Low’, a ₂ = ‘Low’, b ₁ = ‘Low’, and b ₂ = ‘High’ in case 1, a ₁ = ‘Low’, a ₂ = ‘Low’, b ₁ = ‘Mid.’, and b ₂ = ‘High’ in case 2, and a ₁ = ‘Low’, a ₂ = ‘Mid.’, b ₁ = ‘Low’, and b ₂ = ‘High’ in case 3. ‘Low’, ‘Mid.’, and ‘High’ are 1/3, 1, and 3, respectively. The hemodynamic responses (top), the error between the model response and the predicted response by NN during control (middle), and infusion rates of DBT and SNP (bottom) in ΔCO (left) or ΔMAP (right).

To elucidate the control performance without the limitation of infusion rates of drug inputs, the MAPC_NN was also tested under the same cases 1, 2, and 3 (Fig. 7B). However, great hypertension was not improved by SNP infusion without the drug input limitation (63.8, 39.8, and 38.4 mmHg in cases 1, 2, and 3) compared with those with the limitation of the drug input. In cases 2 and 3, the drug infusion rates in DBT were disturbed and the estimation errors between the NN and ΔCO responses had some oscillations. In addition, in all cases, the infusion rates of SNP were decreased after increasing to around 10 μg kg⁻¹ min⁻¹, irrespective of the remaining the hypertension.

Figure 8 shows the closed-loop control by MAPC_NN (N _p1 = N _p2 = 12, q ₁ = q ₂ = 0.01, and K _n1 = K _n2 = 0.2) under (A) very low (a ₁ = a ₂ = b ₁ = b ₂ = 1/5 or 1/10) and (B) high (a ₁ = a ₂ = b ₁ = b ₂ = 5 or 10) sensitivities to drugs and drug interactions. Under very low sensitivities to drugs and drug interactions, the ΔCO response was not able to reach the setpoint (±35 ml kg⁻¹ min⁻¹) because of the slight sensitivity of CO to DBT (a ₁ = 1/5 or 1/10), but it finally converged on a stable value (Fig. 8A, left). Estimation errors between the NN and model responses in ΔCO had oscillations under such conditions. On the other hand, under very high sensitivities to drugs and drug interactions, both the ΔCO and ΔMAP responses oscillated depending on the degree of sensitivities to drugs [maximum amplitudes between observed values and setpoints: (+)9.2 ml kg⁻¹ min⁻¹ and (−)18.8 mmHg in a ₁ = a ₂ = b ₁ = b ₂ = 5 or (+)35.9 ml kg⁻¹ min⁻¹ and (−)54.1 mmHg in a ₁ = a ₂ = b ₁ = b ₂ = 10, Fig. 8B]. However, the MAPC_NN had a tendency to suppress oscillations in the ΔCO and ΔMAP responses gradually. The estimation error between the NN and model responses and the infusion rates of drugs were also disturbed.

FIGURE 8.

Simulation results of MAPC_NN with (A) very low (a ₁ = a ₂ = b ₁ = b ₂ = 1/5 or 1/10) or (B) very high (a ₁ = a ₂ = b ₁ = b ₂ = 5 or 10) patients’ sensitivities to drugs and drug interactions. The hemodynamic responses (top), the error between the model response and the predicted response by NN during control (middle), and infusion rates of DBT and SNP (bottom) in ΔCO (left) or ΔMAP (right). The limitations of drug infusion rates were eliminated.

Acute Disturbances

Figure 9 shows closed-loop control by MAPC_NN (N _p1 = N _p2 = 12, q ₁ = q ₂ = 0.01, and K _n1 = K _n2 = 0.2) under unexpected acute disturbances with background noise. The ΔCO converged on the setpoint (+35 ml kg⁻¹ min⁻¹) within approximately 12 min according to the guided setpoint during the first 40-min control period, regardless of the added random noise (±3 ml kg⁻¹ min⁻¹) with expected patient responses (a ₁ = a ₂ = b ₁ = b ₂ = 1). The appropriate infusion of SNP for control of ΔMAP_mod with random noise (±2 mmHg) suppressed hypertension (+3.5 vs. +15.4 mmHg with or without SNP treatment) induced by DBT infusion. At 40 and 80 min of the closed-loop control, acute disturbances were added to the ΔCO_mod (+10 and −10 ml kg⁻¹ min⁻¹) and ΔMAP_mod (+20 and +10 mmHg) responses. The MAPC_NN suppressed the transient change of CO minimally and the CO converged on the setpoint as quickly as possible. The induced transient hypertension was suppressed robustly by optimal infusion of SNP (+6.7 vs. +31.0 mmHg with or without SNP treatment in the added disturbance at 40 min). Average errors between setpoints and observed responses were 3.7 ml kg⁻¹ min⁻¹ in CO and 1.4 mmHg (20.8 mmHg without SNP) in MAP during 120-min closed-loop control. Weights and biases of two NN were adjusted to the optimal values when the unexpected changes occurred [Fig. 9A(c) and B(c)]. Changes of biases were linked to those of the acute disturbances and physiological variation. The infusion rates of DBT and SNP were adjusted smoothly to optimal levels corresponding to the unknown disturbances.

FIGURE 9.

Simulation results of MAPC_NN with unknown disturbances. In the graph, A(a) and B(a) show changes of acute disturbances and random noise added to ΔCO_mod and ΔMAP_mod responses. A(b) and B(b) represent setpoints, ΔCO_mod and ΔMAP_mod (solid lines), and predicted outputs by NNs (dashed lines). A(c) and B(c) indicate weight changes in NN for the controllers in DBT-CO and SNP-MAP loops: weights between the input and first hidden layers (top), first and second hidden layers (middle), and second hidden and output layers (bottom). C. Infusion rates of DBT (solid line) and SNP (dashed line).

Figure 10 shows the closed-loop control by MAPC_NN (N _p1 = N _p2 = 12, q ₁ = q ₂ = 0.01, and K _n1 = K _n2 = 0.2) under very large amplitudes of random noise [levels 1 (±5 in both ΔCO_mod and ΔMAP_mod), 2 (±25), and 3 (±50), Fig. 10A(a) and B(a); a ₁ = a ₂ = b ₁ = b ₂ = 1]. Although the ΔCO and ΔMAP responses [Figs. 10A(b) and B(b)] approached the setpoints, they depended directly on the appearance patterns and the amplitudes in large random noise. The infusion rates in DBT and SNP and the estimation errors between the NN and model responses were also reflected by changes of random noise.

FIGURE 10.

Simulation results of MAPC_NN with large random noise. A(a) and B(a) show changes of large random noise added to ΔCO_mod and ΔMAP_mod responses. Noise level 1 (±5 in ΔCO_mod and ±5 mmHg in ΔMAP_mod), level 2 (±25 ml kg⁻¹ min⁻¹ and ±25 mmHg), and level 3 (±50 ml kg⁻¹ min⁻¹ and ±50 mmHg). A(b) and B(b) represent setpoints, ΔCO_mod and ΔMAP_mod responses (a ₁ = a ₂ = b ₁ = b ₂ = 1). A(c) and B(c) indicate the error between the model response and the predicted response by NN during control. A(d) and B(d) are Infusion rates of DBT and SNP. The limitations of drug infusion rates were eliminated.

Figure 11A shows the closed-loop control by MAPC_NN (N _p1 = N _p2 = 12, q ₁ = q ₂ = 0.01, and K _n1 = K _n2 = 0.2) under huge disturbances [levels 1 (−20 ml kg⁻¹ min⁻¹ in ΔCO_mod and +20 mmHg in ΔMAP_mod), 2 (−50 ml kg⁻¹ min⁻¹ and +50 mmHg), and 3 (−100 ml kg⁻¹ min⁻¹ and +100 mmHg); a ₁ = a ₂ = b ₁ = b ₂ = 1]. In the disturbance of level 1, the ΔCO and ΔMAP responses converged on the setpoints, whereas the estimation error between the NN and model responses in ΔCO exhibited some oscillations around 60–65 min. At level 2, the CO control was completed well. However, the MAP did not reach the setpoint because of the large disturbance and the interaction from DBT used for the CO control. In addition, the infusion rate of SNP was decreased at around 30 min irrespective of the hypertension of approximately 40 mmHg. The MAP response did not oscillate and converged on a steady state being far from the setpoint. At level 3, neither the ΔCO nor ΔMAP responses reached the setpoints. The infusion rate of SNP was increased instantaneously by receiving a great disturbance at 20 min, and it was decreased at around 25 min regardless of remaining the large hypertension of approximately 120 mmHg. The estimation error between the NN and model responses in ΔCO showed oscillation at approximately 40 min.

FIGURE 11.

Simulation results of MAPC_NN with (A) very huge and (B) acute disturbances. A: huge disturbance level 1 (−20 ml kg⁻¹ min⁻¹ in ΔCO_mod and +20 mmHg in ΔMAP_mod), level 2 (−50 ml kg⁻¹ min⁻¹ and +50 mmHg), and level 3 (−100 ml kg⁻¹ min⁻¹ and +100 mmHg). B: acute disturbance level 1 (10 min to the added disturbance of −50 ml kg⁻¹ min⁻¹ in ΔCO and +50 mmHg in ΔMAP), level 2 (5 min), and level 3 (0 min); a ₁ = a ₂ = b ₁ = b ₂ = 3 in Eq. (4). The hemodynamic responses (top), the error between the model response and the predicted response by NN during control (middle), and infusion rates of DBT and SNP (bottom) in ΔCO (left) or ΔMAP (right). The limitations of drug infusion rates were eliminated.

Figure 11B shows the closed-loop control by MAPC_NN (N _p1 = N _p2 = 12, q ₁ = q ₂ = 0.01, and K _n1 = K _n2 = 0.2) under very acute disturbances with high sensitivities to drugs and drug interactions [levels 1 (10 min to the added disturbances of −50 ml kg⁻¹ min⁻¹ in ΔCO and +50 mmHg in ΔMAP), 2 (5 min), and 3 (0 min); a ₁ = a ₂ = b ₁ = b ₂ = 3]. At levels 1 and 2, although the ΔCO and ΔMAP responses showed small oscillations because of the acute disturbances, they converged on the setpoints by 80 min. In level 3 (step input of disturbance), the ΔCO and ΔMAP received the effects of the step input of the very acute disturbances at 20 min directly. Although ΔCO and ΔMAP responses were disturbed until 70 min, those values tended to converge on the setpoints eventually. In particular, the infusion rate of SNP at level 3 was increased acutely to 20 μg kg⁻¹ min⁻¹ against the large hypertension of 50–60 mmHg induced by the very acute disturbance with high amplitude of 50 mmHg at 20 min; in turn, the excessive hypotension after the acute hypertension was induced by the high sensitivity of MAP to the SNP used for the MAP control (b ₁ = 3). However, the infusion rate of SNP was adjusted to the optimal value at approximately 70 min.

Time-variant Changes

Figure 12 shows the closed-loop control by MAPC_NN (N _p1 = N _p2 = 12, q ₁ = q ₂ = 0.01, and K _n1 = K _n2 = 0.2) under the unknown time-variant responses containing time delays to therapeutic agents with acute disturbances. The ΔCO converged on the setpoint (+35 ml kg⁻¹ min⁻¹) within approximately 10 min according to the guided setpoint during the first 40-min control, irrespective of time-variant patient sensitivities and drug interactions with added noise; it showed only slight oscillation (±5 ml kg⁻¹ min⁻¹). The SNP infusion for the control of ΔMAP suppressed the hypertension (+6.1 and +22.8 mmHg with and without SNP treatment) induced by DBT infusion beforehand. At 40 min of the closed-loop control, acute disturbances (−10 ml kg⁻¹ min⁻¹ in CO and +20 mmHg in MAP) were added to the patient responses. The ΔCO and ΔMAP quickly converged on the setpoint within approximately 10 min, whereas the transient hypertension was induced (+9.4 vs. +38.9 mmHg with or without SNP). The average errors between setpoints and observed responses were 4.2 ml kg⁻¹ min⁻¹ in CO and 2.7 mmHg (15.3 mmHg without a vasodilator) in MAP during 120-min closed-loop control. The weights and biases of two NN were adjusted to optimal values when the unexpected changes occurred [Fig. 12B(c) and C(c)]. Changes of biases were linked to those of the time-variant responses and acute disturbances. The infusion rates of DBT and SNP were adjusted to optimal levels corresponding to the unknown time-variant responses.

FIGURE 12.

Simulation results of MAPC_NN under unknown time-variant responses with disturbances. A(a) in the graph displays changes of the parameters of the time delays (L) in the unit impulse response of Eq. (2). A(b) indicates changes of the parameters (a ₁, a ₂, b ₁, b ₂) in Eq. (4). B(a) and C(a) show changes of acute disturbances and random noise added to ΔCO_mod and ΔMAP_mod responses. B(b) and C(b) denote setpoints, ΔCO_mod and ΔMAP_mod (solid lines), and predicted outputs by NN (dashed lines). B(c) and C(c) are the weight changes in NN for controllers in DBT-CO and SNP-MAP loops: weights between input and first hidden layers (top), first and second hidden layers (middle), and second hidden and output layers (bottom). D. Infusion rates of DBT (solid line) and SNP (dashed line).

Animal Study

Figure 13 shows results of closed-loop control by MAPC_NN (N _p1 = N _p2 = 8, q ₁ = q ₂ = 0.3, and K _n1 = K _n2 = 0.2) under the actual response of canine left heart failure. The CO in the DBT-CO loop converged on the setpoint within approximately 10 min regardless of the unknown response containing nonlinear, drug interaction, and the effects of arterial baroreflex with physiological variation in actual heart failure. The control for the ΔMAP in the SNP-MAP loop suppressed the acute hypertension (+20 mmHg) induced by the DBT infusion. Whereas the large disturbance like arrhythmia was induced at around 38 min during the control, the CO and MAP were appropriately controlled by the MAPC_NN adjusting the infusion rates of DBT and SNP to the optimal levels. The average errors between setpoints and observed responses were 7.3 ml kg⁻¹ min⁻¹ in CO and 6.4 mmHg in MAP during 60-min closed-loop control.

FIGURE 13.

Results of hemodynamic regulation by means of MAPC_NN under canine left heart failure. Raw data of CO (top) and MAP (middle) responses during DBT and SNP infusions. Bottom in the graph is the Infusion rates of DBT (solid line) and SNP (dashed line).

DISCUSSION

The development of automatic drug-delivery systems requires a controller that can adapt to the various patient responses in clinical situations. The MAPC_NN was confirmed to be robust with respect to uncertainty in drug interactions, acute disturbances, time-variant responses containing time delays to therapeutic agents (Figs. 6, 9, and 12), and the actual response of a dog with heart failure (Fig. 13) because of its ability to learn nonlinear and time-variant changes of the system during the on-line control.

The infusion of DBT increased MAP as well as CO in acute left heart failure of dogs [Fig. 1A(a)]. The DBT infusion does not generally act on the SVR whereas both the CO and SVR affect MAP.14 Therefore, the increase of MAP during DBT infusion in this study would have resulted mainly from increasing the CO induced by the actions of the beta receptors (β ₁, β ₂) of cardiac smooth muscle30 rather than the SVR. On the other hand, the SNP infusion resulted in the decrease of MAP and the increase of CO between middle and high doses [Fig. 1A(b)] because of the decrease of SVR (afterload of a heart) by SNP and secondarily increased CO.6 In addition, the SNP treatment might have suppressed the increase in preload through the decrease of SVR because of increasing venous compliance for retaining the blood in the veins and lowering the venous return to the heart in case of congestive heart failure. In the simultaneous regulation of CO and MAP, the control for CO induced hypertension (Figs. 6, 9, 12, and 13) because the primary control target in this study was the increase of low CO in acute heart failure. The MAPC_NN was able to suppress the hypertension using optimal infusion of SNP as well as increasing CO using DBT to an optimal target value because the combined infusion of an inotropic agent and a vasodilator would have acted effectively.12,13,25

Application of MAPC_NN to a multiple hemodynamic control accomplished the regulation of CO and MAP under various changes of the patient’s responses to drugs and disturbances (Figs. 6, 9, 12, and 13). In particular, regardless of the large change of patient sensitivities as shown in Fig. 6, the MAPC_NN robustly adjusted the acute and large changes to generate a stable condition. Similarly to the previous controllers,7,10 the MAPC_NN suppressed those disturbances performed stabilized control under unexpected acute disturbances (Figs. 9 and 12). Under time-variant patient responses with pure time delays, which are a crucial obstacle to stable control,2,26 the MAPC_NN provided sufficient control performance (Fig. 12). Regardless of actual nonlinear response, drug interaction, and partial disturbances with arrhythmia, the MAPC_NN could regulate the CO and MAP simultaneously (Fig. 13). The superior control performance resulted from the function to adjust the weights and biases of the NN to optimal points during the on-line control (Figs. 6, 9, and 12).

Only the two-NN models of average responses with heart failure were considered in the calculation of the appropriate multiple drug infusion rates of DBT and SNP (Fig. 3) to mitigate the enormous number of trials associated with the control design. Model predictive controllers or fuzzy controllers might require an extremely lengthy set-up stage to prepare the model banks as linear models of patients’ responses to drugs or to provide the experienced rules describing various cases in clinical settings whereas the controllers are an effective means of adjusting to various patients’ sensitivities to drugs21 and describing nonlinear responses to drugs5. A controller based on NN solves those problems because it decreases the number of models required for the control design of the various changes of hemodynamics clearly.

Irrespective of the wide range over physiological responses (Fig. 2) in simulations and actual response of a dog with heart failure containing the effects of baroreflex and the full renin–angiotensin system induced by long-term control,14 CO and MAP in the MAPC_NN promptly approached the setpoints because of the optimization of both the stability and speed for the MAPC_NN (Figs. 6, 9, 12, and 13). Therefore, the designed MAPC_NN will be feasible for application to automatic drug therapy in heart failures. However, when rapid treatment using drugs against more acute and large disturbances is required during hemodynamic controls, another supplemental system might be required.9 Diagnoses of characteristics of patients’ responses to drugs or tuning weights of NNs during closed-loop controls1 may also be effective for hemodynamic controls to accelerate the NN learning speed. In addition, because the fluid infusion, blood transfusion, anesthesia, and muscular blockade as well as the therapeutic agents controlled in this study are common in clinical practice,22,29 the controllers that can adjust physiological responses to further multiple drugs will be desired. The MAPC_NN tested herein can be extended simply to multivariate control systems under such clinical conditions for drug therapy with heart failure.

The MAPC_NN was tested under various conditions over physiological responses to elucidate the limitations of the control performance (Figs. 7, 8, 10, and 11). Regardless of such severe conditions, the hemodynamics during MAPC_NN learning very large changes of the sensitivities to drugs and drug interactions and the disturbances using NN tended to converge on the setpoints with some oscillations observed, insofar as those responses were within the possible range of the control using DBT and SNP (e.g., Figures 8B and 11B). On the other hand, there existed cases for which it was obviously impossible for MAPC_NN to control the hemodynamics (Figs. 7, 8A, and 11A). For example, when the interaction from DBT for the CO control to MAP was very large (b ₂ = 3) and the sensitivity of MAP to SNP was small (b ₁ = 1/3 or 1, Fig. 7), the MAPC_NN was actually incapable of attenuating the induced hypertension. Those cases resulted from nonlinear model responses to drugs in the present study; the hemodynamic responses, therefore, would have saturated because of the range of nonlinear model responses, regardless of the increase of drug infusion rates.

The NN in the MAPC_NN under the severe conditions seems to have tried to learn and adapt to the situations during real-time control. In cases of large error between actual responses and predicted responses where the NN in the MAPC_NN learned the average responses before the control (e.g. the CO responses of case 2 in Fig. 7A, cases 2 and 3 in Fig. 7B, and cases 1 and 3 in Fig. 11A), the hemodynamics during the control exhibited oscillation and the NN in the MAPC_NN would have tried to learn the severely changed situation again. In addition, when the hemodynamic response showed incorrect or opposite response to the drug input compared with the learned average response by the NN before the control [e.g. cases of not reaching the setpoint and converging on a stable state regardless of the DBT infusion for CO control because of very low sensitivity to the drug (a ₁ = 1/5 or 1/10, Fig. 8A) and of the opposite effect of SNP on MAP, speciously, because of the very large disturbances (levels 2 and 3 in Fig. 11A) compared with the learning response], the drug infusion rate of the MAPC_NN was decreased despite the remainder of the low CO or hypertension during the closed-loop control with real-time NN learning. Therefore, these simulation results suggest the following two points. First, in those cases such as the high interaction over the drug effect on the target physiological parameter and the incorrect or opposite responses to drugs compared with those of previously learned NN, improvement of the strategy in drug treatment would be required; alternatively, the MAPC_NN would fall into a situation of control impossibility. Second, the physiological variations related with responses to anesthesia, antiarrhythmic drug, and muscle relaxant, the external disturbances, and the artificial background noise must be set to the smallest possible values to bring out the best performance of the controller.

Several limitations are apparent in present study. First, the modeling for the CO and MAP responses to drugs might depend on the protocols of animal experiments such as the order and washout period of drug infusions. The protocol in the present animal study was, therefore, described in detail (reference ‘Modeling of Pharmacological Response’ in the ‘MATERIALS AND METHODS’ section). Second, one (low CO and normal MAP) of the heart-failure conditions was tested using the present animal study. The kinds of heart failures are various in actual patients. Therefore, further animal studies will be required. Finally, an electromagnetic flow probe was used for CO measurement in the present study. However, in an actual clinical setting, the common technique for CO measurement (e.g., the thermodilution technique using a pulmonary artery catheter19) has a slower response than that of an electromagnetic probe. The high accuracy of a flow probe, such as the time resolution (at least 30 s as used for this experiment), and the pure time delay from an actual response (as few response delays as possible) would be required to acquire the results that were obtained in the present study.

CONCLUSIONS

The MAPC_NN was designed and evaluated in simulation and animal studies to regulate the nonlinear responses of CO and MAP in acute heart failure using DBT and SNP under unexpected changes of patient sensitivities to drugs, drug interactions, acute disturbances, and time-variant responses to therapeutic agents. The MAPC_NN showed robust control performance irrespective of various unexpected responses to drugs over actual physiological responses (Fig. 2) and actual response of a dog in heart failure. Flexibility of a NN coupled with an adaptive control mechanism will enable the regulation of various physiological responses to drugs with heart failures.

APPENDIX

The error signal is propagated back through the network, thereby modifying the weights before the next input presentation. The backpropagation algorithm is performed in the following order: output layer, second hidden layer, and first hidden layer. All connection weights are adjusted to decrease the error function by the backpropagation learning rule based on the gradient decent method.23 The error function, E is

$$ {\rm E} = \frac{1} {2} \varepsilon ^2 = \frac{1} {2} \left( {\Delta y_{{\rm{mod}}} - \Delta y_{{\rm{NN}}} } \right)^2 $$

, where Δy _mod is the model response as a supervised signal, Δy _NN is the Δy _mod predicted by the NN before update of the connection weights, and ε is the error between Δy _mod and Δy _NN. The error is back-propagated through the network. The connection weight is generally updated by gradient descent of E as a function of weights:9,27

$$ w^* = w + K_{\rm{n}} \Delta w $$

, where

$$ \Delta w = \frac{{\partial E}} {{\partial w}} = \frac{{\partial E}} {{\partial \varepsilon }}\frac{{\partial \varepsilon }} {{\partial y_{NN} }}\frac{{\partial y_{{\rm{NN}}} }} {{\partial w}} = - \varepsilon \frac{{\partial y_{{\rm{NN}}} }} {{\partial w}} $$

, and in which w* is the single weight of each connection after updating, w is the single weight of each connection before updating, Δw is the modified weight, and K _n (K _n1 and K _n2 in the DBT-CO and SNP-MAP loops) is the learning rate. For detailed equations, refer to the previous article.9