Abstract
The handling of left ventricular assist devices (LVADs) can be challenging for patients and requires appropriate training. The devices’ usability impacts patients’ safety and quality of life. In this study, an eye tracking supported human factors testing was performed to reveal problems during use and test the trainings’ effectiveness. In total 32 HeartWare HVAD patients (including 6 pre-VAD patients) and 3 technical experts as control group performed a battery change (BC) and a controller change (CC) as an everyday and emergency scenario on a training device. By tracking the patients’ gaze point, task duration and pump-off time were evaluated. Patients with LVAD support ≥1 year showed significantly shorter BC task duration than patients with LVAD support <1 year (p = 0.008). In contrast their CC task duration (p = 0.002) and pump-off times (median = 12.35 s) were higher than for LVAD support patients <1 year (median = 5.3 s) with p = 0.001. The shorter BC task duration for patients with LVAD support ≥1 year indicate that with time patients establish routines and gain confidence using their device. The opposite effect was found for CC task duration and pump-off times. This implies the need for intermittent re-training of less frequent tasks to increase patients’ safety.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
This study investigates the human factors of the HeartWare HVAD (Medtronic, Minneapolis, MN, USA) left ventricular assist device (LVAD) system using eye tracking.
Due to the shortage of donor hearts, LVADs are a common solution for patients with heart failure [1, 2]. The HVAD is one of the commercially available continuous flow LVADs with more than 16,000 implantations worldwide [3, 4]. Although medical complications of HVADs are researched extensively, [5,6,7,8] less attention is given to the usability of its battery and controller system including malfunctions [9,10,11]. The human factors of LVAD wearable components remain a crucial challenge [12, 13].
Because LVADs are critical life-support systems, the device should be self-explanatory and optimization of usability, including intuitive handling and enhanced device-user communication is needed to increase the safety and quality of life [14]. A broad systematic review by Dunn et al. [11] showed existing human factor issues and how they influence user experience. However, an objective and quantitative assessment of human factors of LVAD systems without the use of questionnaires is still an ongoing challenge.
A useful tool to overcome this challenge is eye tracking, which enables to uncover common problems during the use of medical devices [15], provides valuable objective insights for user interface design [16] and allows to assess the effectiveness of trainings [17]. It proves as a successful skill and training assessment tool, [18] is applicable in medical education, [19] and allows for comparison between expert and novice surgeons [20,21,22]. Eye tracking glasses (see Fig. 1) do not restrict the participant in movement, thus allowing them to behave naturally. From the collected data the gaze point of the participant can be computed and displayed in the first person’s view video documentation (“scan path”, see Fig. 2). Furthermore, a comparison between different object and interest points can be investigated by the definition of areas of interest (AOIs). Using quantitative specific AOI measures like dwell time, the total time spend looking at an AOI, allows to draw conclusions regarding the visual behavior and attention.
Materials and methods
Out of the 240 patients currently supported by HVAD systems at this institution (out of a total of 350 long-term LVAD patients), 44 patients were enrolled in the study after informed consent and 32 patients were included in the analysis. Eight patients (six for analysis) were pre-operative patients, that serve as “novice” control group, while additional three experts (VAD-technicians or perfusionists) formed an expert control group. Patients had to be ≥18 years old, able to consent and speak German (B2). Twelve patients (including two pre-operative patients) had to be excluded from the analysis: three were deficient in reading German, one performed an early start of the controller change, for two patients the study had to be aborted due to an adverse emotional reaction to the controllers’ alarm, technical difficulties arose with three patients and three patients failed to successfully perform the controller change task. The characteristics of participants are shown in Table 1. The patients were stratified based on the duration of LVAD support. The cut-off was defined for the duration of LVAD support of ≥1 year based on the impact of LVAD implantation on patients’ cognitive function [23].
The standardized institutional training protocol is adapted to the mental ability of the individual patient. Hence the amount of training time depends on numerous variables. Patients have to pass defined criteria to successfully complete their training. This includes the ability of performing multiple battery changes (minimum of eight) and controller changes (minimum of three) with confidence and without any technical mistakes. The training is repeated as many times as needed till the criteria are fulfilled, otherwise a care giver or other means of continuous supervision must be guaranteed before discharge. All patients, except for pre patients, completed their training before taking part in this study. The pre patients were introduced to the HVAD systems right before the study, the components and their handling were explained.
The study was performed at the German Heart Center Berlin and the Paulinenkrankenhaus Berlin in closed rooms with constant conditions. After the participants read the instruction paper, they had to first perform a battery change (BC, everyday scenario) followed by a controller change (CC, rare emergency scenario) using a training HVAD unit. The start signal for the CC was an alarm activated by the study conductor using a separate device. The trained procedure for a controller change is as follows: switch alarm plug from backup controller to malfunctioning controller, switch first battery to backup controller, switch driveline to backup controller, switch second battery to backup controller. After each task the short stress state questionnaire was filled out [24, 25]. Patients not performing the two tasks correctly were directly trained after the study. Video data of the battery and controller change including the gaze point for patients and experts is accessible as supplementary material (see Video 1).
The SMI ETG 2 Wireless mobile eye tracking glasses (Senso Motoric Instruments, Teltow, Germany) with a sampling rate of 60 Hz and a gaze tracking accuracy of 0.5° were used to record the data (see Fig. 1). The system records the scene, including audio, as well as the eyes of the participant to allow the calculation of the participant’s gaze point. Using the gaze point the scan path can be observed displaying where the attention of the participant lies frame by frame.
The task durations of BC and CC were measured. The gaze point was used to find the start and end instants, which are defined as the first time the patients take their gaze from the instruction paper to the HVAD set up and the moment the patients look back at the instruction paper after task completion to turn the page. Furthermore, the pump-off time was measured, which states how long the pump is completely turned off during the CC. The pump is turned off as soon as the driveline is disconnected or if both batteries are disconnected from the malfunctioning controller, the pump is turned on again as soon as it is connected to the backup controller and if it provides power through a connected battery.
Dwell times (DTs) for AOIs of the relevant and irrelevant controller during the CC were calculated. The relevant controller is the one the pump is connected to, while the irrelevant controller is the one without pump. This stimulus switches as soon as the pump is disconnected from the malfunctioning controller and connected to the backup controller. Using the pump-off time, 10 s were used as a comparison criterion [26]. Data analysis was performed with SMI BeGaze 3.6 (Senso Motoric Instruments, Teltow, Germany).
A Pearson Chi-Square test was performed to evaluate gender effects. Differences between age, BC task duration, CC task duration and pump-off time, comparing pre patients, patients with LVAD support <1 year, patients with LVAD support ≥1 year and experts, were analyzed by a Kruskal-Wallis test. The same test was used for DT on irrelevant controller and DT on relevant controller comparing pre patients, LVAD patients with a pump-off time <10 s, LVAD patients with a pump-off time ≥10 s and experts.
It was hypothesized that these groups perform best to worst: Experts, patients ≥1 year, patients <1 year, pre patients.
-
H1 (hypothesis 1): Task durations are shorter for patients with a duration of LVAD support ≥1 year compared to patients with LVAD support <1 year.
-
H2: The pump-off time is shorter for patients with a duration of LVAD support ≥1 year compared to patients with LVAD support <1 year.
-
H3: Patients with a pump-off time ≥10 s have a higher dwell time on the irrelevant controller.
Results
The groups did not differ by participants gender C2(2, N = 35) = 1.14, p = 0.567 or age C2(3, N = 35) = 4.675, p = 0.197.
The task durations and the pump-off time results are displayed in Table 2 and Fig. 3.
Patients with LVAD support ≥1 year showed significantly shorter BC task duration than patients with LVAD support <1 year as expected. But their CC task duration were higher than for LVAD support patients <1 year. As expected, the experts have the shortest task duration and the pre patients the longest. This behavior is significant for BC task duration C2(3, N = 35) = 11.95, p = 0.008 and CC task duration C2(3, N = 35) = 14.49, p = 0.002.
Counter intuitively, the pump-off time for LVAD support patients ≥1 year is higher than for LVAD support patients <1 year. Again, the control groups set the minimal (experts) and maximal (pre patients) pump-off times like expected. The differences between the groups are significant: C2(3, N = 35) = 17.22, p = 0.001.
Displayed in Table 3 and Fig. 4 are the dwell times (DTs) on the irrelevant and relevant controller.
For the patients with a pump-off time ≥10s the DTs for relevant and irrelevant controller are higher than for patients with pump-off time <10 s. The differences between the pump-off time <10 s patients and the patients with a pump-off time ≥10 s normed with the DT of patients with a pump-off time ≥10 s are 29.26% for the irrelevant and 14.73% for the relevant controller. The DT on irrelevant controller is low as expected for experts and highest for pre patients. In every group, more DT is spent on the relevant controller. The differences show a significance for the DT on the irrelevant controller with C2(3, N = 35) = 8.95, p = 0.03 and for the relevant controller with C2(3, N = 35) = 8.76, p = 0.033.
Discussion
Five patients were excluded from the analysis due to an adverse emotional reaction to the device or because the patient failed to successfully perform the controller change task. Having to exclude 11.4% of the enrolled patients due to human factors related issues, demonstrates the general need for improvement of the design and handling of LVAD devices. The fact, that the BC task duration is shorter for patients with LVAD support ≥1 year (H1) implies that patients establish a routine with their device. But the opposite effect was found for CC task duration, it is higher for patients with LVAD support ≥1 year (H1). This shows that patients might have lost skills either because they were not trained enough, before their discharge from hospital, or because their training happened too long ago. Both aspects result in a need for adaption of the training processes.
Opposing the expectations of H2, the results show that the average pump-off time is shorter for patients with LVAD support <1 year compared to patients with LVAD support ≥1 year. The difference in quartiles might suggest that the differences between patients’ performances diverge with the increasing time gap to their training. It is suggested that a yearly training of the controller change should become mandatory to increase patients’ safety. That the pump-off time upper ranges are alarmingly high underlines the relevance of training patients to minimize their pump-off time. By analyzing the scan paths, it was discovered that the deviation from the trained order of controller change led to long pump-off times for some participants. This deviation in order was that they first disconnected both batteries instead of only changing one followed by the pump. Not only an increased regularity of CC training, but also the awareness of the pump-off time and its implication should help to prevent this deviation.
As expected for the control groups, the pre patients had the highest BC and CC task durations, pump-off time and dwell times, while the expert had the lowest. That LVAD supported patients were always performing better than pre patients, allows the conclusion that the provided training was effective. But the fact that experts are still able to perform better than LVAD supported patients shows the possibility for further improvement of the training. Regarding H3, i.e. patients with a pump-off time ≥10 s have a higher dwell time on the irrelevant controller than on the relevant controller, no significant results were found. Although the dwell time on the irrelevant controller is smaller than on the relevant controller for every group, the attention spent on the irrelevant controller is too high. The results for the expert group show an approximate ratio of 1:3 between the DT on the irrelevant versus the relevant controller, demonstrating the amount of attention needed for the irrelevant controller.
Contemplating limitations, the criteria of 10 s to separate patients through their pump-off time does not allow conclusions on the physical state of the patients after their pump is off. The patients’ ventricular function strongly influences at which point in time they lose consciousness. Three pre patients had undergone surgery to receive HVAD devices after the study was finished. For one pre patient the decision was made not to implant any LVAD system due to incapability of managing the device. Observations from the study were used in the respective discussions. Although this observation is anecdotal it points to the fact that eye tracking could be used as a tool for patient assessment and to objectively measure their interaction with the system.
Agreeing with Schlögelhofer and Schima’s [13] statement that safety flaws in battery and pump connection to VAD controllers still exist, the study implies that further improvements should be made to increase the usability of LVAD systems in particular with regard to the controller interface. A simple modification would be to add markers to both controllers, providing a clear guidance in case of a controller change emergency. Dunn et al. [11] states resources how to further improve LVAD design for human factors. The next step for further research would be to evaluate and compare multiple LVAD devices regarding their design and user experience.
To improve the training effect a regular repetition (once a year) of an emergency controller change, is suggested. The results of our study underline the statement of Kormos et al. [10] that although non-pump related device malfunctions occur infrequently, it is highly relevant to carefully prepare patients, caregivers, allied medical personnel and physicians. This was confirmed by a study with paramedics, that came to the conclusion that LVAD device usability and training for emergencies must be improved because the LVAD handling is not intuitive enough [14]. Our study is the first one demonstrating the use of eye tracking for the assessment of the usability and handling of LVAD systems. Its use may be expanded to other critical medical devices in the in-hospital or ambulatory setting.
In conclusion, eye tracking was successfully used to quantitatively and objectively measure the patient’s performance in changing the battery or controller of a training HVAD. It is a valuable tool to control the success of the HVAD handling training. This training shows to be effective, but regular re-training of emergency scenarios will add safety. The gaze point during BC and CC should be analyzed by the eye tracking system during training and on regular basis during follow up to monitor a constant level of life important skills. Based on the results and conclusions of this study the institutional HVAD training routine has been improved to increase patients’ safety.
References
Dickstein K, Cohen-Solal A, Filippatos G, McMurray JJ V, Ponikowski P, Poole-Wilson PA, et al. ESC guidelines for the diagnosis and treatment of acute and chronic heart failure 2008: The task force for the diagnosis and treatment of acute and chronic heart failure 2008 of the European Society of Cardiology. Develop. Eur. Heart. J. 2008;29:2388–442. https://doi.org/10.1093/eurheartj/ehn309.
Gustafsson F, Rogers JG. Left ventricular assist device therapy in advanced heart failure: Patient selection and clinical outcomes 2017. https://doi.org/10.1002/ejhf.779.
Slaughter MS, Rogers JG, Milano CA, Russell SD, Conte J V., Feldman D, et al. Advanced heart failure treated with continuous-flow left ventricular assist device. N. Engl. J. Med. 2009;361:2241–51. https://doi.org/10.1056/NEJMoa0909938.
Rogers JG, Pagani FD, Tatooles AJ, Bhat G, Slaughter MS, Birks EJ, et al. Intrapericardial left ventricular assist device for advanced heart failure. N. Engl. J. Med. 2017;376:451–60. https://doi.org/10.1056/NEJMoa1602954.
Kirklin JK, Xie R, Cowger J, de By TMMH, Nakatani T, Schueler S, et al. Second annual report from the ISHLT mechanically assisted circulatory support registry. J. Hear. Lung. Transplant. 2018;37:685–91. https://doi.org/10.1016/j.healun.2018.01.1294.
Kirklin JK, Pagani FD, Kormos RL, Stevenson LW, Blume ED, Myers SL, et al. Eighth annual INTERMACS report: Special focus on framing the impact of adverse events. J. Hear. Lung. Transplant. 2017;36:1080–6. https://doi.org/10.1016/j.healun.2017.07.005.
Kormos RL, Cowger J, Pagani FD, Teuteberg JJ, Goldstein DJ, Jacobs JP, et al. The society of thoracic surgeons intermacs database annual report: Evolving indications, outcomes, and scientific partnerships. Ann. Thorac. Surg. 2019;107:341–53. https://doi.org/10.1016/j.athoracsur.2018.11.011.
Schmid Daners M, Kaufmann F, Amacher R, Ochsner G, Wilhelm MJ, Ferrari A, et al. Left ventricular assist devices: Challenges toward sustaining long-term patient care. Ann. Biomed. Eng. 2017;45:1836–51. https://doi.org/10.1007/s10439-017-1858-9.
Throckmorton AL, Patel-Raman SM, Fox CS, Bass EJ. Beyond the VAD: Human factors engineering for mechanically assisted circulation in the 21st century. Artif. Organs. 2016;40:539–48. https://doi.org/10.1111/aor.12600.
Kormos RL, McCall M, Althouse A, Lagazzi L, Schaub R, Kormos MA, et al. Left ventricular assist device malfunctions. Circulation. 2017;136:1714–25. https://doi.org/10.1161/circulationaha.117.027360.
Dunn JL, Nusem E, Straker K, Gregory S, Wrigley C. Human factors and user experience issues with ventricular assist device wearable components: A systematic review. Ann. Biomed. Eng. 2019;47:2431–88. https://doi.org/10.1007/s10439-019-02303-3.
Schima H, Schlöglhofer T, Zu Dohna R, Drews T, Morshuis M, Roefe D, et al. Usability of ventricular assist devices in daily experience: A multicenter study. Artif. Organs. 2014;38:751–60. https://doi.org/10.1111/aor.12394.
Schlöglhofer T, Schima H. Wearable systems. 2018. https://doi.org/10.1016/B978-0-12-810491-0.00022-9.
Geidl L, Deckert Z, Zrunek P, Gottardi R, Sterz F, Wieselthaler G, et al. Intuitive use and usability of ventricular assist device peripheral components in simulated emergency conditions. Artif. Organs. 2011;35:773–80.
Lohmeyer Q, Schneider A, Jordi C, Lange J, Meboldt M. Toward a new age of patient centricity? The application of eye-tracking to the development of connected self-injection systems. Expert. Opin. Drug. Deliv. 2019;16:163–75. https://doi.org/10.1080/17425247.2019.1563070.
Wegner S, Lohmeyer Q, Wahlen D, Neumann S, Groebli JC, Meboldt M. Value of eye-tracking data for classification of information processing-intensive handling tasks: Quasi-experimental study on cognition and user interface design. JMIR. Hum. Factors. 2020;7:1–11. https://doi.org/10.2196/15581.
Lohmeyer Q, Meboldt M. The Integration of Quantitative Biometric Measures and Experimental Design Research. Experiment. Springer Berlin, Heidelberg; 2016.
Tien T, Pucher PH, Sodergren MH, Sriskandarajah K, Yang GZ, Darzi A. Eye tracking for skills assessment and training: A systematic review. J. Surg. Res. 2014;191:169–78. https://doi.org/10.1016/j.jss.2014.04.032.
Ashraf H, Sodergren MH, Merali N, Mylonas G, Singh H, Darzi A. Eye-tracking technology in medical education: A systematic review. Med. Teach. 2018;40:62–9. https://doi.org/10.1080/0142159X.2017.1391373.
Erridge S, Ashraf H, Purkayastha S, Darzi A, Sodergren MH. Comparison of gaze behaviour of trainee and experienced surgeons during laparoscopic gastric bypass. Br. J. Surg. 2018;105:287–94. https://doi.org/10.1002/bjs.10672.
Tien T, Pucher PH, Sodergren MH, Sriskandarajah K, Yang GZ, Darzi A. Differences in gaze behaviour of expert and junior surgeons performing open inguinal hernia repair. Surg. Endosc. Other. Interv. Tech. 2015;29:405–13. https://doi.org/10.1007/s00464-014-3683-7.
Zimmermann JM, Vicentini L, Lohmeyer Q, Taramasso M, Maisano F, Meboldt M. Visual behaviour strategies of operators during catheter-based cardiovascular interventions. J. Med. Syst. 44, 2020. https://doi.org/10.1007/s10916-019-1480-5.
Fendler TJ, Spertus JA, Gosch KL, Jones PG, Bruce JM, Nassif ME, et al. Incidence and predictors of cognitive decline in patients with left ventricular assist devices. Circ. Cardiovasc. Qual. Outcomes. 2015;8:285–91. https://doi.org/10.1161/CIRCOUTCOMES.115.001856.
Helton WS, Nöswall K. Short stress state questionnaire: Factor structure and state change assessment. Eur. J. Psychol. Assess. 2010;31:20–30. https://doi.org/10.1027/1015-5759/a000200.
Helton WS. Validation of a short stress state questionnaire. Proc. Hum. Factors. Ergon. Soc. Annu. Meet. 2004;48:1238–42. https://doi.org/10.1177/154193120404801107.
Ziegenfuß T. Notfallmedizin. Springer Berlin, Heidelberg; 2017.
Acknowledgements
The authors express their gratitude to all patients and experts who participated in the eye tracking study. Furthermore, we would like to thank Julia Stein for her statistical support. This work is part of the Zurich Heart project under the umbrella of “University Medicine Zurich”.
Availability of data and material
Not applicable.
Code availability
Not applicable.
Funding
Open Access funding provided by ETH Zurich.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Ethics approval
This study was performed in line with the principles of the Declaration of Helsinki. Approval was granted by the Ethics Committee of Charité – Universitätsmedizin Berlin (EA2/081/19).
Informed consent
Informed consent was obtained from all individual participants included in the study.
Consent for publication
Not applicable since no identifying information is included in this article.
Conflict of interest
The authors declare no conflict of interest.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This article is part of the Topical Collection on Education & Training
Supplementary Information
Video 1
Eye tracking scan path video of HVAD battery and controller change for patients and experts (Video1_ScanPaths.mp4) (MP4 41,025 kb)
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Weiss, K.E., Hoermandinger, C., Mueller, M. et al. Eye Tracking Supported Human Factors Testing Improving Patient Training. J Med Syst 45, 55 (2021). https://doi.org/10.1007/s10916-021-01729-4
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10916-021-01729-4