Auditory display for fluorescence-guided open brain tumor surgery

  • David BlackEmail author
  • Horst K. Hahn
  • Ron Kikinis
  • Karin Wårdell
  • Neda Haj-Hosseini
Original Article



Protoporphyrin (PpIX) fluorescence allows discrimination of tumor and normal brain tissue during neurosurgery. A handheld fluorescence (HHF) probe can be used for spectroscopic measurement of 5-ALA-induced PpIX to enable objective detection compared to visual evaluation of fluorescence. However, current technology requires that the surgeon either views the measured values on a screen or employs an assistant to verbally relay the values. An auditory feedback system was developed and evaluated for communicating measured fluorescence intensity values directly to the surgeon.


The auditory display was programmed to map the values measured by the HHF probe to the playback of tones that represented three fluorescence intensity ranges and one error signal. Ten persons with no previous knowledge of the application took part in a laboratory evaluation. After a brief training period, participants performed measurements on a tray of 96 wells of liquid fluorescence phantom and verbally stated the perceived measurement values for each well. The latency and accuracy of the participants’ verbal responses were recorded. The long-term memorization of sound function was evaluated in a second set of 10 participants 2–3 and 7–12 days after training.


The participants identified the played tone accurately for 98% of measurements after training. The median response time to verbally identify the played tones was 2 pulses. No correlation was found between the latency and accuracy of the responses, and no significant correlation with the musical proficiency of the participants was observed on the function responses. Responses for the memory test were 100% accurate.


The employed auditory display was shown to be intuitive, easy to learn and remember, fast to recognize, and accurate in providing users with measurements of fluorescence intensity or error signal. The results of this work establish a basis for implementing and further evaluating auditory displays in clinical scenarios involving fluorescence guidance and other areas for which categorized auditory display could be useful.


Fluorescence-guided resection (FGR) Spectroscopy 5-Aminolevulinic acid (5-ALA) Protoporphyrin (PpIX) Surgical navigation Neurosurgery Human–computer interaction User interfaces Sonification LabVIEW 



The authors would like to thank Johan Richter, neurosurgeon at the Department of Neurosurgery in Linköping University and the participants for feedback on the sound system.

Funding   The study was supported by Swedish Childhood Cancer foundation (Grant No. MT 2013-0043), the Cancer network at Linköping University (LiU-cancer), and National Institutes of Health Grants P41 EB015902, P41 EB015898, R01EB014955, and U24CA180918.

Compliance with ethical standards

Conflict of interest

The authors state that they have no conflict of interest.

Ethical approval

No ethical approval is required, as the study was not a type that could affect the subjects physically or psychologically. Data used in Fig. 1 are from a study with ethical approval described in [5].

Informed consent

Informed consent was obtained from all individual participants included in the study.

Supplementary material

Supplementary material 1 (mp4 5384 KB)


  1. 1.
    Stummer W, Pichlmeier U, Meinel T, Wiestler OD, Zanella F, Reulen H-J (2006) Fluorescence-guided surgery with 5-aminolevulinic acid for resection of malignant glioma: a randomised controlled multicentre phase III trial. Lancet Oncol 7(5):392–401. doi: 10.1016/S1470-2045(06)70665-9 CrossRefPubMedGoogle Scholar
  2. 2.
    Haj-Hosseini N, Richter J, Andersson-Engels S, Wårdell K (2010) Optical touch pointer for fluorescence guided glioblastoma resection using 5-aminolevulinic acid. Lasers Surg Med 42(1):9–14. doi: 10.1002/lsm.20868 CrossRefPubMedGoogle Scholar
  3. 3.
    Haj-Hosseini N, Richter J, Hallbeck M, Wårdell K (2015) Low dose 5-aminolevulinic acid: implications in spectroscopic measurements during brain tumor surgery. Photodiagn Photodyn Ther 12(2):209–214CrossRefGoogle Scholar
  4. 4.
    Richter J, Haj-Hosseini N, Andersson-Engel S, Wårdell K (2011) Fluorescence spectroscopy measurements in ultrasonic navigated resection of malignant brain tumors. Lasers Surg Med 43(1):8–14. doi: 10.1002/lsm.21022 CrossRefPubMedGoogle Scholar
  5. 5.
    Richter J, Haj-Hosseini N, Hallbeck M, Wårdell K (2017) Combination of hand-held probe and microscopy for fluorescence guided surgery in the brain tumor marginal zone. Photodiagn Photodyn Ther 18:185–192. doi: 10.1016/j.pdpdt.2017.01.188 CrossRefGoogle Scholar
  6. 6.
    Utsuki S, Oka H, Miyajima Y, Shimizu S, Suzuki S, Fujii K (2008) Auditory alert system for fluorescence-guided resection of gliomas. Technical note. Neurol Med Chir 48(2):95–7. doi: 10.2176/nmc.48.95 CrossRefGoogle Scholar
  7. 7.
    Sanderson P, Watson M, Russell W (2005) Advanced patient monitoring displays: tools for continuous informing. Anesth Analg 101:161168. doi: 10.1213/ CrossRefGoogle Scholar
  8. 8.
    Woerdeman P, Willems P, Noordmans H, van der Sprenkel J (2009) Auditory feedback during frameless image-guided surgery in a phantom model and initial clinical experience. Neurosurgery 110:257–262. doi: 10.3171/2008.3.17431 CrossRefGoogle Scholar
  9. 9.
    Willems P, Noordmans H, van Overbeeke J, Viergever M, Tulleken C, van der Sprenkel J (2005) The impact of auditory feedback on neuronavigation. Acta Neurochirur 147:167–173. doi: 10.1007/s00701-004-0412-3 CrossRefGoogle Scholar
  10. 10.
    Voormolen E, Woerdeman P, van Stralen M, Noordmans H, Viergever M, Regli L, van der Sprenkel J (2012) Validation of exposure visualization and audible distance emission for navigated temporal bone drilling in phantoms. PLoS ONE 7(7):e41262. doi: 10.1371/journal.pone.0041262 CrossRefPubMedPubMedCentralGoogle Scholar
  11. 11.
    Cho B, Oka M, Matsumoto N, Ouchida R, Hong J, Hashizume M (2013) Warning navigation system using real-time safe region monitoring for otologic surgery. Int J Comput Assist Radiol Surg 8(3):395–405. doi: 10.1007/s11548-012-0797-z CrossRefPubMedGoogle Scholar
  12. 12.
    Hansen C, Black D, Lange C, Rieber F, Lamadé W, Donati M, Oldhafer K, Hahn H (2013) Auditory support for resection guidance in navigated liver surgery. Med Robot Comput Assist Surg 9(1):36. doi: 10.1002/rcs.1466 CrossRefGoogle Scholar
  13. 13.
    Bork F, Fuerst B, Schneider A, Pinto F, Graumann C, Navab N (2015) Auditory and visio-temporal distance coding for 3-dimensional perception in medical augmented reality. In: Proceedings of 2015 IEEE international symposium on mixed and augmented reality (ISMAR), pp 7–12. doi: 10.1109/ISMAR.2015.16
  14. 14.
    Black D, Hettig J, Luz M, Hansen C, Kikinis R, Hahn H (2017) Auditory feedback to support image-guided medical needle placement. Int J Comput Assist Radiol Surg. doi: 10.1007/s11548-017-1537-1 PubMedCentralGoogle Scholar
  15. 15.
    Strauß G, Schaller S, Zaminer B, Heininger S, Hofer M, Manzey D, Meixensberger J, Dietz S, Luth T (2010) Klinische Erfahrungen mit einem Kollisionswarnsystem. HNO 59:470479. doi: 10.1007/s00106-010-2237-0 Google Scholar
  16. 16.
    Black D, Hansen C, Nabavi A, Kikinis R, Hahn H (2017) A Survey of auditory display in image-guided interventions. Int J Comput Assist Radiol Surg. doi: 10.1007/s11548-017-1547-z PubMedCentralGoogle Scholar
  17. 17.
    Wright M, Freed A, Momeni A (2003) OpenSound Control: state of the Art 2003. In: Proceedings of 2003 international conference on new interfaces for musical expression (NIME), pp 153–159Google Scholar
  18. 18.
    Haj-Hosseini N, Kistler B, Wårdell K (2014) Development and characterization of a brain tumor mimicking fluorescence phantom. Proc SPIE 8945:894505–894507. doi: 10.1117/12.2039861 CrossRefGoogle Scholar
  19. 19.
    Blattner M, Sumikawa D, Greenberg R (1989) Earcons and icons: their structure and common design principles. Hum Comput Interact 4(1):11–44. doi: 10.1207/s15327051hci0401_1 CrossRefGoogle Scholar
  20. 20.
    DuBois T (1983) Christian Friedrich Daniel Schubart’s Ideen Zu Einer sthetik Der Tonkunst: An Annotated Translation. Doctoral Dissertation, University of Southern California, Los Angeles. pp 1–33Google Scholar
  21. 21.
    Puckette M (1996) Pure data: another integrated computer music environment. In: Second intercollege computer music concerts, 1996, pp 37–41Google Scholar
  22. 22.
    Haahr M. (2016) Website, School of Computer Science and Statistics at Trinity College, Dublin. Accessed Dec 2016
  23. 23.
    Kim A, Khurana M, Moriyama Y, Wilson B (2010) Quantification of in vivo fluorescence decoupled from the effects of tissue optical properties using fiber-optic spectroscopy measurements. J Biomed Opt 15(6):067006. doi: 10.1117/1.3523616
  24. 24.
    Aalders M, Sterenborg H, Stewart F, van der Vange N (2000) Photodetection with 5-aminolevulinic acid-induced protoporphyrin IX in the rat abdominal cavity: drug-dose-dependent fluorescence kinetics. Photochem Photobiol 72(4):521–525. doi: 10.1562/0031-8655(2000)0720521pwaaip2.0.co2 CrossRefPubMedGoogle Scholar
  25. 25.
    Stummer W, Tonn J, Goetz C, Ullrich W, Stepp H, Bink A, Pietsch T, Pichlmeier U (2014) 5-Aminolevulinic acid-derived tumor fluorescence: the diagnostic accuracy of visible fluorescence qualities as corroborated by spectrometry and histology and postoperative imaging. Neurosurgery 74(3):310–320. doi: 10.1227/NEU.0000000000000267 CrossRefPubMedGoogle Scholar
  26. 26.
    Eljamel S, Petersen M, Valentine R, Buist R, Goodman C, Moseley H, Eljamel S (2013) Comparison of intraoperative fluorescence and MRI image guided neuronavigation in malignant brain tumours, a prospective controlled study. Photodiagn Photodyn Ther 10(4):356–361. doi: 10.1016/j.pdpdt.2013.03.006 CrossRefGoogle Scholar
  27. 27.
    Edworthy J, Hellier E (2006) Alarms and human behaviour: implications for medical alarms. Br J Anaesth 97(1):12–17. doi: 10.1093/bja/ael114 CrossRefPubMedGoogle Scholar
  28. 28.
    Parseihian G, Ystad S, Aramaki M, Kronland-Martinet R (2015) The process of sonification design for guidance tasks. J Mob Med 9(2)Google Scholar
  29. 29.
    Mondor T, Finley G (2003) The perceived urgency of auditory warning alarms used in the hospital operating room is inappropriate. Can J Anaesth 50(3):221–228. doi: 10.1007/bf03017788 CrossRefPubMedGoogle Scholar
  30. 30.
    Sanderson P, Wee A, Lacherez P (2006) Learnability and discriminability of melodic medical equipment alarms. Anaesthesia 61(2):142–147. doi: 10.1111/j.1365-2044.2005.04502.x CrossRefPubMedGoogle Scholar
  31. 31.
    Cvach M (2012) Monitor alarm fatigue: an integrative review. Biomed Instrum Technol 46(4):268–277. doi: 10.2345/0899-8205-46.4.268 CrossRefPubMedGoogle Scholar
  32. 32.
    Edworthy J (2013) Medical audible alarms: a review. J Am Med Inform Assoc 20(3):584–589. doi: 10.1136/amiajnl-2012-001061 CrossRefPubMedGoogle Scholar
  33. 33.
    Haj-Hosseini N, Richter J, Milos P, Hallbeck, Wårdell K (2017) Optical guidance for stereotactic brain tumor procedures—preliminary clinical evaluation. PhotonicsWestGoogle Scholar
  34. 34.
    Markwardt N, von Berg A, Fiedler S, Goets M, Haj-Hosseini N, Polzer C, Stepp H, Zelenkov P, Rühm A (2015) Optical Spectroscopy for Stereotactic Biopsy of Brain Tumors. Proc. SPIE 9542, Medical Laser Applications and Laser-Tissue Interactions VII, 954208. doi: 10.1117/12.2183741
  35. 35.
    Dixon B, Daly M, Chan H, Vescan A, Witterick I, Irish J (2013) Surgeons blinded by enhanced navigation: the effect of augmented reality on attention. Surg Endosc 27:454–461. doi: 10.1007/s00464-012-2457-3 CrossRefPubMedGoogle Scholar

Copyright information

© CARS 2017

Authors and Affiliations

  1. 1.Medical Image ComputingUniversity of BremenBremenGermany
  2. 2.Jacobs UniversityBremenGermany
  3. 3.Fraunhofer MEVISBremenGermany
  4. 4.Brigham and Women’s Hospital and Harvard Medical SchoolBostonUSA
  5. 5.Department of Biomedical EngineeringLinköping UniversityLinköpingSweden

Personalised recommendations