Advertisement

Towards Advanced Interactive Visualization for Virtual Atlases

  • Noeska SmitEmail author
  • Stefan Bruckner
Chapter
Part of the Advances in Experimental Medicine and Biology book series (AEMB, volume 1156)

Abstract

An atlas is generally defined as a bound collection of tables, charts or illustrations describing a phenomenon. In an anatomical atlas for example, a collection of representative illustrations and text describes anatomy for the purpose of communicating anatomical knowledge. The atlas serves as reference frame for comparing and integrating data from different sources by spatially or semantically relating collections of drawings, imaging data, and/or text. In the field of medical image processing, atlas information is often constructed from a collection of regions of interest, which are based on medical images that are annotated by domain experts. Such an atlas may be employed, for example, for automatic segmentation of medical imaging data. The combination of interactive visualization techniques with atlas information opens up new possibilities for content creation, curation, and navigation in virtual atlases. With interactive visualization of atlas information, students are able to inspect and explore anatomical atlases in ways that were not possible with the traditional method of presenting anatomical atlases in book format, such as viewing the illustrations from other viewpoints. With advanced interaction techniques, it becomes possible to query the data that forms the basis for the atlas, thus empowering researchers to access a wealth of information in new ways. So far, atlas-based visualization has been employed mainly for medical education, as well as biological research. In this survey, we provide an overview of current digital biomedical atlas tasks and applications and summarize relevant visualization techniques. We discuss recent approaches for providing next-generation visual interfaces to navigate atlas data that go beyond common text-based search and hierarchical lists. Finally, we reflect on open challenges and opportunities for the next steps in interactive atlas visualization.

Keywords

Biomedical visualization Virtual atlases Interactive visualization Atlases Visualization 

Notes

Acknowledgments

The research presented in this paper was supported by the Trond Mohn Foundation under grant ID BFS2016TMT01 and the MetaVis project (#250133) funded by the Research Council of Norway.

References

  1. Aljabar P, Heckemann RA, Hammers A, Hajnal JV, Rueckert D (2009) Multi-atlas based segmentation of brain images: atlas selection and its effect on accuracy. NeuroImage 46(3):726–738.  https://doi.org/10.1016/j.neuroimage.2009.02.018 CrossRefPubMedGoogle Scholar
  2. Armit C, Venkataraman S, Richardson L, Stevenson P, Moss J, Graham L, Ross A, Yang Y, Burton N, Rao J et al (2012) eMouseAtlas, EMAGE, and the spatial dimension of the transcriptome. Mamm Genome 23(9–10):514–524.  https://doi.org/10.1007/s00335-012-9407-1 CrossRefPubMedPubMedCentralGoogle Scholar
  3. Bruckner S, Gröller ME (2005) VolumeShop: an interactive system for direct volume illustration. In: Proceedings of IEEE visualization, pp 671–678.  https://doi.org/10.1109/VISUAL.2005.1532856
  4. Bruckner S, Gröller ME (2006) Exploded views for volume data. IEEE Trans Vis Comput Graph 12(5):1077–1084.  https://doi.org/10.1109/TVCG.2006.140 CrossRefPubMedGoogle Scholar
  5. Bruckner S, Gröller ME (2007) Style transfer functions for illustrative volume rendering. Comput Graphics Forum 26(3):715–724.  https://doi.org/10.1111/j.14678659.2007.01095.x CrossRefGoogle Scholar
  6. Bruckner S, Grimm S, Kanitsar A, Gröller ME (2006) Illustrative context-preserving exploration of volume data. IEEE Trans Vis Comput Graph 12(6):1559–1569.  https://doi.org/10.1109/TVCG.2006.96 CrossRefPubMedGoogle Scholar
  7. Bruckner S, Soltészová V, Groller E, Hladuvka J, Buhler K, Jai YY, Dickson BJ (2009) Braingazer – visual queries for neurobiology research. IEEE Trans Vis Comput Graph 15(6):1497–1504.  https://doi.org/10.1109/TVCG.2009.121 CrossRefPubMedGoogle Scholar
  8. Burns M, Klawe J, Rusinkiewicz S, Finkelstein A, DeCarlo D (2005) Line drawings from volume data. ACM Trans Graph 24(3):512–518.  https://doi.org/10.1145/1073204.1073222 CrossRefGoogle Scholar
  9. Cabezas M, Oliver A, Lladó X, Freixenet J, Cuadra MB (2011) A review of atlasbased segmentation for magnetic resonance brain images. Comput Methods Prog Biomed 104(3):e158–e177.  https://doi.org/10.1016/j.cmpb.2011.07.015 CrossRefGoogle Scholar
  10. Cammarano M, Dong X, Chan B, Klingner J, Talbot J, Halevey A, Hanrahan P (2007) Visualization of heterogeneous data. IEEE Trans Vis Comput Graph 13(6):1200–1207.  https://doi.org/10.1109/TVCG.2007.70617 CrossRefPubMedGoogle Scholar
  11. Chintalapani G, Ellingsen LM, Sadowsky O, Prince JL, Taylor RH (2007) Statistical atlases of bone anatomy: construction, iterative improvement and validation. In: International conference on medical image computing and computer-assisted intervention, Springer, pp 499–506.  https://doi.org/10.1007/978-3-540-75757-3_61
  12. Clark K, Vendt B, Smith K, Freymann J, Kirby J, Koppel P, Moore S, Phillips S, Maffitt D, Pringle M et al (2013) The Cancer imaging archive (TCIA): maintaining and operating a public information repository. J Digit Imaging 26(6):1045–1057.  https://doi.org/10.1007/s10278-013-9622-7 CrossRefPubMedPubMedCentralGoogle Scholar
  13. Clarkson MD (2016) Representation of anatomy in online atlases and databases: a survey and collection of patterns for interface design. BMC Dev Biol 16(1):18.  https://doi.org/10.1186/s12861-016-0116-y CrossRefPubMedPubMedCentralGoogle Scholar
  14. Congote J, Segura A, Kabongo L, Moreno A, Posada J, Ruiz O (2011) Interactive visualization of volumetric data with WebGL in real-time. In: Proceedings of the 16th international conference on 3D web technology, ACM, pp 137–146.  https://doi.org/10.1145/2010425.2010449
  15. Correa C, Silver D, Chen M (2006) Feature aligned volume manipulation for illustration and visualization. IEEE Trans Vis Comput Graph 12(5):1069–1076.  https://doi.org/10.1109/TVCG.2006.144 CrossRefPubMedGoogle Scholar
  16. Danchilla B (2012) Three.Js framework. In: Beginning WebGL for HTML5, Springer, pp 173–203.  https://doi.org/10.1007/978-1-4302-3997-0_7
  17. Diepstraten J, Weiskopf D, Ertl T (2003) Interactive cutaway illustrations. Comput Graphics Forum 22(3):523–532.  https://doi.org/10.1111/1467-8659.t01-3-00700 CrossRefGoogle Scholar
  18. Dogdas B, Stout D, Chatziioannou AF, Leahy RM (2007) Digimouse: a 3D whole body mouse atlas from CT and cryosection data. Phys Med Biol 52(3):577.  https://doi.org/10.1088/0031-9155/52/3/003 CrossRefPubMedPubMedCentralGoogle Scholar
  19. Dumpuri P, Thompson RC, Dawant BM, Cao A, Miga MI (2007) An atlas-based method to compensate for brain shift: preliminary results. Med Image Anal 11(2):128–145.  https://doi.org/10.1016/j.media.2006.11.002 CrossRefPubMedGoogle Scholar
  20. Feiner SK, Seligmann DD (1992) Cutaways and ghosting: satisfying visibility constraints in dynamic 3D illustrations. Vis Comput 8(5):292–302.  https://doi.org/10.1007/BF01897116 CrossRefGoogle Scholar
  21. Gleicher M, Albers D, Walker R, Jusufi I, Hansen CD, Roberts JC (2011) Visual comparison for information visualization. Inf Vis 10(4):289–309.  https://doi.org/10.1177/1473871611416549 CrossRefGoogle Scholar
  22. Hauser H (2006) Generalizing focus+context visualization, Springer, pp 305–327.  https://doi.org/10.1007/3-540-30790-7_18
  23. Hawrylycz MJ, Lein ES, Guillozet-Bongaarts AL, Shen EH, Ng L, Miller JA, Van De Lagemaat LN, Smith KA, Ebbert A, Riley ZL et al (2012) An anatomically comprehensive atlas of the adult human brain transcriptome. Nature 489(7416):391.  https://doi.org/10.1038/nature11405 CrossRefPubMedPubMedCentralGoogle Scholar
  24. Hernell F, Ljung P, Ynnerman A (2010) Local ambient occlusion in direct volume rendering. IEEE Trans Vis Comput Graph 16(4):548–559.  https://doi.org/10.1109/TVCG.2009.45 CrossRefPubMedGoogle Scholar
  25. Jones AR, Overly CC, Sunkin SM (2009) The Allen brain atlas: 5 years and beyond. Nat Rev Neurosci 10(11):821.  https://doi.org/10.1038/nrn2722 CrossRefPubMedGoogle Scholar
  26. Jönsson D, Sundén E, Ynnerman A, Ropinski T (2014) A survey of volumetric illumination techniques for interactive volume rendering. Comput Graphics Forum 33(1):27–51.  https://doi.org/10.1111/cgf.12252 CrossRefGoogle Scholar
  27. Kelc R (2012) Zygote body: a new interactive 3-dimensional didactical tool for teaching anatomy. WebmedCentral ANATOMY 3(1):WMC002903.  https://doi.org/10.9754/journal.wmc.2012.002889 CrossRefGoogle Scholar
  28. Kikinis R, Shenton ME, Iosifescu DV, McCarley RW, Saiviroonporn P, Hokama HH, Robatino A, Metcalf D, Wible CG, Portas CM et al (1996) A digital brain atlas for surgical planning, model-driven segmentation, and teaching. IEEE Trans Vis Comput Graph 2(3):232–241.  https://doi.org/10.1109/2945.537306 CrossRefGoogle Scholar
  29. Kim K, Carlis JV, Keefe DF (2017) Comparison techniques utilized in spatial 3D and 4D data visualizations: a survey and future directions. Comput Graph 67:138–147.  https://doi.org/10.1016/j.cag.2017.05.005 CrossRefGoogle Scholar
  30. Kniss J, Premoze S, Hansen C, Shirley P, McPherson A (2003) A model for volume lighting and modeling. IEEE Trans Vis Comput Graph 9(2):150–162.  https://doi.org/10.1109/TVCG.2003.1196003 CrossRefGoogle Scholar
  31. Krüger J, Schneider J, Westermann R (2006) Clearview: an interactive context preserving hotspot visualization technique. IEEE Trans Vis Comput Graph 12(5):941–948.  https://doi.org/10.1109/TVCG.2006.124 CrossRefGoogle Scholar
  32. Kuß A, Gensel M, Meyer B, Dercksen VJ, Prohaska S (2010) Effective techniques to visualize filament-surface relationships. Comput Graphics Forum 29:1003–1012.  https://doi.org/10.1111/j.1467-8659.2009.01703 CrossRefGoogle Scholar
  33. Lawonn K, Viola I, Preim B, Isenberg T (2018) A survey of surface-based illustrative rendering for visualization. Comput Graphics Forum.  https://doi.org/10.1111/cgf.13322
  34. Lee KKY, Tang WC, Choi KS (2013) Alternatives to relational database: comparison of NoSQL and XML approaches for clinical data storage. Comput Methods Prog Biomed 110(1):99–109.  https://doi.org/10.1016/j.cmpb.2012.10.018 CrossRefGoogle Scholar
  35. Levoy M (1988) Display of surfaces from volume data. IEEE Comput Graph Appl 8(3):29–37.  https://doi.org/10.1109/38.511 CrossRefGoogle Scholar
  36. Li W, Agrawala M, Curless B, Salesin D (2008) Automated generation of interactive 3D exploded view diagrams. ACM Trans Graph 27(3):101:1–101:7.  https://doi.org/10.1145/1360612.1360700 CrossRefGoogle Scholar
  37. Lu A, Morris CJ, Ebert DS, Rheingans P, Hansen C (2002) Non-photorealistic volume rendering using stippling techniques. In: Proceedings of IEEE visualization, pp 211–218.  https://doi.org/10.1109/VISUAL.2002.1183777
  38. Magnus JG, Bruckner S (2018) Interactive dynamic volume illumination with refraction and caustics. IEEE Trans Vis Comput Graph 24(1):984–993.  https://doi.org/10.1109/TVCG.2017.2744438 CrossRefPubMedGoogle Scholar
  39. Miao H, De Llano E, Isenberg T, Gröller ME, Barišić I, Viola I (2018) DimSUM: dimension and scale unifying map for visual abstraction of DNA origami structures. Comput Graphics Forum 37(3):403–413.  https://doi.org/10.1111/cgf.13429 CrossRefGoogle Scholar
  40. Molloy JC (2011) The open knowledge foundation: open data means better science. PLoS Biol 9(12):e1001195.  https://doi.org/10.1371/journal.pbio.1001195 CrossRefPubMedPubMedCentralGoogle Scholar
  41. Netter FH (2017) Atlas of human anatomy E-book. Elsevier Health SciencesGoogle Scholar
  42. Oh SW, Harris JA, Ng L, Winslow B, Cain N, Mihalas S, Wang Q, Lau C, Kuan L, Henry AM et al (2014) A mesoscale connectome of the mouse brain. Nature 508(7495):207.  https://doi.org/10.1038/nature13186 CrossRefPubMedPubMedCentralGoogle Scholar
  43. Potter K, Rosen P, Johnson CR (2012) From quantification to visualization: a taxonomy of uncertainty visualization approaches. In: Uncertainty quantification in scientific computing, Springer, pp 226–249.  https://doi.org/10.1007/978-3-642-326776_15
  44. Preim B, Saalfeld P (2018) A survey of virtual human anatomy education systems. Comput Graph 71:132–153.  https://doi.org/10.1016/j.cag.2018.01.005 CrossRefGoogle Scholar
  45. Qualter J, Sculli F, Oliker A, Napier Z, Lee S, Garcia J, Frenkel S, Harnik V, Triola M (2012) The biodigital human: a web-based 3D platform for medical visualization and education. Stud Health Technol Inform 173:359–361.  https://doi.org/10.3233/978-1-61499-022-2-359 CrossRefPubMedGoogle Scholar
  46. Raj M, Mirzargar M, Preston JS, Kirby RM, Whitaker RT (2016) Evaluating shape alignment via ensemble visualization. IEEE Comput Graph Appl 36(3):60–71.  https://doi.org/10.1109/MCG.2015.70 CrossRefPubMedGoogle Scholar
  47. Rautek P, Bruckner S, Gröller E (2007) Semantic layers for illustrative volume rendering. IEEE Trans Vis Comput Graph 13(6):1336–1343.  https://doi.org/10.1109/TVCG.2007.70591 CrossRefPubMedGoogle Scholar
  48. Shen EH, Overly CC, Jones AR (2012) The Allen human brain atlas: comprehensive gene expression mapping of the human brain. Trends Neurosci 35(12):711–714.  https://doi.org/10.1016/j.tins.2012.09.005 CrossRefPubMedGoogle Scholar
  49. Shneiderman B (1996) The eyes have it: a task by data type taxonomy for information visualizations. In: Proceedings of the IEEE symposium on visual languages, pp 336–343.  https://doi.org/10.1109/VL.1996.545307
  50. Smit N, Kraima A, Jansma D, de Ruiter M, Botha C (2012) A unified representation for the model-based visualization of heterogeneous anatomy data. In: EuroVis short papers, pp 85–89.  https://doi.org/10.2312/PE/EuroVisShort/EuroVisShort2012/085089
  51. Smit N, Hofstede CW, Kraima A, Jansma D, de Ruiter M, Eisemann E, Vilanova A (2016) The Online Anatomical Human: web-based anatomy education. In: Proceedings of the 37th annual conference of the European Association for computer graphics: education papers, Eurographics Association, pp 37–40.  https://doi.org/10.2312/eged.20161025
  52. Smit N, Lawonn K, Kraima A, DeRuiter M, Sokooti H, Bruckner S, Eisemann E, Vilanova A (2017) Pelvis: atlas-based surgical planning for oncological pelvic surgery. IEEE Trans Vis Comput Graph 23(1):741–750.  https://doi.org/10.1109/TVCG.2016.2598826 CrossRefPubMedGoogle Scholar
  53. Svakhine N, Ebert DS, Stredney D (2005) Illustration motifs for effective medical volume illustration. IEEE Comput Graph Appl 25(3):31–39.  https://doi.org/10.1109/MCG.2005.60 CrossRefPubMedGoogle Scholar
  54. Swoboda N, Moosburner J, Bruckner S, Yu JY, Dickson BJ, Bühler K (2017) Visualization and quantification for interactive analysis of neural connectivity in drosophila. Comput Graphics Forum 36(1):160–171.  https://doi.org/10.1111/cgf.12792 CrossRefGoogle Scholar
  55. Taerum T, Sousa MC, Samavati F, Chan S, Mitchell JR (2006) Real-time super resolution contextual close-up of clinical volumetric data. In: Proceedings of EuroVis, pp 347–354.  https://doi.org/10.2312/VisSym/EuroVis06/347-354
  56. Termeer M, Bescós JO, Breeuwer M, Vilanova A, Gerritsen F, Gröller E (2007) CoViCAD: comprehensive visualization of coronary artery disease. IEEE Trans Vis Comput Graph 13(6):1632–1639.  https://doi.org/10.1109/TVCG.2007.70550 CrossRefPubMedGoogle Scholar
  57. Thomas JJ, Cook KA (2006) A visual analytics agenda. IEEE Comput Graph Appl 26(1):10–13.  https://doi.org/10.1109/MCG.2006.5 CrossRefPubMedGoogle Scholar
  58. Tominski C, Gladisch S, Kister U, Dachselt R, Schumann H (2017) Interactive lenses for visualization: an extended survey. Comput Graphics Forum 36(6):173–200.  https://doi.org/10.1111/cgf.12871 CrossRefGoogle Scholar
  59. Trapp M, Schulze F, Bühler K, Liu T, Dickson BJ (2013) 3D object retrieval in an atlas of neuronal structures. Vis Comput 29(12):1363–1373.  https://doi.org/10.1007/s00371-013-0871-8 CrossRefGoogle Scholar
  60. van der Maaten L, Postma E, van den Herik J (2009) Dimensionality reduction: a comparative review. J Mach Learn Res 10:66–71. doi:10.1.1.112.5472Google Scholar
  61. Viola I, Gröller E (2005) Smart visibility in visualization. In: Computational aesthetics, pp 209–216.  https://doi.org/10.2312/COMPAESTH/COMPAESTH05/209-216
  62. Viola I, Isenberg T (2018) Pondering the concept of abstraction in (illustrative) visualization. IEEE Trans Vis Comput Graph 24(9):2573–2588.  https://doi.org/10.1109/TVCG.2017.2747545 CrossRefPubMedGoogle Scholar
  63. Viola I, Kanitsar A, Gröller ME (2005) Importance-driven feature enhancement in volume visualization. IEEE Trans Vis Comput Graph 11(4):408–418.  https://doi.org/10.1109/TVCG.2005.62 CrossRefPubMedGoogle Scholar
  64. Walter T, Shattuck DW, Baldock R, Bastin ME, Carpenter AE, Duce S, Ellenberg J, Fraser A, Hamilton N, Pieper S et al (2010) Visualization of image data from cells to organisms. Nat Methods 7(3s):S26.  https://doi.org/10.1038/nmeth.1431 CrossRefPubMedPubMedCentralGoogle Scholar
  65. Weaver C (2004) Building highly-coordinated visualizations in improvise. In: Proceedings of IEEE InfoVis, pp 159–166.  https://doi.org/10.1109/INFVIS.2004.12
  66. Xu R, Wunsch D (2005) Survey of clustering algorithms. IEEE Trans Neural Netw 16(3):645–678.  https://doi.org/10.1109/TNN.2005.845141 CrossRefPubMedGoogle Scholar
  67. Yeatman JD, Richie-Halford A, Smith JK, Keshavan A, Rokem A (2018) A browserbased tool for visualization and analysis of diffusion MRI data. Nat Commun 9(1):940.  https://doi.org/10.1038/s41467-018-03297-7 CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of InformaticsUniversity of BergenBergenNorway
  2. 2.Mohn Medical Imaging and Visualization CentreHaukeland University HospitalBergenNorway

Personalised recommendations