Visualization and User Interaction Methods for Multiscale Biomedical Data

  • Ricardo Manuel Millán VaqueroEmail author
  • Jan Rzepecki
  • Karl-Ingo Friese
  • Franz-Erich Wolter


The need for handling huge amounts of data from several sources is becoming increasingly important for biomedical scientists. In the past, there were mainly different modalities in imaging techniques that had to be combined. Those modalities usually measured different physical effects from the same object and shared dimensions and resolution. Nowadays, an increasing number of complex use cases exist in biomedical science and clinical diagnostics that require data from various domains, each one related to a different spatiotemporal scale. Multiscale spatial visualization and interaction can help physicians and scientists to explore and understand this data. In the recent years, the number of published articles on efficient scientist-centric visualization and interaction methods has drastically increased. This chapter describes current techniques on multiscale visualization and user interaction and proposes strategies to accommodate current needs in biomedical data analysis.


Multiscale visualization Multiscale interaction Biomedical imaging HCI Virtual reality 



This work was supported by the framework of the EU Marie Curie Project MultiScaleHuman (FP7-PEOPLE-2011-ITN, Grant agreement no.: 289897).


  1. 1.
    Walter, T., et al. (2010). Visualization of image data from cells to organisms. Nature Methods, 7, S26–S41.CrossRefGoogle Scholar
  2. 2.
    Levenson, R. M., & Mansfield, J. R. (2006). Multispectral imaging in biology and medicine: Slices of life. Cytometry Part A, 69A(8), 748–758.CrossRefGoogle Scholar
  3. 3.
    Chiang, Y. J., et al. (2003). Out-of-core algorithms for scientific visualization and computer graphics. IEEE Visualization, 22(1), 35–48.Google Scholar
  4. 4.
    Friese, K.-I., et al. (2013). Analysis of tomographic mineralogical data using YaDiV–Overview and practical case study. Computers and Geosciences, 56, 92–103.CrossRefGoogle Scholar
  5. 5.
    Friese, K. I., et al. (2011). YaDiV–an open platform for 3D visualization and 3D segmentation of medical data. The Visual Computer, 27(2), 129–139.CrossRefGoogle Scholar
  6. 6.
    Auer, M., et al. (2007). Development of multiscale biological image data analysis: Review of 2006 international workshop on multiscale biological imaging, data mining and informatics, Santa Barbara, USA (BII06). BMC Cell Biology, 8(1), S1.CrossRefGoogle Scholar
  7. 7.
    Jmol: an open-source Java viewer for chemical structures in 3D,
  8. 8.
    OsiriX Imaging Software, DICOM sample image sets,
  9. 9.
    Fishman, E. K., & Kuszyk, B. (2001). 3D imaging: Musculoskeletal applications. Critical Reviews in Diagnostic Imaging, 42(1), 59–100.Google Scholar
  10. 10.
    Oden, J. T. et al. (2006). Simulation-based engineering science: Revolutionizing engineering science through simulation,
  11. 11.
    Han, L., et al. (2011). Nanomechanics of the cartilage extracellular matrix. Annual Review of Materials Research, 41, 133.CrossRefGoogle Scholar
  12. 12.
    Testi, D. et al. (2012). New interactive visualisation of multiscale biomedical data. ACM SIGGRAPH 2012 Posters, (pp. 76:1–76:1), ACM, New York.Google Scholar
  13. 13.
    Viceconti, M., et al. (2007). Multimod Data Manager: A tool for data fusion. Computer Methods and Programs in Biomedicine, 87(2), 148–159.CrossRefMathSciNetGoogle Scholar
  14. 14.
    MultiScaleHuman Project (2012). MultiScaleHuman Project,
  15. 15.
    Kleemann, R. U., et al. (2005). Altered cartilage mechanics and histology in knee osteoarthritis: Relation to clinical assessment (ICRS Grade). Osteoarthritis Cartilage, 13(11), 958–963.CrossRefGoogle Scholar
  16. 16.
    Loeuille, D., et al. (2005). Macroscopic and microscopic features of synovial membrane inflammation in the osteoarthritic knee: Correlating magnetic resonance imaging findings with disease severity. Arthritis and Rheumatism, 52(11), 3492–3501.CrossRefGoogle Scholar
  17. 17.
    Liu, W. K., et al. (2006). Immersed finite element method and its applications to biological systems. Computer Methods in Applied Mechanics and Engineering, 195(13–16), 1722–1749.CrossRefzbMATHMathSciNetGoogle Scholar
  18. 18.
    Mcfarlane, N. J. B. et al. (2012). A survey and classification of visualisation in multiscale biomedical applications. Information Visualisation (IV), 2012 16th International Conference on. pp. 561–566.Google Scholar
  19. 19.
    Chen, J., et al. (2005). Grand challenges for multimodal bio-medical systems. IEEE Circuits and Systems Magazine, 5(2), 46–52.CrossRefGoogle Scholar
  20. 20.
    Lorensen, B. (2004). On the death of visualization. Proceedings of the NIH/NSF Fall 2004 Workshop on Visualization Research Challenges.Google Scholar
  21. 21.
    O’Donoghue, S. I., et al. (2010). Visualizing biological data–now and in the future. Nature Methods, 7, S2–S4.CrossRefGoogle Scholar
  22. 22.
    Nielson, G. M., et al. (Eds.). (1997). Scientific visualization, overviews, methodologies, and techniques. Washington: IEEE Computer Society.Google Scholar
  23. 23.
    Huang, N. E., & Shen, S. S. P. (2005). Hilbert-Huang Transform and Its Applications. World Scientific, 5, 1–15.CrossRefGoogle Scholar
  24. 24.
    Helgason, S. (1999). The Radon transform, Springer.Google Scholar
  25. 25.
    Leach, A. R. (2001). Molecular modelling: Principles and applications. Harlow: Pearson Education.Google Scholar
  26. 26.
    Card, S. K., et al. (1999). Readings in information visualization: Using vision to think. San Francisco, CA: Morgan Kaufmann.Google Scholar
  27. 27.
    Johnson, C. (2004). Top scientific visualization research problems. IEEE Computer Graphics and Applications, 24(4), 13–17.CrossRefGoogle Scholar
  28. 28.
    Van Wijk, J. J. (2005). The value of visualization. Proceedings of the 16th Conference IEEE Visualization (VIS 05), pp. 79–86.Google Scholar
  29. 29.
    Evanko, D. (2010). Supplement on visualizing biological data. Nature Methods, 7, S1–S1.CrossRefGoogle Scholar
  30. 30.
    Rhyne, T.-M. (2003). Does the difference between information and scientific visualization really matter? IEEE Computer Graphics and Applications, 23(3), 6–8.CrossRefGoogle Scholar
  31. 31.
    Rhyne, T. M. et al. (2003). Information and scientific visualization: Separate but equal or happy together at last. Proceedings of the 14th IEEE Visualization, p. 115.Google Scholar
  32. 32.
    Healey, C. G., & Enns, J. T. (1998). On the use of perceptual cues & Data mining for effective visualization of scientific datasets. In Proceedings Graphics, Interface, pp. 177–184.Google Scholar
  33. 33.
    Kosara, R., et al. (2003). Thoughts on user studies: Why, how, and when. IEEE Computer Graphics and Applications, 23(4), 20–25.CrossRefGoogle Scholar
  34. 34.
    Tory, M., & Moller, T. (2004). Human factors in visualization research. IEEE Transactions on Visualization and Computer Graphics, 10(1), 72–84.CrossRefGoogle Scholar
  35. 35.
    Damle, A. (2002). Explain me visually: Exploring information design through multimedia. Information visualisation, 2002. Proceedings of the Sixth International Conference on, pp. 265–267.Google Scholar
  36. 36.
    Springmeyer, R. R., et al. (1992). A characterization of the scientific data analysis process. Proceedings of the 3rd Conference on Visualization’92, pp. 235–242.Google Scholar
  37. 37.
    Ibrahim, N., & Noor, N. F. M. (2004). Navigation technique in 3D information visualisation. Proceedings IEEE Region 10 Conference TENCON 2004, pp. 379–382.Google Scholar
  38. 38.
    Bajaj, C., et al. (2003). Volumetric feature extraction and visualization of tomographic molecular imaging. Journal of Structural Biology, 144(1–2), 132–143.CrossRefGoogle Scholar
  39. 39.
    Yu, Z., & Bajaj, C. (2004). A fast and adaptive method for image contrast enhancement. Proceedings of 2004 IEEE International Conference on Image Processing (ICIP’04). Vol., 2, 1001–1004.Google Scholar
  40. 40.
    Bajaj, C., & Goswami, S. (2009). Modeling cardiovascular anatomy from patient-specific imaging. In J. M. R. S. Tavares & R. M. N. Jorge (Eds.), Advances in computational vision and medical image processing (pp. 1–28). Netherlands: Springer.CrossRefGoogle Scholar
  41. 41.
    Johnson, C. R., & Sanderson, A. R. (2003). A next step: Visualizing errors and uncertainty. IEEE Computer Graphics and Applications, 23(5), 6–10.CrossRefGoogle Scholar
  42. 42.
    Westerhoff, H. (2012). White paper: VPH, Molecular Systems Biology (MSB), and their ITFoM,
  43. 43.
    O’Donoghue, S. I., et al. (2004). The SRS 3D module: Integrating structures, sequences and features. Bioinformatics, 20(15), 2476–2478.CrossRefGoogle Scholar
  44. 44.
    Rhead, B., et al. (2010). The UCSC Genome Browser database: Update 2010. Nucleic Acids Research, 38(1), D613–D619.CrossRefGoogle Scholar
  45. 45.
    Gehlenborg, N., et al. (2010). Visualization of omics data for systems biology. Nature Methods, 7, S56–S68.CrossRefGoogle Scholar
  46. 46.
    McFarlane, N., et al. (2012). Report on best practice, Multiscale Spatio-temporal Visualisation Project.Google Scholar
  47. 47.
    Hansen, C. D., & Johnson, C. R. (2005). Visualization handbook. San Diego: Academic Press.Google Scholar
  48. 48.
    Luebke, D., et al. (2002). Level of detail for 3D graphics. San Francisco: Morgan Kaufmann.Google Scholar
  49. 49.
    Staadt, O. G., et al. (2007). Interactive processing and visualization of image data for biomedical and life science applications. BMC Cell Biology, 8(1), S10.CrossRefGoogle Scholar
  50. 50.
    Biodigital Human (2012). Biodigital Human,
  51. 51.
    Zygote Body (2012). Zygote Body
  52. 52.
    Hunter, P., et al. (2010). A vision and strategy for the virtual physiological human in 2010 and beyond. Philosophical Transactions of The Royal Society: A Mathematical Physical and Engineering Sciences, 368(1920), 2595–2614.Google Scholar
  53. 53.
    Testi, D., et al. (2011). Interactive visualization of multiscale biomedical data: An integrated approach. Proceedings of the 1st IEEE Symposium on Biological Data Visualization (BioVis), pp. 3–4.Google Scholar
  54. 54.
    Visualization Toolkit (2012). Visualization Toolkit,
  55. 55.
    Caban, J. J., et al. (2007). Rapid development of medical imaging tools with open-source libraries. Journal of Digital Imaging, 20(Suppl 1), 83–93.CrossRefGoogle Scholar
  56. 56.
    Leardini, A., et al. (2005). Advanced multimodal visualisation of clinical gait and fluoroscopy analyses in the assessment of total knee replacement. Computer Methods and Programs in Biomedicine, 79(3), 227–240.CrossRefGoogle Scholar
  57. 57.
    Karray, F., et al. (2008). Human-Computer Interaction: Overview on state of the art. International Journal on Smart Sensing and Intelligent Systems, 1(1), 137–159.Google Scholar
  58. 58.
    McNamara, N., & Kirakowski, J. (2006). Functionality, usability, and user experience: Three areas of concern. Interactions., 13(6), 26–28.CrossRefGoogle Scholar
  59. 59.
    St Amant, R., & Riedl, M. O. (2001). A perception/action substrate for cognitive modeling in HCI. International Journal of Human-Computer Studies, 55(1), 15–39.CrossRefzbMATHGoogle Scholar
  60. 60.
    Cutrell, E., & Tan, D. (2008). BCI for passive input in HCI. Proceedings of CHI.Google Scholar
  61. 61.
    Picard, R. W. (1999). Affective computing for HCI. Proceedings of HCI International (8th International Conference on Human-Computer Interaction): Ergonomics and User Interfaces, pp. 829–833.Google Scholar
  62. 62.
    Myers, B. A. (1998). A brief history of human-computer interaction technology. Interactions, 5(2), 44–54.Google Scholar
  63. 63.
    Sutherland, I. E. (1968). A head-mounted three dimensional display. Proceedings of Fall Joint Computer Conf., pp. 757–764, Washington: Thompson Books.Google Scholar
  64. 64.
    Friedewald, M., & Raabe, O. (2011). Ubiquitous computing: An overview of technology impacts. Telematics Informatics, 28(2), 55–65.CrossRefGoogle Scholar
  65. 65.
    Maybury, M. (1998). Intelligent user interfaces: an introduction. Proceedings of the 4th International Conference on Intelligent User interfaces, pp. 3–4.Google Scholar
  66. 66.
    How, Y., & Kan. M. Y. (2005). Optimizing predictive text entry for short message service on mobile phones. Proceedings of HCII.Google Scholar
  67. 67.
    Jaimes, A., & Sebe, N. (2007). Multimodal human-computer interaction: A survey. Computer Vision and Image Understanding, 108(1–2), 116–134.CrossRefGoogle Scholar
  68. 68.
    Hjelmås, E., & Low, B. K. (2001). Face detection: A survey. Computer Vision and Image Understanding, 83(3), 236–274.CrossRefzbMATHGoogle Scholar
  69. 69.
    Herda, L., et al. (2000). Skeleton-based motion capture for robust reconstruction of human motion. In Proceedings of Computer Animation, 2000, 77–83.Google Scholar
  70. 70.
    Lange, B., et al. (2011). Markerless full body tracking: Depth-sensing technology within virtual environments. Simulation and Education Conference (I/ITSEC) : The Interservice/Industry Training.Google Scholar
  71. 71.
    Mitra, S., & Acharya, T. (2007). Gesture recognition: A survey. IEEE Transactions on Systems, Man and Cybernetics, Part C: Applications and Reviews, 37(3), 311–324.Google Scholar
  72. 72.
    Jacob, R. J. K., & Karn, K. S. (2003). Eye tracking in human-computer interaction and usability research: Ready to deliver the promises. Mind, 2(3), 4.Google Scholar
  73. 73.
    Bennett, I. M., et al. (2003). Distributed realtime speech recognition system.Google Scholar
  74. 74.
    Zwyssig, E., et al. (2012). Determining the number of speakers in a meeting using microphone array features. 2012 IEEE International Conference on Acoustics Speech and, Signal Processing (ICASSP), (pp. 4765–4768).Google Scholar
  75. 75.
    Vogt, T., et al. (2008). EmoVoice–a framework for online recognition of emotions from voice. In E. André (Ed.), Perception in multimodal dialogue systems (pp. 188–199). Berlin Heidelberg: Springer.CrossRefGoogle Scholar
  76. 76.
    Hume, S. (2001). Pen-based computing. Applied Clinical Trials, 10(7), 32.Google Scholar
  77. 77.
    Ueberle, M., et al. (2009). Haptic feedback systems for virtual reality and telepresence applications. Feedback, 56, 97.Google Scholar
  78. 78.
    Okamura, A. M. (2009). Haptic feedback in robot-assisted minimally invasive surgery. Current Opinion in Urology, 19(1), 102.CrossRefGoogle Scholar
  79. 79.
    McCrae, J., et al. (2009). Multiscale 3D navigation. Proceedings of the 2009 Symposium on Interactive 3D Graphics and Games, p. 714.Google Scholar
  80. 80.
    Darken, R. P., & Peterson, B. (2002). Spatial orientation, wayfinding, and representation (pp. 493–518). Handbook of Virtual Environments, Mahwah NJ : Lawrence Erlbaum Associates.Google Scholar
  81. 81.
    McFarlane, N. J. B., et al. (2008). 3D Multiscale visualisation for medical datasets. BioMedical Visualization, 2008. MEDIVIS’08. Fifth International Conference. pp. 47–52.Google Scholar
  82. 82.
    Hu, Z., et al. (2009). VisANT 3.5: multi-scale network visualization, analysis and inference based on the gene ontology. Nucleic Acids Research, 37(suppl 2), W115–W121.CrossRefGoogle Scholar
  83. 83.
    Gene Ontology Project (2012). Gene Ontology Project,
  84. 84.
    Catalano, C. E., et al. (2011). Semantics and 3D media: Current issues and perspectives. Computers and Graphics, 35(4), 869–877.CrossRefMathSciNetGoogle Scholar
  85. 85.
    Keim, D., et al. (2008). Visual analytics: Definition, process, and challenges. In A. Kerren (Ed.), Information Visualization (pp. 154–175). Berlin Heidelberg: Springer.CrossRefGoogle Scholar
  86. 86.
    Bowman, D. A., et al. (2008). 3D user interfaces: New directions and perspectives. IEEE Computer Graphics and Applications, 28(6), 20–36.CrossRefGoogle Scholar
  87. 87.
    Hanson, A. J., & Wernert, E. A. (1997). Constrained 3D navigation with 2D controllers. Proceedings of Visualization ’97, pp. 175–182.Google Scholar
  88. 88.
    Azuma, R. T. (1997). A survey of augmented reality. Presence: Teleoperators and Virtual Environments, 6(4), 355–385.Google Scholar
  89. 89.
    Hodson, H. (2012). Wearable gadgets enhance symbiosis of man and machine. New Scientist, 216(2886), 22.CrossRefGoogle Scholar
  90. 90.
    Vlasov, R., et al. (2013). Haptic rendering of volume data with collision detection guarantee using path finding. In Transactions on Computational Science XVIII (pp. 212–231). Berlin Heidelberg: Springer.Google Scholar
  91. 91.
    Vlasov, R., et al. (2012). Haptic rendering of volume data with collision determination guarantee using ray casting and implicit surface representation. 2012 International Conference on Cyberworlds (CW), pp. 91–98.Google Scholar
  92. 92.
    Vlasov, R., et al. (2012). Ray casting for collision detection in haptic rendering of volume data. Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games. (pp. 215–215), New York: ACM.Google Scholar
  93. 93.
    Abásolo, M. J., & Della, J. M. (2007). Magallanes: 3d navigation for everybody. Proceedings of the 5th international conference on Computer graphics and interactive techniques in Australia and Southeast, Asia. pp. 135–142.Google Scholar

Copyright information

© Springer-Verlag London 2014

Authors and Affiliations

  • Ricardo Manuel Millán Vaquero
    • 1
    Email author
  • Jan Rzepecki
    • 1
  • Karl-Ingo Friese
    • 1
  • Franz-Erich Wolter
    • 1
  1. 1.Welfenlab, Division of Computer GraphicsLeibniz Universität HannoverHanoverGermany

Personalised recommendations