Evolving Systems

, Volume 8, Issue 3, pp 193–201 | Cite as

Immersive visualisation of 3-dimensional spiking neural networks

Original Paper

Abstract

Recent development in artificial neural networks has led to an increase in performance, but also in complexity and size. This poses a significant challenge for the exploration and analysis of the spatial structure and temporal behaviour of such networks. Several projects for the 3D visualisation of neural networks exist, but they focus largely on the exploration of the spatial structure alone, and are using standard 2D screens as output and mouse and keyboard as input devices. In this article, we present NeuVis, a framework for an intuitive and immersive 3D visualisation of spiking neural networks in virtual reality, allowing for a larger variety of input and output devices. We apply NeuVis to NeuCube, a 3-dimensional spiking neural network learning framework, significantly improving the user’s abilities to explore, analyse, and also debug the network. Finally, we discuss further venues of development and alternative render methods that are currently under development and will increase the visual accuracy and realism of the visualisation, as well as further extending its analysis and exploration capabilities.

Keywords

Spiking neural network 3-Dimensional Visualisation Virtual reality Immersive 

References

  1. Armstrong JD, van Hemert JI (2009) Towards a virtual fly brain. Philos Transact A Math Phys Eng Sci 367:2387–2397CrossRefGoogle Scholar
  2. Bruckner S, Šoltészová V, Gröller ME, Hladuvka J, Buhler K, Yu JY, Dickson BJ (2009) BrainGazer—visual queries for neurobiology research. IEEE Trans Vis Comput Graph 15(6):1497–1504. doi:10.1109/TVCG.2009.121 CrossRefGoogle Scholar
  3. Foottit J, Brown D, Marks S, Connor A (2014) An intuitive tangible game controller. In: Nesbitt K, Blackmore K (eds) The 10th Australasian conference on interactive entertainment (IE 2014), Newcastle, NSW, Australia. doi:10.1145/2677758.2677774
  4. Furber S, Galluppi F, Temple S, Plana L (2014) The SpiNNaker Project. Proceedings of the IEEE 102(5):652–665. doi:10.1109/JPROC.2014.2304638
  5. HTC Corporation (2016) Vive. https://www.vive.com/anz/
  6. von Kapri A, Rick T, Potjans TC, Diesmann M, Kuhlen T (2011) Towards the visualization of spiking neurons in virtual reality. Stud Health Technol Inf 163:685–687Google Scholar
  7. Kasabov N (2012) NeuCube EvoSpike architecture for spatio-temporal modelling and pattern recognition of brain signals. In: Artificial neural networks in pattern recognition, Springer, pp 225–243. doi:10.1007/978-3-642-33212-8_21
  8. Kasabov N, Scott NM, Tu E, Marks S, Sengupta N, Capecci E, Othman M, Doborjeh MG, Murli N, Hartono R, Espinosa-Ramos JI, Zhou L, Alvi FB, Wang G, Taylor D, Feigin V, Gulyaev S, Mahmoud M, Hou ZG, Yang J (2016) Evolving spatio-temporal data machines based on the neucube neuromorphic framework: design methodology and selected applications. Neural Netw 78:1–14. doi:10.1016/j.neunet.2015.09.011, http://www.sciencedirect.com/science/article/pii/S0893608015001860 (special issue on neural network learning in big data)
  9. Kolasinski EM (1995) Simulator sickness in virtual environments. Tech. rep., Defense Technical Information Center. www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA295861
  10. Lancaster J, Rainey L, Summerlin J, Freitas C, Fox P, Evans A, Toga A, Mazziotta J (1997) Automated labeling of the human brain. Hum Brain Mapp 5(4):238–242. doi:10.1002/(SICI)1097-0193(1997)5:4<238::AID-HBM6>3.0.CO;2-4, http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2860189/
  11. Lancaster JL, Woldorff MG, Parsons LM, Liotti M, Freitas CS, Rainey L, Kochunov PV, Nickerson D, Mikiten SA, Fox PT (2000) Automated Talairach atlas labels for functional brain mapping. Hum Brain Mapp 10(3):120–131CrossRefGoogle Scholar
  12. Lin CY, Tsai KL, Wang SC, Hsieh CH, Chang HM, Chiang AS (2011) The neuron navigator: exploring the information pathway through the neural maze. In: 2011 IEEE Pacific visualization symposium (PacificVis), pp 35–42. doi:10.1109/PACIFICVIS.2011.5742370
  13. Maciel PWC, Shirley P (1995) Visual navigation of large environments using textured clusters. In: Proceedings of the 1995 symposium on interactive 3D graphics, ACM, New York, NY, USA, I3D ’95, pp 95–ff. doi:10.1145/199404.199420, http://doi.acm.org/10.1145/199404.199420
  14. Oculus VR (2015) Best practices guide. Tech. rep., Oculus VR. http://developer.oculus.com/best-practices
  15. Oculus VR (2016) Oculus rift. https://www3.oculus.com/en-us/rift/
  16. Ridder Md, Jung Y, Huang R, Kim J, Feng DD (2015) Exploration of virtual and augmented reality for visual analytics and 3d volume rendering of functional magnetic resonance imaging (fMRI) data. In: Big data visual analytics (BDVA), 2015, pp 1–8. doi:10.1109/BDVA.2015.7314293
  17. Scott N, Kasabov N, Indiveri G (2013) NeuCube neuromorphic framework for spatio-temporal brain data and its python implementation. In: Neural information processing, Springer, pp 78–84. doi:10.1007/978-3-642-42051-1_11
  18. Sherif T, Kassis N, Rousseau ME, Adalat R, Evans AC (2015) BrainBrowser: distributed, web-based neurological data visualization. Front Neuroinf 8:89. doi:10.3389/fninf.2014.00089, http://journal.frontiersin.org/article/10.3389/fninf.2014.00089/full
  19. Wejchert J, Tesauro G (1990) Neural network visualization. In: Touretzky DS (ed) Advances in neural information processing systems 2, Morgan Kaufmann Publishers Inc., San Francisco, pp 465–472. http://dl.acm.org/citation.cfm?id=109230.109287
  20. Xia M, Wang J, He Y (2013) BrainNet Viewer: a network visualization tool for human brain connectomics. PloS One 8(7):e68,910. doi:10.1371/journal.pone.0068910, http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0068910

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  1. 1.Auckland University of TechnologyAucklandNew Zealand

Personalised recommendations