Journal of Visualization

, Volume 22, Issue 5, pp 991–1003 | Cite as

Deep learning-based viewpoint recommendation in volume visualization

  • Changhe Yang
  • Yanda Li
  • Can Liu
  • Xiaoru YuanEmail author
Regular Paper


Viewpoint is vital in guiding the user to understand the volume data. However, a model that can recommend viewpoints conforming to user preference is hard to be represented explicitly. In this work, we propose an implicit model for the best viewpoint recommendation of volume visualization with CNN-based models to learn the traditional scoring method and user preference. Residual structures are applied for reducing overfitting in simple scalar regression and solving the problem of accuracy getting lower as the network getting deeper. Multi-level-based structures are applied to imitate the coarse and fine level in human perception. The detailed experiments of comparison between our model and traditional methods confirm the efficiency of our work. A case of application verifies that our model can flexibly realize a user preference-based best viewpoint selection in volume visualization.

Graphic abstract


Scientific visualization Machine learning Computing methodologies 



This work is supported by the National Key Research and Development Program of China (2016QY02D0304), NSFC No. 61672055, and the National Program on Key Basic Research Project (973 Program) No. 2015CB352503.


  1. Behera L, Kumar S, Patnaik A (2006) On adaptive learning rate that guarantees convergence in feedforward networks. IEEE Trans Neural Netw 17(5):1116–1125CrossRefGoogle Scholar
  2. Bianco S, Celona L, Napoletano P, Schettini R (2016) Predicting image aesthetics with deep learning. In: International Conference on advanced concepts for intelligent vision systems, pp 117–125. SpringerGoogle Scholar
  3. Bordoloi UD, Shen H-W (2005) View selection for volume rendering. In: Visualization, 2005. VIS 05. IEEE, pp 487–494. IEEEGoogle Scholar
  4. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: IEEE computer society conference on computer vision & pattern recognition, pp 886–893Google Scholar
  5. Guralnik G, Zemach C, Warnock T (1985) An algorithm for uniform random sampling of points in and on a hypersphere. Inf Process Lett 21(1):17–21MathSciNetCrossRefGoogle Scholar
  6. Hawkins DM (2004) The problem of overfitting. J Chem Inf Comput Sci 44(1):1–12CrossRefGoogle Scholar
  7. He K, Zhang X, Ren S, Sun J (2015) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intell 37(9):1904–1916CrossRefGoogle Scholar
  8. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778, 2016Google Scholar
  9. Hicks J, Wheeling R (1959) An efficient method for generating uniformly distributed points on the surface of an n-dimensional sphere. Commun ACM 2(4):17–19CrossRefGoogle Scholar
  10. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167
  11. Ji G, Shen H-W (2006) Dynamic view selection for time-varying volumes. IEEE Trans Vis Comput Graph 12(5):1109–1116MathSciNetCrossRefGoogle Scholar
  12. Karp RM (1972) Reducibility among combinatorial problems. In: Complexity of computer computations. Springer, pp 85–103Google Scholar
  13. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324CrossRefGoogle Scholar
  14. Levin DT, Simons DJ (1997) Failure to detect changes to attended objects in motion pictures. Psychon Bull Rev 4(4):501–506CrossRefGoogle Scholar
  15. Lin M, Chen Q, Yan S (2013) Network in network. arXiv preprint arXiv:1312.4400
  16. Lindemann F, Ropinski T (2011) About the influence of illumination models on image comprehension in direct volume rendering. IEEE Trans Vis Comput Graph 17(12):1922–1931CrossRefGoogle Scholar
  17. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440Google Scholar
  18. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110CrossRefGoogle Scholar
  19. Lu X, Lin Z, Jin H, Yang J, Wang JZ (2015) Rating image aesthetics using deep learning. IEEE Trans Multimed 17(11):2021–2034CrossRefGoogle Scholar
  20. Mai L, Jin H, Liu F (2016) Composition-preserving deep photo aesthetics assessment. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 497–506Google Scholar
  21. Malu G, Bapi RS, Indurkhya B (2017) Learning photography aesthetics with deep CNNS. arXiv preprint arXiv:1707.03981,
  22. Marsaglia G (2003) Seeds for random number generators. Commun ACM 46(5):90–93CrossRefGoogle Scholar
  23. Ojala T, Pietikainen M, Maenpaa T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24(7):971–987CrossRefGoogle Scholar
  24. Secord A, Lu J, Finkelstein A, Singh M, Nealen A (2011) Perceptual models of viewpoint preference. ACM Trans Graph (TOG) 30(5):109CrossRefGoogle Scholar
  25. Shen H-W, Johnson CR (1995) Sweeping simplices: a fast iso-surface extraction algorithm for unstructured grids. In: Proceedings of the 6th conference on Visualization’95, p 143. IEEE computer societyGoogle Scholar
  26. Shi N, Tao Y (2019) Cnns based viewpoint estimation for volume visualization. ACM Trans Intell Syst Technol (TIST) 10(3):27Google Scholar
  27. Srivastava RK, Greff K, Schmidhuber J (2015) Highway networks. arXiv preprint arxiv:1505.00387
  28. Stanley RP (1975) The Fibonacci lattice. Fibonacci Q 13(3):215–232MathSciNetzbMATHGoogle Scholar
  29. Takahashi S, Fujishiro I, Takeshima Y, Nishita T (2005) A feature-driven approach to locating optimal viewpoints for volume visualization. In: Visualization, 2005. VIS 05. IEEE, pp 495–502. IEEEGoogle Scholar
  30. Tao Y, Wang Q, Chen W, Wu Y, Lin H (2016) Similarity voting based viewpoint selection for volumes. In: Computer graphics forum, vol 35. Wiley, pp 391–400Google Scholar
  31. Vázquez P-P, Feixas M, Sbert M, Heidrich W (2001) Viewpoint selection using viewpoint entropy. VMV 1:273–280Google Scholar
  32. Viola I, Feixas M, Sbert M, Groller ME (2006) Importance-driven focus of attention. IEEE Trans Vis Comput Graph 12(5):933–940CrossRefGoogle Scholar
  33. Viola I, Kanitsar A, Gröller ME (2005) Importance-driven feature enhancement in volume visualization. IEEE Trans Vis Comput Graph 11(4):408–418CrossRefGoogle Scholar
  34. Weaver W (1949) The mathematics of communication. Sci Am 181(1):11–15CrossRefGoogle Scholar
  35. Zhao H, Shi J, Qi X, Wang X, Jia J (2017) Pyramid scene parsing network. In: IEEE Conference on computer vision and pattern recognition (CVPR), pp 2881–2890Google Scholar
  36. Zheng Z, Ahmed N, Mueller K (2011) iView: a feature clustering framework for suggesting informative views in volume visualization. IEEE Trans Vis Comput Graph 17(12):1959–1968CrossRefGoogle Scholar

Copyright information

© The Visualization Society of Japan 2019

Authors and Affiliations

  1. 1.Key Laboratory of Machine Perception (Ministry of Education), and School of EECSPeking UniversityBeijingChina
  2. 2.National Engineering Laboratory for Big Data Analysis and ApplicationPeking UniversityBeijingChina

Personalised recommendations