Skip to main content
SpringerLink
Log in

Sketch-based interactive visualization: a survey

  • Review paper
  • Published:
Journal of Visualization Aims and scope Submit manuscript

Abstract

Interactive visualization has been an active subfield of scientific visualization for a longtime, in which user interfaces have traditionally followed the WIMP (Window, Icon, Menu, Pointer) paradigm. Though functional and powerful, they can also be cumbersome and daunting to a novice user, and exploring complex data requires considerable expertise and effort. A recent trend is toward more easy-to-use and natural user interfaces, which has led to interfaces like sketch-based one for interactive visualization (SIV). The goal is to allow intuitive interactive visualization by sketching in the visualization process, from data filtering to rendering. In this state-of-the-art report, we give an overview of relevant research works related to sketch-based interface in visualization. We discuss how sketch-based interaction takes effect at specific stages of the visualization pipeline. We present a categorization based on the aim of interaction, of which there are seven primary categories: selection, cutting, segmentation, matching, coloring, augmentation, and illustration. What is more, we present important items related to SIV interface design, including SIV system modes, necessary tools and some fundamental principles. This survey also provides an overview of some specific applications of SIV and a discussion of important challenges and open problems for researchers to tackle in the coming years.

Graphical Abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22

Similar content being viewed by others

References

  • 3Dconnexion (2013). http://www.3dconnexion.com

  • Acheson DJ (1990) Elementary fluid dynamics. Oxford University Press, Oxford

    MATH  Google Scholar 

  • Adobe Photoshop (2013). http://www.adobe.com/cn/products/photoshop.com

  • Akers D (2006a) Cinch: a cooperatively designed marking interface for 3D pathway selection. UIST 6:33–42

    Google Scholar 

  • Akers D (2006b) Wizard of oz for participatory design: inventing a gestural interface for 3D selection of neural pathway estimates. In: Proceedings of CHI’06 extended abstracts on human factors in computing systems, ACM pp 454–459

  • Akers D, Sherbondy A, Mackenzie R, Dougherty R, Wandell B (2004) Exploration of the brain’s white matter pathways with dynamic queries

  • Andrews K (2013) Human-computer interaction. Lecture notes, TU Graz

  • Becker RA, Cleveland WS (1987) Brushing scatterplots. Technometrics 29(2):127–142

    Article  MathSciNet  Google Scholar 

  • Blaas J, Botha CP, Post FH (2007) Interactive visualization of multi-field medical data using linked physical and feature-space views. In: Proceedings of EuroVis, pp 123–130

  • Bødker S, Grønbæk K (1991) Cooperative prototyping: users and designers in mutual activity. Int J Man-Mach Stud 34(3):453–478

    Article  Google Scholar 

  • Bruckner S, Groller ME (2005) Volumeshop: an interactive system for direct volume illustration. In: Proceedings of IEEE visualization 2005, pp 671–678

  • Bruyns CD, Senger S (2001) Interactive cutting of 3D surface meshes. Comput Gr 25(4):635–642

    Article  Google Scholar 

  • Bryson S, Levit C (1992) The virtual wind tunnel. IEEE Comput Gr Appl 12(4):25–34

    Article  Google Scholar 

  • Buja A, McDonald JA, Michalak J, Stuetzle W (1991) Interactive data visualization using focusing and linking. In: Proceedings of visualization, IEEE conference on IEEE 1991, vol 419, pp 156–163

  • Burger K, Kruger J, Westermann R (2008) Direct volume editing. IEEE Trans Vis Comput Gr 14(6):1388–1395

    Article  Google Scholar 

  • Bürger K, Schneider J, Kondratieva P, Krger J, Westermann R (2007) Interactive visual exploration of unsteady 3D flows. In: Proceedings of the 9th joint Eurographics/IEEE VGTC conference on visualization, Eurographics Association, pp 251–258

  • Buxton B (2010) Sketching user experiences: getting the design right and the right design: getting the design right and the right design. Morgan Kaufmann, Burlington

    Google Scholar 

  • Card SK, Mackinlay JD, Schneiderman B (1999) Readings in information visualization: using vision to think. Morgan Kaufmann, Burlington

    Google Scholar 

  • CAVE (2013). http://www.mechdyne.com/cave.aspx

  • Chen H-LJ, Samavati FF, Sousa MC, Mitchell JR (2006) Sketch-based volumetric seeded region growing. In: Proceedings of the third Eurographics conference on sketch-based interfaces and modeling, Eurographics Association, pp 123–130

  • Chen H-LJ, Samavati FF, Sousa MC (2008) Gpu-based point radiation for interactive volume sculpting and segmentation. Vis Comput 24(7–9):689–698

    Article  Google Scholar 

  • Cockburn A, Karlson A, Bederson BB (2008) A review of overview+ detail, zooming, and focus+ context interfaces. ACM Comput Surv (CSUR) 41(1):2

    Article  Google Scholar 

  • Cook MT, Agah A (2009) A survey of sketch-based 3D modeling techniques. Interact Comput 21(3):201–211

    Article  Google Scholar 

  • Dahlbäck N, Jönsson A, Ahrenberg L (1993) Wizard of oz studieswhy and how. Knowl Based Syst 6(4):258–266

    Article  Google Scholar 

  • De Araujo B, Jorge J (2003) Blobmaker: free-form modelling with variational implicit surfaces. In: Proceedings of, vol 12. Citeseer, pp 17–26

  • Doleisch H, Hauser H (2002) Smooth brushing for focus+context visualization of simulation data in 3D. In: Proceedings of WSCG, pp 147–154

  • Donna J (2008) Cox. Using the supercomputer to visualize higher dimensions: An artist’s contribution to scientific visualization. Leonardo 41(4):391–400

    Google Scholar 

  • EnSight ( 2013). http://www.ensight.com

  • Force Dimension (2013). http://www.forcedimension.com

  • Fuchs R, Hauser H (2009) Visualization of multi-variate scientific data. In: Proceedings of computer graphics forum, vol 28. Wiley, New York, pp 1670–1690

  • Furnas GW (1986) Generalized fisheye views, vol 17

  • Google Map (2013). http://map.google.com

  • Guimbretiére F, Winograd T (2000) Flowmenu: combining command, text, and data entry. In: Proceedings of the 13th annual ACM symposium on User interface software and technology, pp 213–216

  • Guo HQ, Mao NY, Yuan XR (2011) Wysiwyg (what you see is what you get) volume visualization. IEEE Trans Vis Comput Gr 17(12):2106–2114

    Article  Google Scholar 

  • Hansen CD, Johnson CR (2005) The visualization handbook. Elsevier, New York

    Google Scholar 

  • Hayward V, Astley OR, Cruz-Hernandez M, Grant D, Robles-De-La-Torre G (2004) Haptic interfaces and devices. Sens Rev 24(1):16–29

    Article  Google Scholar 

  • Hibbard B (1999) Top ten visualization problems. SIGGRAPH Comput Gr 33(2):21–22

    Article  Google Scholar 

  • Hinckley K, Baudisch P, Ramos G, Guimbretiere F (2005) Design and analysis of delimiters for selection-action pen gesture phrases in scriboli. In: Proceedings of the SIGCHI conference on human factors in computing systems, ACM, pp 451–460

  • Igarashi T, Hughes JF (2001) A suggestive interface for 3D drawing. In: Proceedings of the 14th annual ACM symposium on user interface software and technology, ACM, pp 173–181

  • Igarashi T, Matsuoka S, Tanaka H (1999) Teddy: a sketching interface for 3D freeform design. In: Proceedings of the 26th annual conference on computer graphics and interactive techniques, ACM Press/Addison-Wesley Publishing Co., pp 409–416

  • Isenberg T, Everts MH, Grubert J, Carpendale S (2008) Interactive exploratory visualization of 2D vector fields. In: Computer graphics forum, vol 27. Wiley, New York pp 983–990

  • Johnson C (2004) Top scientific visualization research problems. IEEE Comput Gr Appl 24(4):13–17

    Article  Google Scholar 

  • Johnson C, Moorhead R, Munzner T, Pfister H, Rheingans P, Yoo TS (2006) Nih/nsf visualization research challenges report. Report, Citeseer

  • Keefe DF, Adviser-Laidlaw DH (2007) Interactive 3D drawing for free-form modeling in scientific visualization and art: tools, methodologies, and theoretical foundations. Brown University, Providence

    Google Scholar 

  • Keefe DF, Acevedo D, Miles J, Drury F, Swartz SM, Laidlaw DH (2008) Scientific sketching for collaborative vr visualization design. Vis Comput Gr IEEE Trans 14(4):835–847

    Article  Google Scholar 

  • Keefe DF (2010) Integrating visualization and interaction research to improve scientific workflows. IEEE Comput Gr Appl 30(2):8–13

    Article  Google Scholar 

  • Keefe DF, Isenberg T (2013) Reimagining the scientific visualization interaction paradigm. Computer 46(5):51–57

    Article  Google Scholar 

  • Keim DA, Mansmann F, Schneidewind J, Ziegler H (2006) Challenges in visual data analysis. In: Proceedings of tenth international conference on information visualization, IEEE 2006 pp 9–16

  • Kindlmann G (2002) Transfer functions in direct volume rendering: Design, interface, interaction. Course notes of ACM SIGGRAPH

  • Konyha Z, Matkovic K, Hauser H (2009) Interactive visual analysis in engineering: a survey. In: Proceedings of Posters at SCCG, pp 31–38

  • Laramee RS, Hauser H, Doleisch H, Vrolijk B, Post FH, Weiskopf D (2004) The state of the art in flow visualization: dense and texture-based techniques. In: Proceedings of computer graphics forum, vol 23. Wiley, New York, pp 203–221

  • Laramee RS, Kosara R (2007) Challenges and unsolved problems. In: Proceedings of lecture notes in computer science, book section 5, vol 4417. Springer, Heidelberg, pp 231–254

  • Li L, Hsieh H-H, Shen H-W (2008) Illustrative streamline placement and visualization. In: Proceedings of visualization symposium, PacificVIS’08, IEEE Pacific, pp 79–86

  • Liu L, Bajaj C, Deasy JO, Low DA, Ju T (2008) Surface reconstruction from non-parallel curve networks. In: Proceedings of computer graphics forum, vol 27. Wiley, New York, pp 155–163

  • Marchesin S, Chen C-K, Ho C, Ma K-L (2010) View-dependent streamlines for 3D vector fields. Vis Comput Gr IEEE Trans 16(6):1578–1586

    Article  Google Scholar 

  • McCormick BH, DeFanti TA, Brown MD (1987) Visualization in scientific computing. Comput Gr 33:247–305

    Google Scholar 

  • McLoughlin T, Laramee RS, Peikert R, Post FH, Chen M (2010) Over two decades of integration-based, geometric flow visualization. In: Proceedings of computer graphics forum, vol 29. Wiley, New York, pp 1807–1829

  • Muigg P, Kehrer J, Oeltze S, Piringer H, Doleisch H, Preim B, Hauser H (2008) A four-level focus+context approach to interactive visual analysis of temporal features in large scientific data. In: Proceedings of computer graphics forum, vol 27. Wiley, New York, pp 775–782

  • Müller W, Schumann H (2002) Visual data mining. NORSIGD Info 2:49–68

    Google Scholar 

  • Olsen L, Samavati FF, Sousa MC, Jorge JA (2009) Sketch-based modeling: a survey. Comput Gr-UK 33(1):85–103

    Article  Google Scholar 

  • Olsen L, Samavati FF, Sousa CM, Jorge J (2008) A taxonomy of modeling techniques using sketch-based interfaces, Eurographics state of the art reports

  • Olwal A, Benko H, Feiner S (2003) Senseshapes: using statistical geometry for object selection in a multimodal augmented reality system. In: Proceedings of the 2nd IEEE/ACM international symposium on mixed and augmented reality, IEEE Computer Society, p 300

  • Owada S, Nielsen F, Igarashi T (2005) Volume catcher. In: Proceedings of the 2005 symposium on interactive 3D graphics and games, ACM, pp 111–116

  • Owada S, Nielsen F, Igarashi T, Haraguchi R, Nakazawa K (2008) Projection plane processing for sketch-based volume segmentation. In: 5th IEEE international symposium on biomedical imaging: from Nano to Macro 2008, ISBI 2008, IEEE 2008, pp 117–120

  • ParaView (2013). http://www.paraview.org

  • Pfister H, Lorensen B, Bajaj C, Kindlmann G, Schroeder W, Avila LS, Raghu KM, Machiraju R, Lee J (2001) The transfer function bake-off. Comput Gr Appl IEEE 21(3):16–22

    Article  Google Scholar 

  • Pike WA, Stasko J, Chang R, O’Connell TA (2009) The science of interaction. Inf Vis 8(4):263–274

    Article  Google Scholar 

  • Pobitzer A, Peikert R, Fuchs R, Schindler B, Kuhn A, Theisel H, Matković K, Hauser H (2011) The state of the art in topology-based visualization of unsteady flow. In: Proceedings of computer graphics forum, vol 30. Wiley, New York, pp 1789–1811

  • Post FH, Vrolijk B, Hauser H, Laramee RS, Doleisch H (2003) The state of the art in flow visualisation: feature extraction and tracking. In: Proceedings of computer graphics forum, vol 22, Wiley, New York, pp 775–792

  • Pühringer N (2009) Sketch-based modelling for volume visualization. Retrieved Jan 2009, vol 5, p 2012

  • Rautek P, Bruckner S, Gröller E, Viola I (2008) Illustrative visualization: new technology or useless tautology? ACM SIGGRAPH Comput Gr 42(3):4

    Google Scholar 

  • Ropinski T, Praßni J, Steinicke F, Hinrichs K (2008) Stroke-based transfer function design. In: Proceedings of the fifth Eurographics/IEEE VGTC conference on point-based graphics, Eurographics Association 2008, pp 41–48

  • Rosanwo O, Petz C, Prohaska S, Hege H-C, Hotz I (2009) Dual streamline seeding. In: Proceedings of visualization symposium, 2009. PacificVis’ 09, IEEE Pacific, pp 9–16

  • Schirski M, Gerndt A, van Reimersdahl T, Kuhlen T, Adomeit P, Lang O, Pischinger S, Bischof C (2003) Vista flowlib-framework for interactive visualization and exploration of unsteady flows in virtual environments. In: Proceedings of the workshop on virtual environments, ACM, pp 77–85

  • Schmidt R, Singh K (2008) Sketch-based procedural surface modeling and compositing using surface trees. In: Proceedings of computer graphics forum, vol 27. Wiley, New York, pp 321–330

  • Schmidt R, Wyvill B, Sousa MC, Jorge JA (2006) Shapeshop: sketch-based solid modeling with blobtrees. In: Proceedings of ACM SIGGRAPH 2006 courses, p 14

  • Schroeder D, Coffey D, Keefe D (2010) Drawing with the flow: a sketch-based interface for illustrative visualization of 2D vector fields. In: Proceedings of the seventh sketch-based interfaces and modeling symposium, Eurographics Association, pp 49–56

  • Sherbondy A, Akers D, Mackenzie R, Dougherty R, Wandell B (2005) Exploring connectivity of the brain’s white matter with dynamic queries. IEEE Trans Vis Comput Gr 11(4):419–430

    Article  Google Scholar 

  • Shneiderman B (1996) The eyes have it: a task by data type taxonomy for information visualizations. In: Proceedings of IEEE symposium on visual languages, IEEE, 1996, pp 336–343

  • Sowell R, Liu L, Ju T, Grimm C, Abraham C, Gokhroo G, Low D (2009) Volume viewer: an interactive tool for fitting surfaces to volume data. In: Proceedings of the 6th Eurographics symposium on sketch-based interfaces and modeling, ACM, pp 141–148

  • Sutherland IE (1963) Sketchpad: a man-machine graphical communication system, Thesis

  • Sutherland IE (1964) Sketch pad: a man-machine graphical communication system. In: Proceedings of the SHARE design automation workshop, ACM on 1964, pp 329–346

  • Sutherland IE (1966) Ten unsolved problems in computer graphics. Datamation 12(5):22–27

    Google Scholar 

  • Taylor II, Russell M (2005) Haptics for scientific visualization. In: Proceedings of ACM SIGGRAPH 2005 courses, pp 174–179

  • Tecplot (2013). http://www.tecplot.com

  • Thomas JJ, Cook KA (2005) Illuminating the path: the research and development agenda for visual analytics. In: Proceedings of IEEE Computer Society Press

  • Tory M, Moller T (2004) Human factors in visualization research. IEEE Trans Vis Comput Gr 10(1):72–84

    Article  Google Scholar 

  • Tzeng F-Y, Lum EB, Ma K-L (2003) A novel interface for higher-dimensional classification of volume data. In: Proceedings of the 14th IEEE visualization 2003 (VIS’03), IEEE Computer Society, p 66

  • Tzeng F-Y, Eric B (2005) An intelligent system approach to higher-dimensional classification of volume data. Vis Comput Gr IEEE Trans 11(3):273–284

    Article  Google Scholar 

  • VisIt (2013). https://wci.llnl.gov/codes/visit/

  • Wacom (2013). http://www.wacom.eu

  • Wei J, Wang C, Yu H, Ma K-L (2010) A sketch-based interface for classifying and visualizing vector fields. In: Proceedings of Pacific visualization symposium (PacificVis), 2010 IEEE, pp 129–136

  • Wigdor D, Wixon D (2011) Brave NUI world: designing natural user interfaces for touch and gesture. Elsevier, New York

    Google Scholar 

  • Wu Y, Qu H (2007) Interactive transfer function design based on editing direct volume rendered images. Vis Comput Gr IEEE Trans 13(5):1027–1040

    Article  Google Scholar 

  • Yu L, Efstathiou K, Isenberg P, Isenberg T (2012) Efficient structure-aware selection techniques for 3D point cloud visualizations with 2D of input. IEEE Trans Vis Comput Gr 18(12):2245–2254

    Article  Google Scholar 

  • Yuan X, Zhang N (2005) Volume cutout. Vis Comput 21(8–10):745–754

    Article  Google Scholar 

  • Zeleznik RC, Herndon KP, Hughes JF (2007) Sketch: an interface for sketching 3D scenes. In: Proceedings of ACM SIGGRAPH 2007 courses, p 19

  • Zhu B, Iwata M, Haraguchi R, Ashihara T, Umetani N, Igarashi T, Nakazawa K (2011) Sketch-based dynamic illustration of fluid systems. In: Proceedings of ACM transactions on graphics (TOG), vol 30, p 134

Download references

Acknowledgments

The authors would like to thank anonymous reviewers at JOV for their comments that helped us to improve the quality of this manuscript. This research is supported by the National Natural Science Foundation of China under Grant No. 61170157, the National Grand Fundamental Research 973 Program of China under Grant No. G2009CB72380, and the Scientific Research Founding Project of NUDT.

Author information

Authors and Affiliations

  1. School of Computer, National University of Defense Technology, Changsha, Hunan, China

    Enya Shen, Sikun Li, Xun Cai, Liang Zeng & Wenke Wang

Authors
  1. Enya Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sikun Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xun Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Liang Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wenke Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Enya Shen.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shen, E., Li, S., Cai, X. et al. Sketch-based interactive visualization: a survey. J Vis 17, 275–294 (2014). https://doi.org/10.1007/s12650-014-0225-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12650-014-0225-2

Keywords

Search

Navigation