Advertisement

Crowdsourcing for Information Visualization: Promises and Pitfalls

  • Rita Borgo
  • Bongshin Lee
  • Benjamin Bach
  • Sara Fabrikant
  • Radu Jianu
  • Andreas Kerren
  • Stephen Kobourov
  • Fintan McGee
  • Luana Micallef
  • Tatiana von Landesberger
  • Katrin Ballweg
  • Stephan Diehl
  • Paolo Simonetto
  • Michelle Zhou
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10264)

Abstract

Crowdsourcing offers great potential to overcome the limitations of controlled lab studies. To guide future designs of crowdsourcing-based studies for visualization, we review visualization research that has attempted to leverage crowdsourcing for empirical evaluations of visualizations. We discuss six core aspects for successful employment of crowdsourcing in empirical studies for visualization – participants, study design, study procedure, data, tasks, and metrics & measures. We then present four case studies, discussing potential mechanisms to overcome common pitfalls. This chapter will help the visualization community understand how to effectively and efficiently take advantage of the exciting potential crowdsourcing has to offer to support empirical visualization research.

References

  1. 1.
    Adams, F.M., Osgood, C.E.: A cross-cultural study of the affective meanings of color. J. Cross Cult. Psychol. 4(2), 135–156 (1973)CrossRefGoogle Scholar
  2. 2.
    Aigner, W., Hoffmann, S., Rind, A.: EvalBench: a software library for visualization evaluation. Comput. Graph. Forum 32(3pt1), 41–50 (2013)CrossRefGoogle Scholar
  3. 3.
    Aigner, W., Miksch, S., Schumann, H., Tominski, C.: Visualization of Time-Oriented Data. Human-Computer Interaction. Springer, London (2011). doi: 10.1007/978-0-85729-079-3 CrossRefGoogle Scholar
  4. 4.
    Albuquerque, G., Lowe, T., Magnor, M.: Synthetic generation of high-dimensional datasets. IEEE Trans. Vis. Comput. Graph. 17(12), 2317–2324 (2011)CrossRefGoogle Scholar
  5. 5.
    Alsallakh, B., Micallef, L., Aigner, W., Hauser, H., Miksch, S., Rodgers, P.: Visualizing sets and set-typed data: state-of-the-art and future challenges. In: Eurographics conference on Visualization (EuroVis)-State of The Art Reports, pp. 1–21 (2014)Google Scholar
  6. 6.
    Alvarez-Garcia, S., Baeza-Yates, R., Brisaboa, N.R., Larriba-Pey, J., Pedreira, O.: Graphgen: a tool for automatic generation of multipartite graphs from arbitrary data. In: 2012 Eighth Latin American Web Congress (LA-WEB), pp. 87–94. IEEE (2012)Google Scholar
  7. 7.
    Álvarez-García, S., Baeza-Yates, R., Brisaboa, N.R., Larriba-Pey, J.L., Pedreira, O.: Automatic multi-partite graph generation from arbitrary data. J. Syst. Softw. 94, 72–86 (2014)CrossRefGoogle Scholar
  8. 8.
    Amar, R., Eagan, J., Stasko, J.: Low-level components of analytic activity in information visualization. In: IEEE Symposium on Information Visualization (INFOVIS 2005), pp. 111–117. IEEE (2005)Google Scholar
  9. 9.
    Andrews, K., Kasanicka, J.: A comparative study of four hierarchy browsers using the hierarchical visualisation testing environment (HVTE). In: 11th International Conference Information Visualization (IV 2007), pp. 81–86. IEEE (2007)Google Scholar
  10. 10.
    Andrienko, G., Andrienko, N.: Privacy issues in geospatial visual analytics. In: Gartner, G., Ortag, F. (eds.) Advances in Location-Based Services. Lecture Notes in Geoinformation and Cartography. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-24198-7_16 Google Scholar
  11. 11.
    Archambault, D., Purchase, H.C.: The mental map and memorability in dynamic graphs. In: Pacific Visualization Symposium (PacificVis), pp. 89–96. IEEE (2012)Google Scholar
  12. 12.
    Archambault, D., Purchase, H.C.: Mental map preservation helps user orientation in dynamic graphs. In: Didimo, W., Patrignani, M. (eds.) GD 2012. LNCS, vol. 7704, pp. 475–486. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-36763-2_42 CrossRefGoogle Scholar
  13. 13.
    Bach, B., Dragicevic, P., Archambault, D., Hurter, C., Carpendale, S.: A descriptive framework for temporal data visualizations based on generalized space-time cubes. Comput. Graph. Forum (2016). http://dx.doi.org/10.1111/cgf.12804
  14. 14.
    Bach, B., Spritzer, A., Lutton, E., Fekete, J.-D.: Interactive random graph generation with evolutionary algorithms. In: Didimo, W., Patrignani, M. (eds.) GD 2012. LNCS, vol. 7704, pp. 541–552. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-36763-2_48 CrossRefGoogle Scholar
  15. 15.
    Bateman, S., Mandryk, R.L., Gutwin, C., Genest, A., McDine, D., Brooks, C.: Useful junk? The effects of visual embellishment on comprehension and memorability of charts. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2573–2582. ACM (2010)Google Scholar
  16. 16.
    Berlin, B., Kay, P.: Basic Color Terms. University of California Press, Berkeley (1969)Google Scholar
  17. 17.
    Bertin, J.: Sémiologie graphique: Les diagrammes-Les réseaux-Les cartes. Gauthier-VillarsMouton & Cie (1973)Google Scholar
  18. 18.
    Borgo, R., Abdul-Rahman, A., Mohamed, F., Grant, P.W., Reppa, I., Floridi, L., Chen, M.: An empirical study on using visual embellishments in visualization. IEEE Trans. Vis. Comput. Graph. 18(12), 2759–2768 (2012)CrossRefGoogle Scholar
  19. 19.
    Boy, J., Rensink, R.A., Bertini, E., Fekete, J.D.: A principled way of assessing visualization literacy. IEEE Trans. Vis. Comput. Graph. 20(12), 1963–1972 (2014)CrossRefGoogle Scholar
  20. 20.
    Boy, J., Detienne, F., Fekete, J.D.: Storytelling in information visualizations: does it engage users to explore data? In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1449–1458. ACM (2015)Google Scholar
  21. 21.
    Brehmer, M., Munzner, T.: A multi-level typology of abstract visualization tasks. IEEE Trans. Vis. Comput. Graph. 19(12), 2376–2385 (2013)CrossRefGoogle Scholar
  22. 22.
    Bremm, S., Von Landesberger, T., Heß, M., Fellner, D.: PCDC-on the highway to data-a tool for the fast generation of large synthetic data sets. In: EuroVis Workshop on Visual Analytics, pp. 7–11 (2012)Google Scholar
  23. 23.
    Brewer, C.A., MacEachren, A.M., Pickle, L.W., Herrmann, D.: Mapping mortality: evaluating color schemes for choropleth maps. Ann. Assoc. Am. Geograph. 87(3), 411–438 (1997)CrossRefGoogle Scholar
  24. 24.
    Brinkmann, G., McKay, B.D.: Fast generation of planar graphs. MATCH Commun. Math. Comput. Chem. 58(2), 323–357 (2007)MATHMathSciNetGoogle Scholar
  25. 25.
    Bristor, V.J., Drake, S.V.: Linking the language arts and content areas through visual technology. THE J. 22(2), 74–77 (1994)Google Scholar
  26. 26.
    Çöltekin, A., Fabrikant, S.I., Lacayo, M.: Exploring the efficiency of users’ visual analytics strategies based on sequence analysis of eye movement recordings. Int. J. Geograph. Inf. Sci. 24(10), 1559–1575 (2010)CrossRefGoogle Scholar
  27. 27.
    Çöltekin, A., Heil, B., Garlandini, S., Fabrikant, S.I.: Evaluating the effectiveness of interactive map interface designs: a case study integrating usability metrics with eye-movement analysis. Cartography Geogr. Inf. Sci. 36(1), 5–17 (2009)CrossRefGoogle Scholar
  28. 28.
    Cernea, D., Kerren, A., Ebert, A.: Detecting insight and emotion in visualization applications with a commercial EEG headset. In: SIGRAD 2011 Conference on Evaluations of Graphics and Visualization-Efficiency, Usefulness, Accessibility, Usability, pp. 53–60 (2011)Google Scholar
  29. 29.
    Cernea, D., Weber, C., Ebert, A., Kerren, A.: Emotion scents - a method of representing user emotions on GUI widgets. In: Proceedings of the SPIE 2013 Conference on Visualization and Data Analysis (VDA 2013). IS&T/SPIE (2013)Google Scholar
  30. 30.
    Cole, F., Sanik, K., DeCarlo, D., Finkelstein, A., Funkhouser, T., Rusinkiewicz, S., Singh, M.: How well do line drawings depict shape? ACM Trans. Graph. 28(3), 28:1–28:9 (2009)CrossRefGoogle Scholar
  31. 31.
    Csikszentmihalyi, M.: Flow: The Psychology of Optimal Experience. Harper Perennia, New York (1990)Google Scholar
  32. 32.
    Dasgupta, A., Kosara, R.: Privacy-preserving data visualization using parallel coordinates. In: IS&T/SPIE Electronic Imaging, pp. 786800-1–786800-12. International Society for Optics and Photonics (2011)Google Scholar
  33. 33.
    Demiralp, Ç., Bernstein, M.S., Heer, J.: Learning perceptual kernels for visualization design. IEEE Trans. Vis. Comput. Graph. 20(12), 1933–1942 (2014)CrossRefGoogle Scholar
  34. 34.
    Elmqvist, N., Vande Moere, A., Jetter, H.C., Cernea, D., Reiterer, H., Jankun-Kelly, T.J.: Fluid interaction for information visualization. Inf. Vis. 10(4), 327–340 (2011)CrossRefGoogle Scholar
  35. 35.
    Fabrikant, S.I., Christophe, S., Papastefanou, G., Maggi, S.: Emotional response to map design aesthetics. In: 7th International Conference on Geographical Information Science, pp. 18–21 (2012)Google Scholar
  36. 36.
    Farrugia, M., Quigley, A.: Effective temporal graph layout: a comparative study of animation versus static display methods. Inf. Vis. 10(1), 47–64 (2011)Google Scholar
  37. 37.
    Fort, K., Adda, G., Cohen, K.B.: Amazon mechanical turk: gold mine or coal mine? Comput. Linguist. 37(2), 413–420 (2011)CrossRefGoogle Scholar
  38. 38.
    Gadiraju, U., Kawase, R., Dietze, S., Demartini, G.: Understanding malicious behavior in crowdsourcing platforms: the case of online surveys. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1631–1640. ACM (2015)Google Scholar
  39. 39.
    Ghani, S., Elmqvist, N.: Improving revisitation in graphs through static spatial features. In: Graphic Interface (GI 2011), pp. 737–743 (2011)Google Scholar
  40. 40.
    Ghani, S., Elmqvist, N., Yi, J.S.: Perception of animated node-link diagrams for dynamic graphs. Comput. Graph. Forum 31(1), 1205–1214 (2012)CrossRefGoogle Scholar
  41. 41.
    Giannotti, F., Pedreschi, D.: Mobility, Data Mining and Privacy: Geographic Knowledge Discovery, p. 410. Springer, Heidelberg (2008). doi: 10.1007/978-3-540-75177-9 CrossRefGoogle Scholar
  42. 42.
    van Ham, F., Rogowitz, B.: Perceptual organization in user-generated graph layouts. IEEE Trans. Vis. Comput. Graph. 14(6), 1333–1339 (2008)CrossRefGoogle Scholar
  43. 43.
    Haroz, S., Kosara, R., Franconeri, S.L.: Isotype visualization-working memory, performance, and engagement with pictographs. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1191–1200. ACM (2015)Google Scholar
  44. 44.
    Harrison, L., Yang, F., Franconeri, S., Chang, R.: Ranking visualizations of correlation using Weber’s law. IEEE Trans. Vis. Comput. Graph. 20(12), 1943–1952 (2014)CrossRefGoogle Scholar
  45. 45.
    Heer, J., Bostock, M.: Crowdsourcing graphical perception: using mechanical turk to assess visualization design. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 203–212. ACM (2010)Google Scholar
  46. 46.
    Hirth, M., Hoßfeld, T., Tran-Gia, P.: Anatomy of a crowdsourcing platform-using the example of microworkers.com. In: 2011 Fifth International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing (IMIS), pp. 322–329. IEEE (2011)Google Scholar
  47. 47.
    Isenberg, P., Zuk, T., Collins, C., Carpendale, S.: Grounded evaluation of information visualizations. In: Proceedings of the 2008 Workshop on BEyond Time and Errors: Novel Evaluation Methods for Information Visualization (BELIV 2008) pp. 6:1–6:8. ACM (2008)Google Scholar
  48. 48.
    Jianu, R., Rusu, A., Hu, Y., Taggart, D.: How to display group information on node-link diagrams: an evaluation. IEEE Trans. Vis. Comput. Graph. 20(11), 1530–1541 (2014)CrossRefGoogle Scholar
  49. 49.
    Kerren, A., Ebert, A., Meyer, J. (eds.): Human-Centered Visualization Environments. LNCS, vol. 4417. Springer, Heidelberg (2007). doi: 10.1007/978-3-540-71949-6 Google Scholar
  50. 50.
    Kerren, A., Schreiber, F.: Network visualization for integrative bioinformatics. In: Chen, M., Hofestädt, R. (eds.) Approaches in Integrative Bioinformatics, pp. 173–202. Springer, Heidelberg (2014). doi: 10.1007/978-3-642-41281-3_7 CrossRefGoogle Scholar
  51. 51.
    Lam, H., Bertini, E., Isenberg, P., Plaisant, C., Carpendale, S.: Empirical studies in information visualization: seven scenarios. IEEE Trans. Vis. Comput. Graph. 18(9), 1520–1536 (2012)CrossRefGoogle Scholar
  52. 52.
    Laramee, R.S., Kosara, R.: Challenges and Unsolved Problems. In: Kerren et al. [49], pp. 231–254Google Scholar
  53. 53.
    Lebreton, P., Mäki, T., Skodras, E., Hupont, I., Hirth, M.: Bridging the gap between eye tracking and crowdsourcing. In: Proceedings of SPIE, vol. 9394, pp. 93940W–93940W-14 (2015)Google Scholar
  54. 54.
    Lee, B., Plaisant, C., Parr, C.S., Fekete, J.D., Henry, N.: Task taxonomy for graph visualization. In: Proceedings of the 2006 AVI Workshop on Beyond Time and Rrrors: Novel Evaluation Methods for Information Visualization, pp. 1–5. ACM (2006)Google Scholar
  55. 55.
    Li, H., Moacdieh, N.: Is “chart junk” useful? An extended examination of visual embellishment. Proc. Hum. Factors Ergon. Soc. Annual Meeting 58(1), 1516–1520 (2014)CrossRefGoogle Scholar
  56. 56.
    Light, A., Bartlein, P.J.: The end of the rainbow? Color schemes for improved data graphics. EOS 85(40), 385–391 (2004)CrossRefGoogle Scholar
  57. 57.
    Mackay, W.E., Appert, C., Beaudouin-Lafon, M., Chapuis, O., Du, Y., Fekete, J.D., Guiard, Y.: Touchstone: exploratory design of experiments. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1425–1434. ACM (2007)Google Scholar
  58. 58.
    Maggi, S., Fabrikant, S.: Embodied decision making with animations. In: Proceedings of International Conference on Geographic Information Science 2014 (2014)Google Scholar
  59. 59.
    Maggi, S., Fabrikant, S.I.: Triangulating eye movement data of animated displays. In: ET4S@GIScience, pp. 27–31 (2014)Google Scholar
  60. 60.
    Maggi, S., Fabrikant, S.I., Imbert, J.P., Hurter, C.: How do display design and user characteristics matter in animations? An empirical study with air traffic control displays. Cartographica 51(1), 25–37 (2016)CrossRefGoogle Scholar
  61. 61.
    Mahyar, N., Kim, S.H., Kwon, B.C.: Towards a taxonomy for evaluating user engagement in information visualization. In: Workshop on Personal Visualization: Exploring Everyday Life (2015)Google Scholar
  62. 62.
    Marriott, K., Purchase, H., Wybrow, M., Goncu, C.: Memorability of visual features in network diagrams. IEEE Trans. Vis. Comput. Graph. 18(12), 2477–2485 (2012)CrossRefGoogle Scholar
  63. 63.
    Martin, D.: Doing Psychology Experiments, 7th edn. Thomson Wadsworth, Belmont (2008)Google Scholar
  64. 64.
    Martin, D., Hanrahan, B.V., O’Neill, J., Gupta, N.: Being a turker. In: Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing, pp. 224–235. ACM (2014)Google Scholar
  65. 65.
    McCloud, S.: Understanding Comics: The Invisible Art. HarperPerennial, New York (1994)Google Scholar
  66. 66.
    McGee, F., Dingliana, J.: An empirical study on the impact of edge bundling on user comprehension of graphs. In: Proceedings of the International Working Conference on Advanced Visual Interfaces, pp. 620–627. ACM (2012)Google Scholar
  67. 67.
    Micallef, L., Dragicevic, P., Fekete, J.D.: Assessing the effect of visualizations on bayesian reasoning through crowdsourcing. IEEE Trans. Vis. Comput. Graph. 18(12), 2536–2545 (2012)CrossRefGoogle Scholar
  68. 68.
    Monreale, A., Andrienko, G.L., Andrienko, N.V., Giannotti, F., Pedreschi, D., Rinzivillo, S., Wrobel, S.: Movement data anonymity through generalization. Trans. Data Priv. 3(2), 91–121 (2010)MathSciNetGoogle Scholar
  69. 69.
    Munzner, T.: A nested model for visualization design and validation. IEEE Trans. Vis. Comput. Graph. 15(6), 921–928 (2009)CrossRefGoogle Scholar
  70. 70.
    Okoe, M., Jianu, R.: Graphunit: evaluating interactive graph visualizations using crowdsourcing. Comput. Graph. Forum 34(3), 451–460 (2015)CrossRefGoogle Scholar
  71. 71.
    Paas, F., Tuovinen, J.E., Tabbers, H., Van Gerven, P.W.: Cognitive load measurement as a means to advance cognitive load theory. Educ. Psychol. 38(1), 63–71 (2003)CrossRefGoogle Scholar
  72. 72.
    Pandey, A.V., Rall, K., Satterthwaite, M.L., Nov, O., Bertini, E.: How deceptive are deceptive visualizations? An empirical analysis of common distortion techniques. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1469–1478. ACM (2015)Google Scholar
  73. 73.
    Papadopoulos, C., Gutenko, I., Kaufman, A.: VEEVVIE: visual explorer for empirical visualization, VR and interaction experiments. IEEE Trans. Vis. Comput. Graph. 22(1), 111–120 (2016)CrossRefGoogle Scholar
  74. 74.
    Peck, E.M.M., Yuksel, B.F., Ottley, A., Jacob, R.J., Chang, R.: Using fNIRS brain sensing to evaluate information visualization interfaces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 473–482. ACM (2013)Google Scholar
  75. 75.
    Plaisant, C.: The challenge of information visualization evaluation. In: Proceedings of the Working Conference on Advanced Visual Interfaces (AVI 2004), pp. 109–116. ACM (2004)Google Scholar
  76. 76.
    Purchase, H.: Which aesthetic has the greatest effect on human understanding? In: DiBattista, G. (ed.) GD 1997. LNCS, vol. 1353, pp. 248–261. Springer, Heidelberg (1997). doi: 10.1007/3-540-63938-1_67 CrossRefGoogle Scholar
  77. 77.
    Purchase, H.C.: Experimental Human-Computer Interaction: A Practical Guide with Visual Examples. Cambridge University Press, Cambridge (2012)CrossRefGoogle Scholar
  78. 78.
    Ross, J., Irani, L., Silberman, M., Zaldivar, A., Tomlinson, B.: Who are the crowdworkers? Shifting demographics in mechanical turk. In: CHI 2010 Extended Abstracts on Human Factors in Computing Systems, pp. 2863–2872. ACM (2010)Google Scholar
  79. 79.
    Saket, B., Scheidegger, C., Kobourov, S.: Towards understanding enjoyment and flow in information visualization. In: EuroVis. The Eurographics Association (Short Paper) (2015)Google Scholar
  80. 80.
    Saket, B., Scheidegger, C., Kobourov, S.: Comparing node-link and node-link-group visualizations from an enjoyment perspective. Comput. Graph. Forum 35(3), 41–50 (2016)CrossRefGoogle Scholar
  81. 81.
    Saket, B., Scheidegger, C., Kobourov, S.G., Börner, K.: Map-based visualizations increase recall accuracy of data. Comput. Graph. Forum 34(3), 441–450. http://dx.doi.org/10.1111/cgf.12656
  82. 82.
    Sakshaug, J.W., Raghunathan, T.E.: Synthetic data for small area estimation. In: Domingo-Ferrer, J., Magkos, E. (eds.) PSD 2010. LNCS, vol. 6344, pp. 162–173. Springer, Heidelberg (2010). doi: 10.1007/978-3-642-15838-4_15 CrossRefGoogle Scholar
  83. 83.
    Sakshaug, J.W., Raghunathan, T.E.: Generating synthetic data to produce public-use microdata for small geographic areas based on complex sample survey data with application to the national health interview survey. J. Appl. Stat. 41(10), 2103–2122 (2014)CrossRefMATHMathSciNetGoogle Scholar
  84. 84.
    Sakshaug, J.W., Raghunathan, T.E.: Nonparametric generation of synthetic data for small geographic areas. In: Domingo-Ferrer, J. (ed.) PSD 2014. LNCS, vol. 8744, pp. 213–231. Springer, Cham (2014). doi: 10.1007/978-3-319-11257-2_17 Google Scholar
  85. 85.
    Shneiderman, B.: The eyes have it: a task by data type taxonomy for information visualizations. In: Proceedings of the 1996 IEEE Symposium on Visual Languages, Boulder, Colorado, USA, 3–6 September 1996, pp. 336–343. IEEE Computer Society (1996)Google Scholar
  86. 86.
    Tanahashi, Y., Ma, K.L.: Stock lamp: an engagement-versatile visualization design. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 595–604. ACM (2015)Google Scholar
  87. 87.
    Theodoridis, Y., Silva, J.R.O., Nascimento, M.A.: On the generation of spatiotemporal datasets. In: Güting, R.H., Papadias, D., Lochovsky, F. (eds.) SSD 1999. LNCS, vol. 1651, pp. 147–164. Springer, Heidelberg (1999). doi: 10.1007/3-540-48482-5_11 CrossRefGoogle Scholar
  88. 88.
    Valiati, E.R., Pimenta, M.S., Freitas, C.M.: A taxonomy of tasks for guiding the evaluation of multidimensional visualizations. In: Proceedings of the 2006 AVI Workshop on Beyond Time and Errors: Novel Evaluation Methods for Information Visualization, pp. 1–6. ACM (2006)Google Scholar
  89. 89.
    Vande Moere, A., Tomitsch, M., Wimmer, C., Christoph, B., Grechenig, T.: Evaluating the effect of style in information visualization. IEEE Trans. Vis. Comput. Graph. 18(12), 2739–2748 (2012)CrossRefGoogle Scholar
  90. 90.
    Wainer, H.: A test of graphicacy in children. Appl. Psychol. Measure. 4(3), 331–340 (1980)CrossRefGoogle Scholar
  91. 91.
    Walny, J., Huron, S., Carpendale, S.: An exploratory study of data sketching for visual representation. Comput. Graph. Forum 34(3), 231–240 (2015)CrossRefGoogle Scholar
  92. 92.
    Wang, B., Ruchikachorn, P., Mueller, K.: SketchPadN-D: WYDIWYG sculpting and editing in high-dimensional space. IEEE Trans. Vis. Comput. Graph. 19(12), 2060–2069 (2013)CrossRefGoogle Scholar
  93. 93.
    Ware, C.: Information Visualization: Preception for Design, 3rd edn. Elsevier, Amsterdam (2013)Google Scholar
  94. 94.
    Ware, C., Mitchell, P.: Visualizing graphs in three dimensions. ACM Trans. Appl. Percept. 5(1), 2:1–2:15 (2008)CrossRefGoogle Scholar
  95. 95.
    Wilkening, J., Fabrikant, S.I.: How users interact with a 3d geo-browser under time pressure. Cartography Geogr. Inf. Sci. 40(1), 40–52 (2013)CrossRefGoogle Scholar
  96. 96.
    Xu, P., Ehinger, K.A., Zhang, Y., Finkelstein, A., Kulkarni, S.R., Xiao, J.: TurkerGaze: crowdsourcing saliency with webcam based eye tracking. CoRR abs/1504.06755 (2015)Google Scholar
  97. 97.
    Yang, H., Li, Y., Zhou, M.X.: Understand users’ comprehension and preferences for composing information visualizations. ACM Trans. Comput. Hum. Interact. 21(1), 6:1–6:30 (2014)CrossRefGoogle Scholar
  98. 98.
    Ying, X., Wu, X.: Graph generation with prescribed feature constraints. In: SDM, vol. 9, pp. 966–977. SIAM (2009)Google Scholar
  99. 99.
    Ziemkiewicz, C., Kosara, R.: Preconceptions and individual differences in understanding visual metaphors. Comput. Graph. Forum 28(3), 911–918 (2009)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Rita Borgo
    • 1
  • Bongshin Lee
    • 2
  • Benjamin Bach
    • 3
  • Sara Fabrikant
    • 4
  • Radu Jianu
    • 5
  • Andreas Kerren
    • 6
  • Stephen Kobourov
    • 7
  • Fintan McGee
    • 8
  • Luana Micallef
    • 9
  • Tatiana von Landesberger
    • 10
  • Katrin Ballweg
    • 10
  • Stephan Diehl
    • 11
  • Paolo Simonetto
    • 13
  • Michelle Zhou
    • 12
  1. 1.King’s College LondonLondonUK
  2. 2.Microsoft ResearchRedmondUSA
  3. 3.Microsoft Research - InriaParisFrance
  4. 4.University of ZurichZurichSwitzerland
  5. 5.City University LondonLondonUK
  6. 6.Linnaeus UniversityVäxjöSweden
  7. 7.University of ArizonaTucsonUSA
  8. 8.Luxembourg Institute of Science and TechnologyEsch-sur-AlzetteLuxembourg
  9. 9.Helsinki Institute for Information TechnologyAaltoFinland
  10. 10.Darmstadt UniversityDarmstadtGermany
  11. 11.University TrierTrierGermany
  12. 12.JujiSaratogaUSA
  13. 13.Swansea UniversitySwanseaUK

Personalised recommendations