Skip to main content
SpringerLink
Log in

Dowsing: a task-driven approach for multiple-view visualizations dynamic recommendation

  • Regular Paper
  • Published:
Journal of Visualization Aims and scope Submit manuscript

Abstract

Most users are able to obtain exploratory ideas from a data table but cannot clearly declare their analysis tasks as visual queries. Visualization recommendation methods can reduce the demand for data and design knowledge by extracting or referring information from existing high-quality views. However, most solutions cannot identify analysis tasks, which limits the accuracy of their recommendations. To address this limitation, we propose a deep learning and answer set programming-based approach to guide visualization recommendations by tracking potential analysis tasks and field preferences in exploration interactions. We demonstrate this approach via Dowsing, a mixed-initiative system for visual data exploration that automatically identifies and presents users’ potential analysis tasks and recommends visualizations during exploration. Additionally, Dowsing allows users to confirm and edit their intentions in multiple ways to adapt to changing analysis requirements. The effectiveness and usability of our approach are validated through quantitative experiments and two user studies.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. https://hmmlearn.readthedocs.io/en/latest/.

  2. https://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeClassifier.html.

  3. https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsClassifier.html.

  4. https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html.

  5. https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPClassifier.html.

References

  • Albers D, Correll M, Gleicher M (2014) Task-driven evaluation of aggregation in time series visualization. In: Proceedings of the SIGCHI conference on human factors in computing systems, pp. 551–560. ACM, Toronto Ontario Canada . https://doi.org/10.1145/2556288.2557200

  • Amar R, Eagan J, Stasko J (2005) Low-level components of analytic activity in information visualization. In: IEEE symposium on information visualization, 2005. INFOVIS 2005., pp. 111–117

  • Battle L, Heer J (2019) Characterizing Exploratory Visual Analysis: A Literature Review and Evaluation of Analytic Provenance in Tableau. Computer Graphics Forum 38(3):145–159. https://doi.org/10.1111/cgf.13678

    Article  Google Scholar 

  • Brehmer M, Munzner T (2013) A multi-level typology of abstract visualization tasks. IEEE Trans Visual Comput Graphics 19(12):2376–2385

    Article  Google Scholar 

  • Breiman L (2001) Random forests. Mach Learn 45:5–32

    Article  Google Scholar 

  • Brown ET, Ottley A, Zhao H, Lin Q, Souvenir R, Endert A, Chang R (2014) Finding Waldo: learning about users from their interactions. IEEE Trans Visual Comput Graphics 20(12):1663–1672. https://doi.org/10.1109/TVCG.2014.2346575

    Article  Google Scholar 

  • Cao Y-R, Xiao-Han Li, Pan J-Y, Lin W-C (2022) VisGuide: user-Oriented Recommendations for Data Event Extraction. In: CHI conference on human factors in computing systems, pp. 1–13. ACM, New Orleans LA USA. https://doi.org/10.1145/3491102.3517648

  • Chen X, Zeng W, Lin Y, AI-maneea HM, Roberts J, Chang R (2021) Composition and configuration patterns in multiple-view visualizations. IEEE Trans Visual Comput Graphics 27(2):1514–1524

    Article  Google Scholar 

  • Collins C, Andrienko N, Schreck T, Yang J, Choo J, Engelke U, Jena A, Dwyer T (2018) Guidance in the human-machine analytics process. Visual Inform 2(3):166–180

    Article  Google Scholar 

  • Dabek F, Caban JJ (2017) A grammar-based approach for modeling user interactions and generating suggestions during the data exploration process. IEEE Trans Visual Comput Graphics 23(1):41–50. https://doi.org/10.1109/TVCG.2016.2598471

    Article  Google Scholar 

  • de Oliveira MCF, Levkowitz H (2003) From visual data exploration to visual data mining: A survey. IEEE Trans Visual Comput Graphics 9(3):378–394. https://doi.org/10.1109/TVCG.2003.1207445

    Article  Google Scholar 

  • Gotz D, Wen Z (2009) Behavior-driven visualization recommendation. In: Proceedings of the 14th international conference on intelligent user interfaces, pp. 315–324. ACM, Sanibel Island Florida USA

  • Gotz D, Zhou MX (2008) An empirical study of user interaction behavior during visual analysis. IBM Research RC24525 (W0803-127)

  • Guo H, Gomez SR, Ziemkiewicz C, Laidlaw DH (2016) A case study using visualization interaction logs and insight metrics to understand how analysts arrive at insights. IEEE Trans Visual Comput Graphics 22(1):51–60. https://doi.org/10.1109/TVCG.2015.2467613

    Article  Google Scholar 

  • Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735

    Article  Google Scholar 

  • Kim Y, Heer J (2018) Assessing effects of task and data distribution on the effectiveness of visual encodings. Computer Graphics Forum 37(3):157–167. https://doi.org/10.1111/cgf.13409

    Article  Google Scholar 

  • Li H, Wang Y, Zhang S, Song Y, Qu H (2022) KG4Vis: a knowledge graph-based approach for visualization recommendation. IEEE Trans Visual Comput Graphics 28(1):195–205. https://doi.org/10.1109/TVCG.2021.3114863

    Article  Google Scholar 

  • Li Y, Qi Y, Shi Y, Chen Q, Cao N, Chen S (2022) Diverse interaction recommendation for public users exploring multi-view visualization using deep learning. IEEE Trans Visual Comput Graphics. https://doi.org/10.1109/TVCG.2022.3209461

    Article  Google Scholar 

  • Lin H, Moritz D, Heer J (2020) Dziban: Balancing Agency & Automation in Visualization Design via Anchored Recommendations. In: Proceedings of the 2020 CHI Conference on human factors in computing systems, pp. 1–12. ACM, Honolulu HI USA

  • Luo X, Yuan Y, Zhang K, Xia J, Zhou Z, Chang L, Gu T (2019) Enhancing statistical charts: toward better data visualization and analysis. J Visualization 22:819–832

    Article  Google Scholar 

  • Luo Y, Qin X, Tang N, Li G (2018) DeepEye: towards automatic data visualization. In: 2018 IEEE 34th international conference on data engineering (ICDE). IEEE, Paris, pp. 101–112

  • Moritz D, Wang C, Nelson GL, Lin H, Smith AM, Howe B, Heer J (2019) Formalizing visualization design knowledge as constraints: actionable and extensible models in Draco. IEEE Trans Visual Comput Graphics 25(1):438–448

    Article  Google Scholar 

  • Narechania A, Srinivasan A, Stasko J (2021) NL4DV: a toolkit for generating analytic specifications for data visualization from natural language queries. IEEE Trans Visual Comput Graphics 27(2):369–379

    Article  Google Scholar 

  • Nguyen QV, Miller N, Arness D, Huang W, Huang ML, Simoff S (2020) Evaluation on interactive visualization data with scatterplots. Visual Inform 4(4):1–10

    Article  Google Scholar 

  • Ojo F, Rossi RA, Hoffswell J, Guo S, Du F, Kim S, Xiao C, Koh E (2022) VisGNN: Personalized Visualization Recommendationvia Graph Neural Networks. In: Proceedings of the ACM web conference 2022, pp. 2810–2818. ACM, Virtual Event, Lyon France. https://doi.org/10.1145/3485447.3512001

  • Peterson LE (2009) K-nearest neighbor. Scholarpedia 4(2):1883

    Article  Google Scholar 

  • Popescu M-C, Balas VE, Perescu-Popescu L, Mastorakis N (2009) Multilayer perceptron and neural networks. WSEAS Trans Circ Syst 8(7):579–588

    Google Scholar 

  • Qian X, Rossi RA, Du F, Kim S, Koh E, Malik S, Lee TY, Ahmed NK (2022) Personalized visualization recommendation. ACM Trans Web (TWEB) 16(3):1–47

    Article  Google Scholar 

  • Qin X, Luo Y, Tang N, Li G (2020) Making data visualization more efficient and effective: a survey. VLDB J 29(1):93–117. https://doi.org/10.1007/s00778-019-00588-3

    Article  Google Scholar 

  • Quadri GJ, Rosen P (2022) A survey of perception-based visualization studies by task. IEEE Trans Visual Comput Graphics 28(12):5026–5048. https://doi.org/10.1109/TVCG.2021.3098240

    Article  Google Scholar 

  • Rabiner L, Juang B (1986) An introduction to hidden Markov models. IEEE ASSP Mag 3(1):4–16. https://doi.org/10.1109/MASSP.1986.1165342

    Article  Google Scholar 

  • Rind A, Aigner W, Wagner M, Miksch S, Lammarsch T (2016) Task Cube: a three-dimensional conceptual space of user tasks in visualization design and evaluation. Inf Vis 15(4):288–300

    Article  Google Scholar 

  • Roberts JC (2007) State of the Art: coordinated & multiple views in exploratory visualization. In: Fifth international conference on coordinated and multiple views in exploratory visualization (CMV 2007), pp. 61–71. https://doi.org/10.1109/CMV.2007.20

  • Roth SF, Mattis J (1990) Data characterization for intelligent graphics presentation. In: Proceedings of the SIGCHI conference on human factors in computing systems empowering people - CHI ’90, pp. 193–200. ACM Press, Seattle, Washington, USA. https://doi.org/10.1145/97243.97273

  • Rubab S, Tang J, Wu Y (2021) Examining interaction techniques in data visualization authoring tools from the perspective of goals and human cognition: a survey. J Visualiz 24:397–418

    Article  Google Scholar 

  • Saket B, Endert A, Demiralp Ç (2019) Task-Based Effectiveness of Basic Visualizations. IEEE Trans Visual Comput Graphics 25(7):2505–2512. https://doi.org/10.1109/TVCG.2018.2829750

    Article  Google Scholar 

  • Sarikaya A, Gleicher M (2018) Scatterplots: tasks, data, and designs. IEEE Trans Visual Comput Graphics 24(1):402–412

    Article  Google Scholar 

  • Schuster M, Paliwal KK (1997) Bidirectional recurrent neural networks. IEEE Trans Signal Process 45(11):2673–2681. https://doi.org/10.1109/78.650093

    Article  Google Scholar 

  • Shen L, Shen E, Tai Z, Song Y, Wang J (2021) TaskVis: task-oriented Visualization Recommendation. EuroVis 2021 - Short Papers, 5

  • Shirato G, Andrienko N, Andrienko G (2023) Exploring and visualizing temporal relations in multivariate time series. Visual Inform 7(4):57–72. https://doi.org/10.1016/j.visinf.2023.09.001

    Article  Google Scholar 

  • Shi D, Shi Y, Xu X, Chen N, Fu S, Wu H, Cao N (2019) Task-oriented optimal sequencing of visualization charts. In: 2019 IEEE visualization in data science (VDS), pp. 58–66

  • Song Y-Y, Ying L (2015) Decision tree methods: applications for classification and prediction. Shanghai Arch Psychiatry 27(2):130

    Google Scholar 

  • Soni P, de Runz C, Bouali F, Venturini G (2024) A survey on automatic dashboard recommendation systems. Visual Informatics. https://doi.org/10.1016/j.visinf.2024.01.002

    Article  Google Scholar 

  • Stoiber C, Ceneda D, Wagner M, Schetinger V, Gschwandtner T, Streit M, Miksch S, Aigner W (2022) Perspectives of visualization onboarding and guidance in VA. Visual Inform 6(1):68–83. https://doi.org/10.1016/j.visinf.2022.02.005

    Article  Google Scholar 

  • Vladimir L (2008) What is answer set programming. In: Proc. 23rd AAAI conference on artificial intelligence, vol. 8, pp. 1594–1597

  • Wongsuphasawat K, Moritz D, Anand A, Mackinlay J, Howe B, Heer J (2016) Towards a general-purpose query language for visualization recommendation. In: Proceedings of the workshop on human-in-the-loop data analytics, pp. 1–6. ACM, San Francisco California. https://doi.org/10.1145/2939502.2939506

  • Wongsuphasawat K, Qu Z, Moritz D, Chang R, Ouk F, Anand A, Mackinlay J, Howe B, Heer J (2017) Voyager 2: augmenting visual analysis with partial view specifications. In: Proceedings of the 2017 CHI conference on human factors in computing systems, pp. 2648–2659. ACM, Denver Colorado USA

  • Wu A, Wang Y, Zhou M, He X, Zhang H, Qu H, Zhang D (2021) Multivision: designing analytical dashboards with deep learning based recommendation. IEEE Trans Visual Comput Graphics 28(1):162–172. https://doi.org/10.1109/tvcg.2021.3114826

    Article  Google Scholar 

  • Wu A, Wang Y, Shu X, Moritz D, Cui W, Zhang H, Zhang D, Qu H (2022) AI4VIS: survey on artificial intelligence approaches for data visualization. IEEE Trans Visual Comput Graphics 28(12):5049–5070. https://doi.org/10.1109/TVCG.2021.3099002

    Article  Google Scholar 

  • Xu K, Attfield S, Jankun-Kelly TJ, Wheat A, Nguyen PH, Selvaraj N (2015) Analytic provenance for sensemaking: a research agenda. IEEE Comput Graphics Appl 35(3):56–64. https://doi.org/10.1109/MCG.2015.50

    Article  Google Scholar 

  • Yu D, Ian O, Jie L, Xiaoru Y, Vinh NQ (2023) User-centered visual explorer of in-process comparison in spatiotemporal space. J Visualization 26(2):403–421. https://doi.org/10.1007/s12650-022-00882-3

    Article  Google Scholar 

  • Zhang X, Ge X, Chrysanthis PK, Sharaf MA (2021) ViewSeeker: an interactive view recommendation framework. Big Data Research 25:100238. https://doi.org/10.1016/j.bdr.2021.100238

    Article  Google Scholar 

  • Zhou Y, Meng X, Wu Y, Tang T, Wang Y, Wu Y (2023) An intelligent approach to automatically discovering visual insights. J Visualiz 26(3):705–722. https://doi.org/10.1007/s12650-022-00894-z

    Article  Google Scholar 

  • Zhu S, Sun G, Jiang Q, Zha M, Liang R (2020) A survey on automatic infographics and visualization recommendations. Visual Inform 4(3):24–40. https://doi.org/10.1016/j.visinf.2020.07.002

    Article  Google Scholar 

  • Zhu Z, Shen Y, Zhu S, Zhang G, Liang R, Sun G (2023) Towards better pattern enhancement in temporal evolving set visualization. J Visualiz 26(3):611–629. https://doi.org/10.1007/s12650-022-00896-x

    Article  Google Scholar 

Download references

Funding

This research is partially supported by National Natural Science Foundation of China (62172289) and the School-City Strategic Cooperation Project (2021CDSN-13).

Author information

Authors and Affiliations

  1. College of Computer Science, Sichuan University, Yihuan Road, Chengdu, 610065, Sichuan, China

    Jiamin Zhu, Meixuan Wu, Yi Zhou, Haotian Zhu & Min Zhu

  2. College of Design and Innovation, Tongji University, Fuxing Road, Shanghai, 200092, China

    Nan Cao

Authors
  1. Jiamin Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Meixuan Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yi Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nan Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Haotian Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Min Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Min Zhu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file 1 (mov 17696 KB)

Supplementary file 2 (mp4 8735 KB)

Supplementary file 3 (pdf 1177 KB)

Supplementary file 4 (docx 301 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, J., Wu, M., Zhou, Y. et al. Dowsing: a task-driven approach for multiple-view visualizations dynamic recommendation. J Vis (2024). https://doi.org/10.1007/s12650-024-00989-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s12650-024-00989-9

Keywords

Search

Navigation