Abstract
Visual interactive system (VIS) has been received significant attention for solving various complex problems. However, designing and implementing a novel VIS with the large scale of data is a challenging task. While existing studies have applied various visual analytics (VA) to analyze and visualize insightful information, deep visual analytics (DVA) have considered as a promising technique to provide input evidences and explain system results. In this study, we present several deep learning (DL) techniques for analyzing data with visualization, which summarizes the state-of-the-art review on (i) big data analysis, (ii) cognitive and perception science, (iii) customer behavior analysis, (iv) natural language processing, (v) recommended system, (vi) healthcare analysis, (vii) fintech ecosystem, and (viii) tourism management. We present open research challenges for emerging DVA in the visualization community. We also highlight some key themes from the existing literature that may help to explore for future study. Thus, our goal is to help readers and researchers in DL and VA to understand key aspects in designing VIS for analysing data.
Article PDF
Avoid common mistakes on your manuscript.
References
R.A. Leite, T. Gschwandtner, S. Miksch, E. Gstrein, J. Kuntner, NEVA: visual analytics to identify fraudulent networks, Computer Graphics Forum 39 (2020), 344–359.
M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, et al., Tensorflow: large-scale machine learning on heterogeneous distributed systems, arXiv preprint arXiv: 1603.04467, 2016.
S.S. Abdullah, N. Rostamzadeh, K. Sedig, A.X. Garg, E. McArthur, Multiple regression analysis and frequent itemset mining of electronic medical records: a visual analytics approach using VISA_M3R3, Data 5 (2020), 33.
H. Arnout, J. Kehrer, J. Bronner, T. Runkler, Visual evaluation of generative adversarial networks for time series data, arXiv preprint arXiv: 2001.00062, 2019.
M. Aupetit, M. Sedlmair, M.M. Abbas, A. Baggag, H. Bensmail, Toward perception-based evaluation of clustering techniques for visual analytics, 2019 IEEE Visualization Conference (VIS), IEEE, Vancouver, BC, Canada, 2019, pp. 141–145.
R.C. Basole, S.S. Patel, Transformation through unbundling: visualizing the global finTech ecosystem, Service Science 10 (2018), 379–396.
A.L. Beam, I.S. Kohane, Big data and machine learning in health care, JAMA 319 (2018), 1317–1318.
R. Bhardwaj, A.R. Nambiar, D. Dutta, A study of machine learning in healthcare, 2017 IEEE 41st Annual Computer Software and Applications Conference (COMPSAC), IEEE, Turin, Italy, 2017, pp. 236–241.
A. Bilal, A. Jourabloo, M. Ye, X. Liu, L. Ren, Do convolutional neural networks learn class hierarchy? IEEE Transactions on Visualization and Computer Graphics 24 (2017), 152–162.
R. Bro, A.K. Smilde, Principal component analysis, Analytical methods 6 (2014), 2812–2831.
J.J. Caban, D. Gotz, Visual analytics in healthcare–opportunities and research challenges, J. Am. Med. Inform. Assoc. 22 (2015), 260–262.
D. Cashman, G. Patterson, A. Mosca, N. Watts, S. Robinson, R. Chang, RNNbow: visualizing learning via backpropagation gradients in RNNs, IEEE Computer Graphics and Applications 38 (2018), 39–50.
J. Chae, S. Gao, A. Ramanathan, C.A. Steed, G. Tourassi, Visualization for classification in deep neural networks, Technical Report, Oak Ridge National Lab (ORNL), Oak Ridge, TN, USA, 2017.
Y.C. Chang, C.H. Ku, C.H. Chen, Using deep learning and visual analytics to explore hotel reviews and responses, Tourism Management 80 (2020), 104129.
H. Chen, H. Li, Y. Fang, Y. Chen, Anisotropic parallel coordinates with adjustment based on distribution features, Journal of Visualization 19 (2016), 327–335.
Y. Chen, J.E. Argentinis, G. Weber, IBM watson: how cognitive computing can be applied to big data challenges in life sciences research, Clinical Therapeutics 38 (2016), 688–701.
M. Cheng, D. Edwards, Social media in tourism: a visual analytic approach, Current Issues in Tourism 18 (2015), 1080–1087.
K. Cho, B. Van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, et al., Learning phrase representations using RNN encoder-decoder for statistical machine translation, arXiv preprint arXiv: 1406.1078, 2014.
J. Choo, S. Liu, Visual analytics for explainable deep learning, IEEE Computer Graphics and Applications 38 (2018), 84–92.
S. Chung, S. Suh, C. Park, K. Kang, J. Choo, B.C. Kwon, ReVACNN: Real-time visual analytics for convolutional neural network, In: ACM SIGKDD Workshop on Interactive Data Exploration and Analytics (IDEA), San Francisco, USA, 2016, p. 7.
Y. Chung, N. Bagheri, J.A. Salinas-Perez, K. Smurthwaite, E. Walsh, M.A. Furst, et al., Role of visual analytics in supporting mental healthcare systems research and policy: a systematic scoping review, International Journal of Information Management 50 (2020), 17–27.
A. Darwish, A.E. Hassanien, M. Elhoseny, A.K. Sangaiah, K. Muhammad, The impact of the hybrid platform of internet of things and cloud computing on healthcare systems: opportunities, challenges, and open problems, Journal of Ambient Intelligence and Humanized Computing 10 (2019), 4151–4166.
A. Da’u, N. Salim, Recommendation system based on deep learning methods: a systematic review and new directions, Artificial Intelligence Review 53 (2019), 2709–2748.
J. Davidson, B. Liebald, J. Liu, P. Nandy, T. Van Vleet, U. Gargi, et al., The YouTube video recommendation system, Proceedings of the fourth ACM Conference on Recommender Systems, 2010, pp. 293–296.
J. Deng, W. Dong, R. Socher, L.J. Li, K. Li, L. Fei-Fei, Imagenet: a large-scale hierarchical image database, 2009 IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Miami, FL, USA, 2009, pp. 248–255.
Z. Deng, D. Weng, J. Chen, R. Liu, Z. Wang, J. Bao, et al., AirVis: visual analytics of air pollution propagation, IEEE Transactions on Visualization and Computer Graphics 26 (2019), 800–810.
A. Dhillon, A. Singh, Machine learning in healthcare data analysis: a survey, Journal of Biology and Today’s World 8 (2019), 1–10.
V. Dibia, Ç. Demiralp, Data2Vis: automatic generation of data visualizations using sequence-to-sequence recurrent neural networks, IEEE Computer Graphics and Applications 39 (2019), 33–46.
Y. Ding, Y. Liu, H. Luan, M. Sun, Visualizing and understanding neural machine translation, Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Association for Computational Linguistics, Vancouver, Canada, 2017, pp. 1150–1159.
S. Dong, P. Wang, K. Abbas, A survey on deep learning and its applications, Computer Science Review 40 (2021), 100379.
N. Elmqvist, P. Dragicevic, J.D. Fekete, Rolling the dice: multidimensional visual exploration using scatterplot matrix navigation, IEEE transactions on Visualization and Computer Graphics 14 (2008), 1539–1148.
J. Evermann, J.R. Rehse, P. Fettke, Predicting process behaviour using deep learning, Decision Support Systems 100 (2017), 129–140.
T. Gao, M. Dontcheva, E. Adar, Z. Liu, K.G. Karahalios, Datatone: managing ambiguity in natural language interfaces for data visualization, Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology, ACM, New York, NY, United States, 2015, pp. 489–500.
R. Garcia, A.C. Telea, B.C. da Silva, J. Tørresen, J.L.D. Comba, A task-and-technique centered survey on visual analytics for deep learning model engineering, Computers & Graphics 77 (2018), 30–49.
H.G. Caballero, M. Westenberg, B. Gebre, J. van Wijk, V-awake: a visual analytics approach for correcting sleep predictions from deep learning models, Computer Graphics Forum, 38 (2019), 1–12.
C.A. Gomez-Uribe, N. Hunt, The netflix recommender system: algorithms, business value, and innovation, ACM Transactions on Management Information Systems 6 (2015), 1–19.
D. Gotz, D. Borland, Data-driven healthcare: challenges and opportunities for interactive visualization, IEEE Computer Graphics and Applications 36 (2016), 90–96.
L. Gou, L. Zou, N. Li, M. Hofmann, A.K. Shekar, A. Wendt, et al., VATLD: a Visual Analytics system to assess, understand and improve Traffic Light Detection, IEEE Transactions on Visualization and Computer Graphics 27 (2020), 261–271.
T.M. Green, W. Ribarsky, B. Fisher, Visual analytics for complex concepts using a human cognition model, 2008 IEEE Symposium on Visual Analytics Science and Technology, IEEE, Columbus, OH, USA, 2008, pp. 91–98.
A. Hajnayeb, A. Ghasemloonia, S.E. Khadem, M.H. Moradi, Application and comparison of an ANN-based feature selection method and the genetic algorithm in gearbox fault diagnosis, Expert Systems with Applications 38 (2011), 10205–10209.
A. Hinterreiter, P. Ruch, H. Stitz, M. Ennemoser, J. Bernard, H. Strobelt, et al., ConfusionFlow: a model-agnostic visualization for temporal analysis of classifier confusion, IEEE Transactions on Visualization and Computer Graphics (2020).
F. Hohman, M. Kahng, R. Pienta, D.H. Chau, Visual analytics in deep learning: an interrogative survey for the next frontiers, IEEE Transactions on Visualization and Computer Graphics 25 (2018), 2674–2693.
F. Hohman, H. Park, C. Robinson, D.H.P Chau, SUMMIT: scaling deep learning interpretability by visualizing activation and attribution summarizations, IEEE Transactions on Visualization and Computer Graphics 26 (2019), 1096–1106.
F. Hohman, K. Wongsuphasawat, M.B. Kery, K. Patel, Understanding and visualizing data iteration in machine learning, Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, ACM, New York, NY, United States, 2020, pp. 1–13.
J. Hollan, E. Hutchins, D. Kirsh, Distributed cognition: toward a new foundation for human-computer interaction research, ACM Transactions on Computer-Human Interaction 7 (2000), 174–196.
K. Hu, M.A. Bakker, S. Li, T. Kraska, C. Hidalgo, VizML: a machine learning approach to visualization recommendation, Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, Glasgow, Scotland, UK, 2019, pp. 1–12.
S. Ingram, T. Munzner, M. Olano, Glimmer: multilevel MDS on the GPU, IEEE Transactions on Visualization and Computer Graphics 15 (2008), 249–261.
A. Iovine, F. Narducci, G. Semeraro, Conversational recommender systems and natural language: a study through the converse framework, Decision Support Systems 131 (2020), 113250.
M.R. Islam, S. Liu, R. Biddle, I. Razzak, X. Wang, P. Tilocca, et al., Discovering dynamic adverse behavior of policyholders in the life insurance industry, Technological Forecasting and Social Change 163 (2021), 120486.
M.R. Islam, S. Liu, I. Razzak, M.A. Kabir, X. Wang, G. Xu, MHIVis: visual analytics for exploring mental illness of policyholders in life insurance industry, 2020 7th International Conference on Behavioral, Economic, and Socio-Cultural Computing (BESC), IEEE, Bournemouth, United Kingdom, 2020, pp. 1–4.
M.R. Islam, S. Liu, X. Wang, G. Xu, Deep learning for misinformation detection on online social networks: a survey and new perspectives, Social Network Analysis and Mining 10 (2020), 82.
M.R. Islam, I. Razzak, X. Wang, P. Tilocca, G. Xu, UCBVis: understanding customer behavior sequences with visual interactive system, 2021 International Joint Conference on Neural Networks (IJCNN), IEEE, China, 2021.
M.T. Islam, M.R. Islam, S. Akter, M. Kawser, Designing dashboard for exploring tourist hotspots in Bangladesh, 2020 23rd International Conference on Computer and Information Technology (ICCIT), IEEE, Dhaka, Bangladesh, 2020, pp. 1–6.
Z. Jin, Y. Wang, Q. Wang, Y. Ming, T. Ma, H. Qu, GNNVis: a visual analytics approach for prediction error diagnosis of graph neural networks, arXiv preprint arXiv: 2011.11048, 2020.
M. Kahng, P.Y. Andrews, A. Kalro, D.H. Chau, ActiVis: visual exploration of industry-scale deep neural network models, IEEE Transactions on Visualization and Computer Graphics 24 (2017), 88–97.
M. Kahng, N. Thorat, D.H. Chau, F.B. Viégas, M. Wattenberg, GAN Lab: understanding complex deep generative models using interactive visual experimentation, IEEE Transactions on Visualization and Computer Graphics 25 (2018), 310–320.
D. Kalaivani, P. Sumathi, Factor based prediction model for customer behavior analysis, International Journal of System Assurance Engineering and Management 10 (2019), 519–524.
A. Karpathy, J. Johnson, L. Fei-Fei, Visualizing and understanding recurrent networks, arXiv preprint arXiv: 1506.02078, 2015.
D. Keim, G. Andrienko, J.D. Fekete, C. Görg, J. Kohlhammer, G. Melançon, Visual analytics: definition, process, and challenges, in: A. Kerren, J.T. Stasko, J.D. Fekete, C. North (Eds.), Information visualization, Springer, Berlin, Heidelberg, 2008, pp. 154–175.
A.A. Khade, Performing customer behavior analysis using big data analytics, Procedia Computer Science 79 (2016), 986–992.
T.N. Kipf, M. Welling, Semi-supervised classification with graph convolutional networks, arXiv preprint arXiv: 1609.02907, 2016.
S. Ko, I. Cho, S. Afzal, C. Yau, J. Chae, A. Malik, et al., A survey on visual analysis approaches for financial data, Computer Graphics Forum, 35 (2016), 599–617.
I.V. Krak, O.V. Barmak, E. Manziuk, Visual analytics to build a machine learning model, in: Research Advancements in Smart Technology, Optimization, and Renewable Energy, IGI Global, Hershey, Pennsylvania, 2021, pp. 313–329.
C.H. Ku, Y.C. Chang, Y. Wang, C.H. Chen, S.H. Hsiao, Artificial intelligence and visual analytics: a deep-learning approach to analyze hotel reviews & responses, Proceedings of the 52nd Hawaii International Conference on System Sciences, Maui, Hawaii, USA, 2019.
T. Kulesza, M. Burnett, W.K. Wong, S. Stumpf, Principles of explanatory debugging to personalize interactive machine learning, Proceedings of the 20th International Conference on Intelligent User Interfaces, ACM, New York, NY, United States, 2015, pp. 126–137.
B.C. Kwon, M.J. Choi, J.T. Kim, E. Choi, Y.B. Kim, S. Kwon, J. Sun, J. Choo, RetainVis: visual analytics with interpretable and interactive recurrent neural networks on electronic medical records, IEEE Transactions on Visualization and Computer Graphics 25 (2018), 299–309.
O.H. Kwon, K.L. Ma, A deep generative model for graph layout, IEEE Transactions on Visualization and Computer Graphics 26 (2019), 665–675.
C. Lee, Y. Kim, S. Jin, D. Kim, R. Maciejewski, D. Ebert, et al., A visual analytics system for exploring, monitoring, and forecasting road traffic congestion, IEEE Transactions on Visualization and Computer Graphics 26 (2020), 3133–3146.
I. Lee, Y.J. Shin, Fintech: ecosystem, business models, investment decisions, and challenges, Business Horizons 61 (2018), 35–46.
R.A. Leite, T. Gschwandtner, S. Miksch, S. Kriglstein, M. Pohl, E. Gstrein, et al., EVA: visual analytics to identify fraudulent events, IEEE Transactions on Visualization and Computer Graphics 24 (2017), 330–339.
H. Li, Y. Tian, K. Mueller, X. Chen, Beyond saliency: understanding convolutional neural networks from saliency prediction on layer– wise relevance propagation, Image and Vision Computing 83 (2019), 70–86.
R. Li, C. Yin, S. Yang, B. Qian, P. Zhang, Marrying medical domain knowledge with deep learning on electronic health records: a deep visual analytics approach, Journal of Medical Internet Research 22 (2020), e20645.
Y. Li, C. Hu, C. Huang, L. Duan, The concept of smart tourism in the context of tourism information services, Tourism Management 58 (2017), 293–300.
Z.C. Lipton, The mythos of model interpretability: in machine learning, the concept of interpretability is both important and slippery, Queue 16 (2018), 31–57.
F. Liu, N.L. Fella, K. Liao, Modeling language vagueness in privacy policies using deep neural networks, arXiv preprint arXiv: 1805.10393, 2018.
H. Liu, T. Taniguchi, Y. Tanaka, K. Takenaka, T. Bando, Visualization of driving behavior based on hidden feature extraction by using deep learning, IEEE Transactions on Intelligent Transportation Systems 18 (2017), 2477–2489.
M. Liu, J. Shi, K. Cao, J. Zhu, S. Liu, Analyzing the training processes of deep generative models, IEEE Transactions on Visualization and Computer Graphics 24 (2017), 77–87.
M. Liu, J. Shi, Z. Li, C. Li, J. Zhu, S. Liu, Towards better analysis of deep convolutional neural networks, IEEE Transactions on Visualization and Computer Graphics 23 (2016), 91–100.
S. Liu, P.T. Bremer, J.J. Thiagarajan, V. Srikumar, B. Wang, Y. Livnat, et al., Visual exploration of semantic relationships in neural word embeddings, IEEE Transactions on Visualization and Computer Graphics 24 (2018), 553–562.
S. Liu, D. Maljovec, B. Wang, P.T. Bremer, V. Pascucci, Visualizing high-dimensional data: advances in the past decade, IEEE Transactions on Visualization and Computer Graphics 23 (2016), 1249–1268.
S. Liu, X. Wang, M. Liu, J. Zhu, Towards better analysis of machine learning models: a visual analytics perspective, Visual Informatics 1 (2017), 48–56.
Y. Lu, R. Garcia, B. Hansen, M. Gleicher, R. Maciejewski, The state-of-the-art in predictive visual analytics, Computer Graphics Forum, 36 (2017), 539–562.
Y. Luo, X. Qin, N. Tang, G. Li, DeepEye: towards automatic data visualization, 2018 IEEE 34th international conference on data engineering (ICDE), IEEE, Paris, France, 2018, pp. 101–112.
A. Mahendran, A. Vedaldi, Visualizing deep convolutional neural networks using natural pre-images, International Journal of Computer Vision 120 (2016), 233–255.
S. Mamonov, The role of information technology in fintech innovation: insights from the european fintech ecosystem, Proceedings of the 54th Hawaii International Conference on System Sciences, Kauai, Hawaii, USA, 2021, p. 5036.
A. Mandal, A. Sinaeepourfard, S.K. Naskar, VDA: deep learning based visual data analysis in integrated edge to cloud computing environment, Adjunct Proceedings of the 2021 International Conference on Distributed Computing and Networking, ACM, New York, NY, United States, 2021, pp. 7–12.
S.J. Miah, H.Q. Vu, J. Gammack, M. McGrath, A big data analytics method for tourist behaviour analysis, Information & Management 54 (2017), 771–785.
Y. Ming, S. Cao, R. Zhang, Z. Li, Y. Chen, Y. Song, et al., Understanding hidden memories of recurrent neural networks, 2017 IEEE Conference on Visual Analytics Science and Technology (VAST), IEEE, Phoenix, AZ, USA, 2017, pp. 13–24.
G. Montavon, W. Samek, K.R. Müller, Methods for interpreting and understanding deep neural networks, Digital Signal Processing 73 (2018), 1–15.
A. Moran, V. Gadepally, M. Hubbell, J. Kepner, Improving big data visual analytics with interactive virtual reality, 2015 IEEE high performance extreme computing conference (HPEC), IEEE, Waltham, MA, USA, 2015, pp. 1–6.
A.A. Mubarak, H. Cao, W. Zhang, W. Zhang, Visual analytics of video-clickstream data and prediction of learners’ performance using deep learning models in MOOCs’ courses, Computer Applications in Engineering Education 29 (2020), 710–732.
P. Muthukannan, B. Tan, D. Gozman, L. Johnson, The emergence of a fintech ecosystem: a case study of the vizag fintech valley in India, Information & Management 57 (2020), 103385.
Y.W. Ng, H. Qu, TrendFocus: visualization of trends in financial news with indicator sets, 2014 International Conference on Big Data and Smart Computing (BIGCOMP), IEEE, Bangkok, Thailand, 2014, pp. 7–12.
A. Nguyen, J. Yosinski, J. Clune, Multifaceted feature visualization: uncovering the different types of features learned by each neuron in deep neural networks, arXiv preprint arXiv: 1602.03616, 2016.
J. Obeid, E. Hoque, Chart-to-text: generating natural language descriptions for charts by adapting the transformer model, arXiv preprint arXiv: 2010.09142, 2020.
J. Oh, X. Guo, H. Lee, R. Lewis, S. Singh, Action-conditional video prediction using deep networks in atari games, arXiv preprint arXiv: 1507.08750, 2015.
M. Oquab, L. Bottou, I. Laptev, J. Sivic, Learning and transferring mid-level image representations using convolutional neural networks, 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Columbus, OH, USA, 2014, pp. 1717–1724.
C. Park, I. Na, Y. Jo, S. Shin, J. Yoo, B.C. Kwon, et al., SANVis: visual analytics for understanding self-attention networks, 2019 IEEE Visualization Conference (VIS), IEEE, Vancouver, BC, Canada, 2019, pp. 146–150.
F.V. Paulovich, L.G. Nonato, R. Minghim, H. Levkowitz, Least square projection: a fast high-precision multidimensional projection technique and its application to document mapping, IEEE Transactions on Visualization and Computer Graphics 14 (2008), 564–575.
A.Z. Peixinho, B.C. Benato, L.G. Nonato, A.X. Falcão, Delaunay triangulation data augmentation guided by visual analytics for deep learning, 2018 31st SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), IEEE, Parana, Brazil, 2018, pp. 384–391.
N. Pezzotti, T. Höllt, J. Van Gemert, B.P.F. Lelieveldt, E. Eisemann, A. Vilanova, DeepEyes: progressive visual analytics for designing deep neural networks, IEEE Transactions on Visualization and Computer Graphics 24 (2017), 98–108.
M.C. Potter, Conceptual short term memory in perception and thought, Frontiers in Psychology 3 (2012), 113.
X. Qian, R.A. Rossi, F. Du, S. Kim, E. Koh, S. Malik, T.Y. Lee, J. Chan, Ml-based visualization recommendation: Learning to recommend visualizations from data, arXiv preprint arXiv: 2009.12316, 2020.
X. Qin, Y. Luo, N. Tang, G. Li, Making data visualization more efficient and effective: a survey, The VLDB Journal 29 (2020), 93–117.
P.E. Rauber, S.G. Fadel, A.X. Falcao, A.C. Telea, Visualizing the hidden activity of artificial neural networks, IEEE Transactions on Visualization and Computer Graphics 23 (2016), 101–110.
X. Rong, E. Adar, Visual tools for debugging neural language models, Proceedings of ICML Workshop on Visualization for Deep Learning, New York, USA, 2016.
D. Sacha, M. Kraus, D.A. Keim, M. Chen, VIS4ML: an ontology for visual analytics assisted machine learning, IEEE Transactions on Visualization and Computer Graphics 25 (2018), 385–395.
D. Sacha, M. Sedlmair, L. Zhang, J.A. Lee, D.Weiskopf, S. North, et al., Human-centered machine learning through interactive visualization, Proceedings of the 24th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, Bruges, Belgium, 2016.
W. Samek, A. Binder, G. Montavon, S. Lapuschkin, K.R. Müller, Evaluating the visualization of what a deep neural network has learned, IEEE Transactions on Neural Networks and Learning Systems 28 (2016), 2660–2673.
W. Samek, T. Wiegand, K.R. Müller, Explainable artificial intelligence: understanding, visualizing and interpreting deep learning models, arXiv preprint arXiv: 1708.08296, 2017.
P. Schueffel, Taming the beast: a scientific definition of fintech, Journal of Innovation Management 4 (2016), 32–54.
C. Seifert, A. Aamir, A. Balagopalan, D. Jain, A. Sharma, S. Grottel, et al., Visualizations of deep neural networks in computer vision: a survey, in: T. Cerquitelli, D. Quercia, F. Pasquale (Eds.), Transparent Data Mining for Big and Small Data, Springer, Cham, 2017, pp. 123–144.
Q. Shen, Y. Wu, Y. Jiang, W. Zeng, A.K.H. Lau, A. Vianova, et al., Visual interpretation of recurrent neural network on multi-dimensional time-series forecast, 2020 IEEE Pacific Visualization Symposium (PacificVis), IEEE, Tianjin, China, 2020, pp. 61–70.
H. Shi, J. Gao, X. Ren, H. Xu, X. Liang, Z. Li, J.T. Kwok, Sparsebert: rethinking the importance analysis in self-attention, arXiv preprint arXiv: 2102.12871, 2021.
D. Shin, S. He, G.M. Lee, A.B. Whinston, S. Cetintas, K.C. Lee, Enhancing social media analysis with visual data analytics: a deep learning approach, MIS Quarterly 44 (2020), 1459–1492.
B. Shneiderman, C. Plaisant, B.W. Hesse, Improving healthcare with interactive visualization, Computer 46 (2013), 58–66.
A.F. Simpao, L. Ahumada, M.A. Rehman, Big data and visual analytics in anaesthesia and health care, British Journal of Anaesthesia 115 (2015), 350–356.
K. Singh, P. Best, Anti-money laundering: using data visualization to identify suspicious activity, International Journal of Accounting Information Systems 34 (2019), 100418.
J. Skan, J. Dickerson, L. Gagliardi, Fintech and the evolving landscape: landing points for the industry, Accenture, 2016, available at: https://www.accenture.com/t20160427T053810__w__/us-en/_acnmedia/PDF-15/Accenture-Fintech-Evolving-Landscape.pdf (accessed 15 January, 2017).
C.O.S Sorzano, J. Vargas, A.P. Montano, A survey of dimensionality reduction techniques, arXiv preprint arXiv: 1403.2877, 2014.
T. Spinner, U. Schlegel, H. Schäfer, M. El-Assady, expIAIner: a visual analytics framework for interactive and explainable machine learning, IEEE Transactions on Visualization and Computer Graphics 26 (2019), 1064–1074.
H. Strobelt, S. Gehrmann, H. Pfister, A.M. Rush, LSTMVis: a tool for visual analysis of hidden state dynamics in recurrent neural networks, IEEE Transactions on Visualization and Computer Graphics 24 (2017), 667–676.
Y. Sun, J. Leigh, A. Johnson, S. Lee, Articulate: a semi-automated model for translating natural language queries into meaningful visualizations, International Symposium on Smart Graphics, Springer, Berlin, Heidelberg, 2010, pp. 184–195.
I. Sutskever, O. Vinyals, Q.V. Le, Sequence to sequence learning with neural networks, arXiv preprint arXiv: 1409.3215, 2014.
D. Varga, Fintech, the new era of financial services, Vezetéstudomány-Budapest Management Review 48 (2017), 22–32.
P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Lio, Y. Bengio, Graph attention networks, arXiv preprint arXiv: 1710.10903, 2017.
A. Vellido, The importance of interpretability and visualization in machine learning for applications in medicine and health care, Neural Computing and Applications 32 (2019), 18069–18083.
T. von Landesberger, S. Bremm, M. Kirschner, S. Wesarg, A. Kuijper, Visual analytics for model-based medical image segmentation: opportunities and challenges, Expert Systems with Applications 40 (2013), 4934–4943.
H.Q. Vu, G. Li, R. Law, Discovering highly profitable travel patterns by high-utility pattern mining, Tourism Management 77 (2020), 104008.
E.I. Walsh, Y. Chung, N. Cherbuin, L. Salvador-Carulla, Experts’ perceptions on the use of visual analytics for complex mental healthcare planning: an exploratory study, BMC Medical Research Methodology 20 (2020), 110.
J. Wang, Interpreting and diagnosing deep learning models: a visual analytics approach, Ph.D. Thesis. The Ohio State University, Columbus, Ohio, 2019.
J. Wang, L. Gou, H.W. Shen, H. Yang, DQNViz: a visual analytics approach to understand deep Q-networks, IEEE Transactions on Visualization and Computer Graphics 25 (2018), 288–298.
J. Wang, L. Gou, H. Yang, H.W. Shen, GANViz: a visual analytics approach to understand the adversarial game, IEEE Transactions on Visualization and Computer Graphics 24 (2018), 1905–1917.
J. Wang, W. Zhang, H. Yang, SCANViz: interpreting the symbol-concept association captured by deep neural networks through visual analytics, 2020 IEEE Pacific Visualization Symposium (PacificVis), IEEE, Tianjin, China, 2020, pp. 51–60.
Z.J. Wang, R. Turko, D.H. Chau, Dodrio: exploring transformer models with interactive visualization, arXiv preprint arXiv: 2103.14625, 2021.
W.N. Wassouf, R. Alkhatib, K. Salloum, S. Balloul, Predictive analytics using big data for increased customer loyalty: syriatel telecom company case study, Journal of Big Data 7 (2020), 29.
G. Wills, L. Wilkinson, AutoVis: automatic visualization, Information Visualization 9 (2010), 47–69.
W. Winn, Visualization in learning and instruction: a cognitive approach, Educational Communication and Technology Journal 30 (1982), 3–25.
K. Wongsuphasawat, D. Moritz, A. Anand, J. Mackinlay, B. Howe, J. Heer, Voyager: exploratory analysis via faceted browsing of visualization recommendations, IEEE Transactions on Visualization and Computer Graphics 22 (2015), 649–658.
K. Wongsuphasawat, D. Moritz, A. Anand, J. Mackinlay, B. Howe, J. Heer, Towards a general-purpose query language for visualization recommendation, Proceedings of the Workshop on Human-In-the-Loop Data Analytics, California, San Francisco, New York, NY, United States, 2016, pp. 1–6.
K. Wongsuphasawat, D. Smilkov, J. Wexler, J. Wilson, D. Mané, D. Fritz, et al., Visualizing dataflow graphs of deep learning models in tensorflow, IEEE Transactions on Visualization and Computer Graphics 24 (2017), 1–12.
A. Yaeli, P. Bak, G. Feigenblat, S. Nadler, H. Roitman, G. Saadoun, et al., Understanding customer behavior using indoor location analysis and visualization, IBM Journal of Research and Development 58 (2014), 3:1–3:12.
Y. Yang, T. Chen, Analysis and visualization implementation of medical big data resource sharing mechanism based on deep learning, IEEE Access 7 (2019), 156077–156088.
J. Yosinski, J. Clune, A. Nguyen, T. Fuchs, H. Lipson, Understanding neural networks through deep visualization, arXiv preprint arXiv: 1506.06579, 2015.
B. Yu, C.T. Silva, FlowSense: a natural language interface for visual data exploration within a dataflow system, IEEE Transactions on Visualization and Computer Graphics 26 (2019), 1–11.
X. Yue, Financial intelligence and strategy extraction via visual analysis approaches, Ph.D. Thesis, The Hong Kong University of Science and Technology, Hong Kong, 2019.
X. Yue, X. Shu, X. Zhu, X. Du, Z. Yu, D. Papadopoulos, et al., BiTexTract: interactive visualization for extracting bitcoin exchange intelligence, IEEE Transactions on Visualization and Computer Graphics 25 (2018), 162–171.
M.D. Zeiler, R. Fergus, Visualizing and understanding convolutional networks, European Conference on Computer Vision, Springer, Cham, 2014, pp. 818–833.
H. Zeng, Towards better understanding of deep learning with visualization, The Hong Kong University of Science and Technology, Hong Kong, 2016.
H. Zeng, H. Haleem, X. Plantaz, N. Cao, H. Qu, CNNcompara-tor: comparative analytics of convolutional neural networks, arXiv preprint arXiv: 1710.05285, 2017.
W. Zeng, C. Lin, J. Lin, J. Jiang, J. Xia, C. Turkay, et al., Revisiting the modifiable areal unit problem in deep traffic prediction with visual analytics, IEEE Transactions on Visualization and Computer Graphics 27 (2020), 839–848.
J. Zerafa, M.R. Islam, M.A. Kabir, G. Xu, ExTraVis: exploration of traffic incidents using a visual interactive system, 2021 The 25th International Conference on Information Visualisation (IV), IEEE, Sydney, Australia, 2021.
C. Zhang, Q.w. Gong, K. Koyamada, Visual analytics and prediction system based on deep belief networks for icing monitoring data of overhead power transmission lines, Journal of Visualization 23 (2020), 1087–1100.
K. Zhang, Y. Chen, C. Li, Discovering the tourists’ behaviors and perceptions in a tourism destination by analyzing photos’ visual content with a computer deep learning model: the case of Beijing, Tourism Management 75 (2019), 595–608.
S. Zhang, D. Metaxas, Large-scale medical image analytics: recent methodologies, applications and future directions 33 (2016), 98–101.
S. Zhang, L. Yao, A. Sun, Y. Tay, Deep learning based recommender system: A survey and new perspectives, ACM Computing Surveys (CSUR) 52 (2019), 1–38.
L.M. Zintgraf, T.S. Cohen, T. Adel, M. Welling, Visualizing deep neural network decisions: Prediction difference analysis, arXiv preprint arXiv: 1702.04595, 2017.
L.M. Zintgraf, T.S. Cohen, M. Welling, A new method to visualize deep neural networks, arXiv preprint arXiv: 1603.02518, 2016.
Author information
Authors and Affiliations
Corresponding author
Additional information
Peer review under responsibility of KEO (Henan) Education Technology Co. Ltd
Rights and permissions
This is an open access article distributed under the CC BY-NC 4.0 license (https://doi.org/creativecommons.org/licenses/by-nc/4.0/).
About this article
Cite this article
Islam, R., Akter, S., Ratan, R. et al. Deep Visual Analytics (DVA): Applications, Challenges and Future Directions. Hum-Cent Intell Syst 1, 3–17 (2021). https://doi.org/10.2991/hcis.k.210704.003
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.2991/hcis.k.210704.003