Abstract
The new generation of intelligent manufacturing systems requires a deep integration of human-cyber-physical spaces. Visual analytics plays a critical role in effectively navigating humans through twin data, enabling them to make better decisions and discover new knowledge, contributing to the evolution of digital twins. However, due to the lack of study of visual analytics components and their inter-relations in the context of the digital twin, the visual analytic process of digital twin systems lacks guidance to fully integrate human domain experience and cognitive abilities into the intelligent decision-making process of digital twin systems. Therefore, this paper proposes a conceptual visual analytics framework for digital twins (DTVA). Moreover, from the perspective of data and models in digital twins, the works concerning key factors of visual analytics, including digital-twin visual representation (DTV), analytical reasoning (DTA), and human–machine interface (DTI), are analyzed from data flow, task flow, and visual flow. At last, a case study of DTVA-based anomaly detection, which focuses on the abnormality detection of crane operation status, is presented. This work can support digital twin visual analytics, put forward ideas for the direction of visual analytics, and provide a promising direction for digital twin visualization services.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abreu, F. H., Soares, A., Paulovich, F. V., & Matwin, S. (2021). A trajectory scoring tool for local anomaly detection in maritime traffic using visual analytics. ISPRS International Journal of Geo-Information, 10(6), 412–433. https://doi.org/10.3390/ijgi10060412
Card, M. (1999). Readings in information visualization: Using vision to think. Morgan Kaufmann.
Choi, S. H., Park, K.-B., Roh, D. H., Lee, J. Y., Mohammed, M., Ghasemi, Y., & Jeong, H. (2022). An integrated mixed reality system for safety-aware human-robot collaboration using deep learning and digital twin generation. Robotics and Computer-Integrated Manufacturing, 73, 102258–102277. https://doi.org/10.1016/j.rcim.2021.102258
Decisionflow. (2014). Visual analytics for high-dimensional temporal event sequence data. IEEE transactions on visualization and computer graphics, 20(12), 1783–1792. https://doi.org/10.1109/tvcg.2014.2346682
Fan, Y., Yang, J., Chen, J., Hu, P., Wang, X., Xu, J., & Zhou, B. (2021). A digital-twin visualized architecture for flexible manufacturing system. Journal of Manufacturing Systems, 60, 176–201. https://doi.org/10.1016/j.jmsy.2021.05.010
Gao, Z., Li, J., Dong, M., Yang, R., & Liu, L. (2022). Human-system interaction based on eye tracking for a virtual workshop. Sustainability, 14(11), 6841–6859. https://doi.org/10.3390/su14116841
Grieves, M. (2014). Digital twin: Manufacturing excellence through virtual factory replication. White Paper, 1(2014), 1–7. https://doi.org/10.5281/zenodo.1493930
Haber, R. B., & McNabb, D. A. (1990). Visualization idioms: A conceptual model for scientific visualization systems. Visualization in Scientific Computing, 74, 74–93.
Han, Z., Li, Y., Yang, M., Yuan, Q., Ba, L., & Xu, E. (2020). Digital twin-driven 3d visualization monitoring and traceability system for general parts in continuous casting machine. Journal of Advanced Mechanical Design, Systems, and Manufacturing, 14(7), 1–15. https://doi.org/10.1299/jamdsm.2020jamdsm0100
He, Y., Guo, J., & Zheng, X. (2018). From surveillance to digital twin: Challenges and recent advances of signal processing for industrial internet of things. IEEE Signal Processing Magazine, 35(5), 120–129. https://doi.org/10.1109/msp.2018.2842228
Hu, F., Hao, Q., Sun, Q., Cao, X., Ma, R., Zhang, T., & Lu, J. (2016). Cyberphysical system with virtual reality for intelligent motion recognition and training. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 47(2), 347–363. https://doi.org/10.1109/tsmc.2016.2560127
Jiang, J., Li, H., Mao, Z., Liu, F., Zhang, J., Jiang, Z., & Li, H. (2022). A digital twin auxiliary approach based on adaptive sparse attention network for diesel engine fault diagnosis. Scientific Reports, 12(1), 1–18. https://doi.org/10.1038/s41598-021-04545-5
Keim, D., & Zhang, L. (2011). Solving problems with visual analytics: Challenges and applications. Proceedings of the 11th international conference on knowledge management and knowledge technologies (pp. 1–4).
Keim, D. A., Mansmann, F., Schneidewind, J., Thomas, J., & Ziegler, H. (2008). Visual analytics: Scope and challenges. Lecture Notes in Computer Science, 4404, 76–90. https://doi.org/10.1007/978-3-540-71080-6_6
Lei, Z., Zhou, H., Hu, W., Liu, G.-P., Guan, S., & Feng, X. (2021). Toward a web-based digital twin thermal power plant. IEEE Transactions on Industrial Informatics, 18(3), 1716–1725. https://doi.org/10.1109/tii.2021.3086149
Li, L., Ma, H., Wei, M., Zhang, X., Chen, Q., Xin, Y., et al. (2022). Thermal power plant turbine rotor digital twin automation construction and monitoring system. Mathematical Problems in Engineering, 2022, 1–11. https://doi.org/10.1155/2022/8527281
Liu, D., Du, Y., Chai, W., Lu, C., & Cong, M. (2022a). Digital twin and data-driven quality prediction of complex die-casting manufacturing. IEEE Transactions on Industrial Informatics, 18(11), 8119–8128. https://doi.org/10.1109/TII.2022.3168309
Liu, J., Zheng, H., Jiang, Y., Liu, T., & Bao, J. (2022b). Visual analytics approach for crane anomaly detection based on digital twin. International conference on cooperative design, visualization and engineering (pp. 1–12).
Liu, M., Li, X., Li, J., Liu, Y., Zhou, B., & Bao, J. (2022c). A knowledge graph-based data representation approach for iiot-enabled cognitive manufacturing. Advanced Engineering Informatics, 51, 101515–101527. https://doi.org/10.1016/j.aei.2021.101515
Liu, S., Bao, J., & Zheng, P. (2023). A review of digital twin-driven machining: From digitization to intellectualization. Journal of Manufacturing Systems, 67, 361–378. https://doi.org/10.1016/j.jmsy.2023.02.010
Liu, S., Lu, S., Li, J., Sun, X., Lu, Y., & Bao, J. (2021). Machining process-oriented monitoring method based on digital twin via augmented reality. The International Journal of Advanced Manufacturing Technology, 113(11), 3491–3508. https://doi.org/10.1007/s00170-021-06838-5
Liu, S., Lu, Y., Li, J., Shen, X., Sun, X., & Bao, J. (2023). A blockchain-based interactive approach between digital twin-based manufacturing systems. Computers & Industrial Engineering, 175, 108827–108841. https://doi.org/10.1016/j.cie.2022.108827
Liu, S., Lu, Y., Shen, X., & Bao, J. (2023). A digital thread-driven distributed collaboration mechanism between digital twin manufacturing units. Journal of Manufacturing Systems, 68, 145–159. https://doi.org/10.1016/j.jmsy.2023.02.014
Lu, Y., Liu, C., Kevin, I., Wang, K., Huang, H., & Xu, X. (2020). Digital twin-driven smart manufacturing: Connotation, reference model, applications and research issues. Robotics and Computer-Integrated Manufacturing, 61, 101837–101851. https://doi.org/10.1016/j.rcim.2019.101837
Maciel, F., Lourenço, A., Carvalho, P., & Melo, P. (2017). Visual and interactive concerns for vr applications: A case study. International conference of design, user experience, and usability (pp. 510–523).
Riveiro, M.J. (2011). Visual analytics for maritime anomaly detection (Unpublished doctoral dissertation). Örebro universitet.
Tao, F., Liu, W., Zhang, M., Hu, T., Qi, Q., Zhang, H., & Huang, Z. (2019). Fivedimension digital twin model and its ten applications. Computer Integrated Manufacturing Systems, 25(1), 1–18. https://doi.org/10.1016/b978-0-12-817630-6.00003-5
Thomas, J.J. (2005). Illuminating the path:[the research and development agenda for visual analytics]. IEEE Computer Society.
Tong, X., Liu, Q., Pi, S., & Xiao, Y. (2020). Real-time machining data application and service based on IMT digital twin. Journal of Intelligent Manufacturing, 31(5), 1113–1132. https://doi.org/10.1007/s10845-019-01500-0
Torres, J. F., Hadjout, D., Sebaa, A., Martínez-Álvarez, F., & Troncoso, A. (2021). Deep learning for time series forecasting: A survey. Big Data, 9(1), 3–21. https://doi.org/10.1089/big.2020.0159
Venna, S., Gottumukkala, R., & Raghavan, V. (2016). Visual analytic decisionmaking environments for large-scale time-evolving graphs. Handbook of statistics (Vol. 35, pp. 81–115). Elsevier.
Victor, V. S., Schmeißer, A., Leitte, H., & Gramsch, S. (2022). Visual parameter space analysis for optimizing the quality of industrial nonwovens. IEEE Computer Graphics and Applications, 42(2), 56–67. https://doi.org/10.1109/MCG.2022.3155867
Wu, M., Su, W., Chen, L., Liu, Z., Cao, W., & Hirota, K. (2019). Weight-adapted convolution neural network for facial expression recognition in human-robot interaction. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 51(3), 1473–1484. https://doi.org/10.1007/978-3-030-61577-2_5
Xiong, J., Hsiang, E.-L., He, Z., Zhan, T., & Wu, S.-T. (2021). Augmented reality and virtual reality displays: Emerging technologies and future perspectives. Light: Science & Applications, 10(1), 216–246. https://doi.org/10.1038/s41377-021-00658-8
Xu, K., Wang, Y., Yang, L., Wang, Y., Qiao, B., Qin, S., & Qu, H. (2019). Clouddet: Interactive visual analysis of anomalous performances in cloud computing systems. IEEE Transactions on Visualization and Computer Graphics, 26(1), 1107–1117. https://doi.org/10.1109/TVCG.2019.2934613
Xu, P., Mei, H., Ren, L., & Chen, W. (2016). Vidx: Visual diagnostics of assembly line performance in smart factories. IEEE Transactions on Visualization and Computer Graphics, 23(1), 291–300. https://doi.org/10.1109/TVCG.2016.2598664
Yi, J. S., Ah Kang, Y., Stasko, J., & Jacko, J. A. (2007). Toward a deeper understanding of the role of interaction in information visualization. IEEE Transactions on Visualization and Computer Graphics, 13(6), 1224–1231. https://doi.org/10.1109/TVCG.2007.70515
Zhan, P., Wang, S., Wang, J., Qu, L., Wang, K., Hu, Y., & Li, X. (2021). Temporal anomaly detection on iiot-enabled manufacturing. Journal of Intelligent Manufacturing, 32(6), 1669–1678. https://doi.org/10.1007/s10845-021-01768-1
Zhang, T., Ma, Y., Xu, D., Liu, C., & Ding, J. (2020). ihppvis: Interactive visual analysis of industrial data in heavy plate production. IFAC-PapersOnLine, 53(2), 12050–12055. https://doi.org/10.1016/j.ifacol.2020.12.745
Zhang, Y., Zhang, Y., Xia, W., Wu, B., & Zhang, D. (2021). Monitoring and visual analysis of processing data based on time-position mapping. China Mechanical Engineering, 32(20), 2449–2457. https://doi.org/10.3969/j.issn.1004-132X.2021.20.007
Zhou, J., Zhou, Y., Wang, B., & Zang, J. (2019). Human-cyber-physical systems (hcpss) in the context of new-generation intelligent manufacturing. Engineering, 5(4), 624–636. https://doi.org/10.1016/j.eng.2019.07.015
Zhou, X., Xu, X., Liang, W., Zeng, Z., Shimizu, S., Yang, L. T., & Jin, Q. (2022). Intelligent small object detection for digital twin in smart manufacturing with industrial cyber-physical systems. IEEE Transactions on Industrial Informatics, 18(2), 1377–1386. https://doi.org/10.1109/TII.2021.3061419
Zhu, Z., Xi, X., Xu, X., & Cai, Y. (2021). Digital twin-driven machining process for thinwalled part manufacturing. Journal of Manufacturing Systems, 59, 453–466. https://doi.org/10.1016/j.jmsy.2021.03.015
Židek, K., Pitel’, J., Adámek, M., Lazorík, P., & Hošovský, A. (2020). Digital twin of experimental smart manufacturing assembly system for industry 4.0 concept. Sustainability, 12(9), 3658–3674. https://doi.org/10.3390/su12093658
Acknowledgements
The authors acknowledge the funding support from the National Key Research and Development Program of China (No. 2019YFB1706300).
Author information
Authors and Affiliations
Contributions
Conceptualization: HZ, JB; Methodology: HZ; Formal analysis and investigation: HZ; JL; Writing—original draft preparation: HZ; Writing—review and editing: JB; Funding acquisition: JB; Supervision: TL, JB.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Zheng, H., Liu, T., Liu, J. et al. Visual analytics for digital twins: a conceptual framework and case study. J Intell Manuf 35, 1671–1686 (2024). https://doi.org/10.1007/s10845-023-02135-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10845-023-02135-y