Skip to main content
Log in

Real-time visual analytics for in-home medical rehabilitation of stroke patient—systematic review

  • Review Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

This paper is focused on real-time visual analytics for home-based rehabilitation dedicated for brain stroke survivors. This research is at the intersection of three main domains: visual analytics for time-oriented data and dynamic visual analytics with specific focus on data analytics for rehabilitation systems. This study has emphasized the analysis of the most important research works in these domains. The studies included in this review are published between January 2008 and December 2020 that met eligibility criteria. From 243 papers retrieved from research including the Google Scholar database and manual research, 69 papers were finally included. This paper presents a classification of the reviewed research based on key features required by the visual analytics for real-time monitoring of patients. The findings suggested that real-time monitoring visual analytics for biodata captured during the rehabilitation sessions was not sufficiently addressed by previous research. To provide real-time monitoring visual analytics of biodata, the concept of a unified framework that combines the processing of batch and stream data in a distributed architecture is proposed. The system is currently under development; its validation will be carried out by an experimental study and the evaluation of the system performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Aigner W (2013) Interactive visualization and data analysis: visual analytics with a focus on time. Habilitation Thesis

  2. Aigner W, Bertone A, Miksch S, Tominski C, Schumann H (2007) Towards a concep-tual framework for visual analytics of time and time-oriented data. In: IEEE WinterSimulation Conference, pp. 721–729

  3. Aigner W, Miksch S, Müller W, Schumann H, Tominski C (2007) Visual methods for analyzing time-oriented data. IEEE Trans Vis Comput Graph 14(1):47–60

    Google Scholar 

  4. Aigner W, Miksch S, Müller W, Schumann H, Tominski C (2007) Visualizing time-oriented data—a systematic view. Comput Graph 31(3):401–409

    Google Scholar 

  5. Ali M, Jones MW, Xie X, Williams M (2019) Time Cluster: dimension reduction applied to temporal data for visual analytics. Vis Comput 35(6-8):1013–1026

    Google Scholar 

  6. Alsallakh B, Bögl M, Gschwandtner T, Miksch S, Esmael B, Arnaout A, Thon-hauser G, Zöllner P (2014) A visual analytics approach to segmenting and labeling multivariate time series data. In: EuroVA@ EuroVis

  7. Amor-Amorós A, Federico P, Miksch S (2014) TimeGraph: a data management framework for visual analytics of large multivariate time-oriented networks. In: IEEE Conference On Visual Analytics Science and Technology (VAST), pp. 217–218

  8. Andrienko G, Andrienko N, Demsar U, Dransch D, Dykes J, Fabrikant SI, Jern M, Kraak MJ, Schumann H, Tominski C (2010) Space, time and visual analytics. Int J Geogr Inf Sci 24(10):1577–1600

    Google Scholar 

  9. Bernold G, Matkovic K, Gröller, E., Raidou, R.G. (2019) preha: establishing preci-sion rehabilitation with visual analytics. In: Eurographics Workshop on VisualComputing for Biology and Medicine. The Eurographics Association. https://doi.org/10.2312/vcbm.20191234

  10. Best DM, Bohn S, Love D, Wynne A, Pike WA (2010) Real-time visualization of network behaviors for situational awareness. In: Proceedings of the seventh international symposium on visualization for cyber security, pp 79–90

    Google Scholar 

  11. Bögl M (2020) Visual analysis of periodic time series data-supporting model selection, pre-diction, imputation, and outlier detection using visual analytics. Ph.D. thesis, Wien

  12. Bögl M, Aigner W, Filzmoser P, Gschwandtner T, Lammarsch T, Miksch S, Rind A (2014) Visual analytics methods to guide diagnostics for time series model predictions. In: IEEE VIS (Visualization) Workshop on Visualization for Predictive Analytics, vol. 1

  13. Boukhelifa N, Chevalier F, Fekete JD (2010) Real-time aggregation of Wikipedia data for visual analytics. In: IEEE Symposium on Visual Analytics Science and Technology, pp 147–154

    Google Scholar 

  14. Buzzi MC, Buzzi M, Trujillo A (2015) Healthy aging through pervasive predictive ana-lytics for prevention and rehabilitation of chronic conditions. In: The 3rd Workshop onICTs for improving Patients Rehabilitation Research Techniques, pp 148–151

    Google Scholar 

  15. Caggianese G, Cuomo S, Esposito M, Franceschini M, Gallo L, Infarinato F, Minutolo A, Piccialli F, Romano P (2018) Serious games and in-cloud data analytics for the virtualization and personalization of rehabilitation treatments. IEEE Trans Ind Inf 15(1):517–526

    Google Scholar 

  16. Calderon NA, Arias-Hernandez R, Fisher B (2014) Studying animation for real-time visual analytics: a design study of social media analytics in emergency management. In: 47thHawaii International Conference on System Sciences. IEEE, pp 1364–1373

    Google Scholar 

  17. Chen Z, Zhou J, Wang X, Swanson J, Chen F, Feng D (2017) Neural net-based and safety-oriented visual analytics for time-spatial data. In: International Joint Conference On Neural Networks (IJCNN). IEEE, pp 1133–1140

    Google Scholar 

  18. Cheng S, Mueller K, Xu W (2016) A framework to visualize temporal behavioral relationships in streaming multivariate data. In: New York Scientific Data Summit (NYSDS), pp. 1–10. IEEE

  19. Chin G Jr, Chen Y, Fitzhenry E, McGary B, Pirrung M, Bruce J, Winner S (2018) A visual analytics platform and advanced visualization tools for interpreting and analyzing wind energy time-series data. IFAC-Papers OnLine 51(28):480–485

    Google Scholar 

  20. Chung S, Suh S, Park C, Kang K, Choo J, Kwon BC (2016) Revacnn: real-time visual analytics for convolutional neural network. In: KDD 16 Workshop on Interactive DataExploration and Analytics

    Google Scholar 

  21. Cook K, Thomas J (2005) Illuminating the path: the research and development agenda for visual analytics, vol 54. IEEE Computer Society

    Google Scholar 

  22. Dill J, Earnshaw R, Kasik D, Vince J, Wong PC (2012) Expanding the frontiers of visual analytics and visualization, 1st edn. Springer

    Google Scholar 

  23. Eaglin T, Cho I, Ribarsky W (2017) Space-time kernel density estimation for real-time interactive visual analytics. In: Proceedings of the 50th Hawaii International Conference On System Sciences

    Google Scholar 

  24. Ferreira C, Guimarães V, Santos A, Sousa I (2014) Gamification of stroke rehabilitation exercises using a smartphone. In: Proceedings of the 8th International Conference onPervasive Computing Technologies for Healthcare, pp. 282–285. ICST

  25. Fischer F, Keim DA (2014) Nstreamaware: real-time visual analytics for data streams to enhance situational awareness. In: Proceedings of the Eleventh Workshop on Visualiza-tion for Cyber Security, pp 65–72

    Google Scholar 

  26. Fischer F, Mansmann F, Keim DA (2012) Real-time visual analytics for event datastreams. In: Proceedings of the 27th Annual ACM Symposium on Applied Computing, pp 801–806

    Google Scholar 

  27. Frank AU (1998) Different types of “times” in gis. Spatial and temporal reasoning in geo-graphic information systems pp. 40–62

  28. Garcìa I, Casado R, Bouchachia A (2016) An incremental approach for real-time big datavisual analytics. In: IEEE 4th International Conference on Future Internet of Thingsand Cloud Workshops (FiCloudW), pp 177–182

    Google Scholar 

  29. Gotz D, Stavropoulos H (2014) Decisionflow: visual analytics for high-dimensional temporal event sequence data. IEEE Trans Vis Comput Graph 20(12):1783–1792

    PubMed  Google Scholar 

  30. Guerra-Gómez JA, Pack ML, Plaisant C, Shneiderman B (2015) Discovering temporal changes in hierarchical transportation data: visual analytics & text reporting tools. Transp Res C Emerg Technol 51:167–179

    Google Scholar 

  31. Haghighati A, Sedig K (2020) Vartta: a visual analytics system for making sense of real-time twitter data. Data 5(1):20

    Google Scholar 

  32. Hamper A, Eigner I, Wickramasinghe N, Bodendorf F (2017) Rehabilitation risk man-agement: enabling data analytics with quantified self and smart home data. In: eHealth, pp. 152–160

  33. Hasani Z (2017) Implementation of infrastructure for streaming outlier detection in big data. In: World Conference on Information Systems and Technologies, pp. 503–511. https://doi.org/10.1007/978-3-319-56538-551

  34. Hochheiser H, Shneiderman B (2004) Dynamic query tools for time series data sets: timebox widgets for interactive exploration. Inf Vis D3(1):1–18

    Google Scholar 

  35. Hoenig H, Horner RD, Duncan PW, Clipp E, Hamilton B (1999) New horizons in stroke rehabilitation research. J Rehabil Res Dev 36(1):19–31

    CAS  PubMed  Google Scholar 

  36. Jarque-Bou NJ, Vergara M, Sancho-Bru JL, Gracia-Ibánez V, Roda-Sales A (2019) A cal-ibrated database of kinematics and emg of the forearm and hand during activities of daily living. Sci Data 6(1):1–11

    Google Scholar 

  37. Johansson C, Nilsson R (2009) Visualizing real-time data designing a visual analytics tool for the stock market. Chalmers University of Technology

    Google Scholar 

  38. Jones M, Collier G, Reinkensmeyer DJ, DeRuyter F, Dzivak J, Zondervan D, Morris J (2020) Big data analytics and sensor-enhanced activity management to improve effectiveness and efficiency of outpatient medical rehabilitation. Int J Environ Res Public Health 17(3):748

    PubMed Central  Google Scholar 

  39. Kandogan E, Soroker D, Rohall S, Bak P, van Ham F, Lu J, Ship HJ, Wang CF, Lai J (2014) A reference web architecture and patterns for real-time visual analytics on large streaming data. Vis Data Anal 9017:81–95. SPIE. https://doi.org/10.1117/12.2040533

    Article  Google Scholar 

  40. Keim D, Kohlhammer J, Ellis G, Mansmann F (2010) Mastering the information age –solving problems with visual analytics. Eurographics Association

  41. Keim, D.A., Krstajic, M., Rohrdantz, C., Schreck, T.: Real-time visual analytics for textstreams. Computer46(7), 47–55 (2013)

  42. Keim DA, Mansmann F, Schneidewind J, Ziegler H (2006) Challenges in visual data analysis. In: Tenth International Conference on Information Visualisation (IV’06). IEEE, pp 9–16

    Google Scholar 

  43. Keogh E, Chakrabarti K, Pazzani M, Mehrotra S (2001) Locally adaptive dimensionality reduction for indexing large time series databases. In: ACM SIGMOD international conference on Management of data, pp 151–162

    Google Scholar 

  44. Keogh E, Lonardi S, Chiu, B.c. (2002) Finding surprising patterns in a time series database in linear time and space. In: Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, pp 550–556

    Google Scholar 

  45. Koutsofios E, North S, Truscott R, Keim D (1999) Visualizing large-scale telecommunication networks and services. In: Visualization ’99. Proceedings. IEEE, pp 457–461. https://doi.org/10.1109/VISUAL.1999.809930

    Chapter  Google Scholar 

  46. Krstajic M (2014) Visual analytics of temporal event sequences in news streams. Ph.D. thesis

  47. Latif S, Varaich ZA, Ali MA, Khan MA, Ayyaz MN (2015) Real-time health data acqui-sition and geospatial monitoring: a visual analytics approach. In: International Conference on Open Source Systems & Technologies (ICOSST). IEEE, pp 146–150

    Google Scholar 

  48. Li C, Baciu G (2014) Valid: A web framework for visual analytics of large streaming data. In: IEEE 13th International Conference on Trust, Security and Privacy in Computing and Communications, pp 686–692

    Google Scholar 

  49. Li P, Yates SN, Lovely JK, Larson DW (2015) Patient-like-mine: a real time, visual ana-lytics tool for clinical decision support. In: IEEE International Conference on Big Data (BigData), pp 2865–2867

    Google Scholar 

  50. Liang J, Fuhry D, Maung D, Borstad A, Crawfis R, Gauthier L, Nandi A, Parthasarathy S (2016) Data analytics framework for a game-based rehabilitation system. In: Proceedings of the 6th International Conference on Digital Health Conference, pp. 67–76

  51. Lin J, Keogh E, Lonardi S (2005) Visualizing and discovering non-trivial patterns in large time series databases. Inf Vis 4(2):61–82

    Google Scholar 

  52. Lin J, Keogh E, Lonardi S, Lankford JP, Nystrom DM (2004) Visually mining and monitor-ing massive time series. In: Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, pp 460–469

    Google Scholar 

  53. Lindsay MP, Norrving B, Sacco RL, Brainin M, Hacke W, Martins S, Pandian J, Feigin V (2019) World stroke organization (wso): Global stroke fact sheet 2019. Int J Stroke 14(8):806–817. https://doi.org/10.1177/1747493019881353

    Article  PubMed  Google Scholar 

  54. Liu Y, Hill D, Myers J, Minsker B (2010) Integrated real time geospatial sensor web and visual analytics for environmental decision support. In: World Environmental and Water Resources Congress 2010: Challenges of Change, pp 325–334

    Google Scholar 

  55. Loh P, Allan L (2005) Medical informatics system with wireless sensor network-enabled for hos-pitals. In: International Conference on Intelligent Sensors, Sensor Networks and Information Processing. IEEE, pp 265–270

    Google Scholar 

  56. Maimon O, Rokach L (2005) Data mining and knowledge discovery handbook. Springer-Verlag, Berlin

    Google Scholar 

  57. Malan DJ, Fulford-Jones T, Welsh M, Moulton S (2004) Codeblue: an ad hoc sensor network infrastructure for emergency medical care. In: International workshop on wearable and implantable body sensor networks

    Google Scholar 

  58. Marz N, Warren J (2015) Big Data: principles and best practices of scalable real-time data systems. Manning Publications Co, New York

    Google Scholar 

  59. McLachlan P, Munzner T, Koutsofios E, North S (2008) Liverac: interactive visual exploration of system management time-series data. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp 1483–1492

    Google Scholar 

  60. Medeiros L, Lauer TR, Psaltis D, Ozel F (2018) Principal component analysis as a tool for characterizing black hole images and variability. Astrophys J 864(1):7

    Google Scholar 

  61. Meschenmoser P, Buchmuller JF, Seebacher D, Wikelski M, Keim DA (2020) Multisegva:Using visual analytics to segment biologging time series on multiple scales. IEEE Trans Vis Comput Graph 27(2):1623–1633

    Google Scholar 

  62. Miksch S, Aigner W (2014) A matter of time: applying a data–users–tasks design triangle to visual analytics of time-oriented data. Comput Graph 38:286–290

    Google Scholar 

  63. Miksch S, Horn W, Popow C, Paky F (1996) Utilizing temporal data abstraction for data vali-dation and therapy planning for artificially ventilated newborn infants. Artif Intell Med 8(6):543–576

    CAS  PubMed  Google Scholar 

  64. Patel P, Keogh E, Lin J, Lonardi S (2002) Mining motifs in massive time series databases. In: IEEE International Conference on Data Mining. Proceedings, pp 370–377

    Google Scholar 

  65. Piotrowicz E, Jasionowska A, Banaszak-Bednarczyk M, Gwilkowska J, Piotrowicz R (2012) Ecg telemonitoring during home-based cardiac rehabilitation in heart failure patients. J Telemed Telecare 18(4):193–197

    PubMed  Google Scholar 

  66. Rahman M, Wadhwa B, Kankanhalli A, Hua YC, Kei CK, Hoon LJ, Jayakkumar S, Lin CC (2016) Gear analytics: a clinician dashboard for a mobile game assisted rehabilitation system. In: 4th International Conference on User Science and Engineering (i-USEr). IEEE, pp 193–198

    Google Scholar 

  67. Reddy, C.K., Aggarwal, C.C.: Healthcare data analytics, vol. 12, first edn. Chapman and Hall/CRC Data Mining and Knowledge Discovery Series (2015). https://doi.org/10.1201/b18588

  68. Rind A (2017) A software framework for visual analytics of time-oriented data. Ph.D. thesis, Wien

  69. Saraee E (2019) Data analytics for image visual complexity and kinect-based videos of rehabili-tation exercises. Ph.D. thesis. Boston University

    Google Scholar 

  70. Seanosky J, Guillot I, Boulanger D, Guillot R, Guillot C, Kumar V, Fraser SN, Aljojo N, Munshi A et al (2017) Real-time visual feedback: a study in coding analytics. In: IEEE 17th International Conference on Advanced Learning Technologies (ICALT), pp 264–266

    Google Scholar 

  71. Shilpika, F., Fujiwara, T., Sakamoto, N., Nonaka, J., Ma, K.L.: A visual analytics approach to monitor time-series data with incremental and progressive functional data analysis. arXiv- CS - Human-Computer Interaction (2020)

    Google Scholar 

  72. Shnayder V, Chen BL, Lorincz K (2005) Sensor networks for medical care. Tech. Rep. HarvardComputer Science Group TR-08-05. Division of Engineering and Applied Sciences

  73. Shneiderman B (1996) The eyes have it: A task by data type taxonomy for information vi-sualizations. In: IEEE Symposium on Visual Languages, pp 336–343. https://doi.org/10.1109/VL.1996.545307

    Chapter  Google Scholar 

  74. Sibolla BH, Coetzee S, Van Zyl TL (2018) A framework for visual analytics of spatio-temporal sensor observations from data streams. ISPRS Int J Geo Inf 7(12):475

    Google Scholar 

  75. Snyder L (2020) Predictive visual analytics of social media data for supporting real-time situa-tional awareness. Ph.D. thesis. Purdue University Graduate School

    Google Scholar 

  76. Snyder LS, Karimzadeh M, Chen R, Ebert DS (2019) City-level geolocation of tweets for real-time visual analytics. In: Proceedings of the 3rd ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge Discovery, pp 85–88

    Google Scholar 

  77. Steed CA, Potok TE, Patton RM, Goodall JR, Maness C, Senter J (2012) Interactive visual analysis of high throughput text streams. In: International visual text analytics workshop, pp 1–4

    Google Scholar 

  78. Stephens P, Young J (2020) Real-time visual analytics: an experiential learning activity for undergraduates. Inf Syst Educ J 18(6):4–12

    Google Scholar 

  79. Strohbach M, Ziekow H, Gazis V, Akiva N (2015) Towards a big data analytics framework for iot and smart city applications. In: Modeling and processing for next-generation big-data technologies. Springer, pp 257–282

    Google Scholar 

  80. Sun BB, Ielonka E, Fritz A, Schofield M, Ringel B, Armstrong B, Ho SS, Bre-itzman A, Snouffer J, Kirschner J et al (2018) Visual analytics for real-time flight behavior threat assessment. In: IEEE International Conference on Big Data, pp 3607–3612

    Google Scholar 

  81. Takami R, Takama Y (2020) Proposal and evaluation of visual analytics interface for time-series data based on trajectory representation. IEICE Trans Inf Syst 103(1):142–151

    Google Scholar 

  82. Tamayo-Serrano P, Garbaya S, Blazevic P (2018) Gamified in-home rehabilitation for stroke survivors: analytical review. Int J Serious Games 5(1)

  83. Tamayo-Serrano P, Jamshidi Farsani H, Garbaya S, Lim T, Blazevic P (2019) Framework of visual analytics for medical rehabilitation. In: Journée Visualisation “19”. Telecom ParisTech, Paris

    Google Scholar 

  84. Tang H, Wei S, Zhou Z, Qian ZC, Chen YV (2019) Treeroses: outlier-centric monitoring and analysis of periodic time series data. J Vis 22(5):1005–1019

    Google Scholar 

  85. Thomas JJ, Cook KA (2006) A visual analytics agenda. IEEE Comput Graph Appl 26(1):10–13

    PubMed  Google Scholar 

  86. Tukey JW (1977) Exploratory data analysis, vol 2. Addison-Wesley series in behavioral sciences, Reading

    Google Scholar 

  87. Urquiaga RR, Valdivia AMC, Zapana RA (2017) A visual analytics approach for exploration of high-dimensional time series based on neighbor-joining tree. In: IEEE International Sym-posium on Signal Processing and Information Technology (ISSPIT), pp 325–330

    Google Scholar 

  88. Valdés BA, Shirzad N, Hung CT, Van der Loos HM, Glegg SM, Reeds E (2015) Visu-alisation of two-dimensional kinematic data from bimanual control of a commercialgaming system used in post-stroke rehabilitation. In: International Conference on Virtual Rehabil-itation (ICVR). IEEE, pp 243–250

    Google Scholar 

  89. Vuckovic M, Schmidt J (2020) Visual analytics approach to comprehensive meteorological time-series analysis. Data 5(4):94

    Google Scholar 

  90. Wagner M, Slijepcevic D, Horsak B, Rind A, Zeppelzauer M, Aigner W (2018) Kavagait: knowledge-assisted visual analytics for clinical gait analysis. IEEE Trans Vis Comput Graph 25(3):1528–1542

    PubMed  Google Scholar 

  91. Webga K, Lu A (2015) Discovery of rating fraud with real-time streaming visual analytics. In: IEEE Symposium on Visualization for Cyber Security (VizSec), pp 1–8

    Google Scholar 

  92. Willmann RD, Lanfermann G, Saini P, Timmermans A, te Vrugt J, Winter S (2007) Homestroke rehabilitation for the upper limbs. In: 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp 4015–4018

    Google Scholar 

  93. Wong PC, Thomas J (2004) Visual analytics. IEEE Comput Graph Appl 5:20–21

    Google Scholar 

  94. Woo WL, Koh B, Gao B, Nwoye E, Wei B, Dlay S (2020) Early warning of health condition and visual analytics for multivariable vital signs. In: International Conference on Computing,Networks and Internet of Things, pp 206–211

    Google Scholar 

  95. Yeon H, Son H, Jang Y (2020) Visual imputation analytics for missing time-series data in Bayesian network. In: IEEE International Conference on Big Data and Smart Computing (BigComp), pp 303–310

    Google Scholar 

  96. Zhang Y, Li G, Lai C, Liu Q, Chen S, Feng L, Ye T, Chen S, Zuo R, Zhang Z et al (2016) Stad-hd: spatial temporal anomaly detection for heterogeneous data through visual analytics. In: Proceedings of IEEE VIS (Visualization)

    Google Scholar 

  97. Zhao K, Ward M, Rundensteiner E, Higgins H (2016) Mavis: machine learning aided multi-model framework for time series visual analytics. Electr Imaging 2016(1):1–10

    Google Scholar 

  98. Zheng H, Davies R, Zhou H, Hammerton J, Mawson SJ, Ware PM, Black ND, Ec-cleston C, Hu H, Stone T et al (2006) Smart project: application of emerging information and communication technology to home-based rehabilitation for stroke patients. Int J Disabil Hum Dev 5(3):271–276

    Google Scholar 

  99. Zohrevandi E, Westin CA, Lundberg J, Ynnerman A (2020) Design of a real time visual analytics support tool for conflict detection and resolution in air traffic control. In: Joint Conferences of Eurographics and Eurovis

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Maryam Boumrah.

Ethics declarations

Ethics approval

This article does not contain any studies with human participants performed by the authors. However, the data used in this study were taken from the research already published by J.Jarque-Bou et al. [98].

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Boumrah, M., Garbaya, S. & Radgui, A. Real-time visual analytics for in-home medical rehabilitation of stroke patient—systematic review. Med Biol Eng Comput 60, 889–906 (2022). https://doi.org/10.1007/s11517-021-02493-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-021-02493-w

Keywords

Navigation