Journal of Signal Processing Systems

, Volume 90, Issue 8–9, pp 1167–1178 | Cite as

Data and Decision Intelligence for Human-in-the-Loop Cyber-Physical Systems: Reference Model, Recent Progresses and Challenges

  • Meng Ma
  • Weilan Lin
  • Disheng Pan
  • Yangxin Lin
  • Ping WangEmail author
  • Yuchen Zhou
  • Xiaoxing Liang


With the rapid development of sensing technology, Cyber-Physical Systems (CPS) are connecting our real-world and cyber spaces by real-time situation awareness and intelligent control. In this process, one of the major challenge is how to make fast, accurate and intelligent decisions based on high-dimension, speed and volume sensing data stream. In this paper, we put human into the traditional CPS data process model and formulate a closed-loop computing paradigm for CPS data and decision intelligence. We propose a human-in-the-loop reference model for CPS, which extends the traditional cyber-physical interaction into a closed-loop process based on cyber, physical and human factors. We define the key features of human-in-the-loop CPS, summarize it as three aspects: semantic, interactive, iterative and analyze the major challenges from the perspective of data characteristics. Recent progresses in three typical application domains are reviewed and examined for their decision models and whether they have solved the target issues of human-in-the-loop CPS. According to the review and comparison, the paper finally summarizes several key future opportunities to establish an intelligent human-in-the-loop CPS.


Cyber-physical systems Data intelligence Decision-making Human-in-the-loop 



This work is supported by National Key R&D Program of China (Grant no.2017YFB1200700), National Natural Science Foundation of China (Grant no.61701007), China Postdoctoral Science Foundation (Grant no.2016M600865) and IBM Shared University Research Project.


  1. 1.
    Atzori, L., Iera, A., & Morabito, G. (2010). The internet of things: A survey. Computer Networks, 54, 2787–2805.CrossRefzbMATHGoogle Scholar
  2. 2.
    Rajkumar, R.R., Lee, I., Sha, L., and Stankovic, J.. (2010) Cyber-physical systems: the next computing revolution. In 47th ACM Design Automation Conference. 731–736.Google Scholar
  3. 3.
    Sowe, S.K., Simmon, E., Zettsu, K., de Vaulx, F., Bojanova, I.: Cyber-Physical-Human Systems: Putting People in the Loop. IT Professional 18, 10-13 (2016).Google Scholar
  4. 4.
    Rowley, J. (2007). The wisdom hierarchy: representations of the DIKW hierarchy. Journal of Information Science, 33, 163–180.CrossRefGoogle Scholar
  5. 5.
    Tsai, C.-W., Lai, C.-F., Chiang, M.-C., & Yang, L. T. (2014). Data mining for internet of things: A survey. Communications Surveys & Tutorials, IEEE, 16, 77–97.CrossRefGoogle Scholar
  6. 6.
    Luckham, D. C. (2001). The Power of Events: An Introduction to Complex Event Processing in Distributed Enterprise Systems. Boston: Addison-Wesley Longman Publishing Co., Inc..Google Scholar
  7. 7.
    Ma, M., Wang, P., Chu, C.-H.. (2013) Data Management for Internet of Things: Challenges, Approaches and Opportunities. In: IEEE International Conference on Internet of Things (iThings). 1144–1151.Google Scholar
  8. 8.
    Niles, I., Pease, A.. (2001) Towards a standard upper ontology. In: Proceedings of the international conference on Formal Ontology in Information Systems-Volume 2001. 2–9. ACM.Google Scholar
  9. 9.
    Doerr, M. (2003). The CIDOC conceptual reference module: an ontological approach to semantic interoperability of metadata. AI Magazine, 24, 75.Google Scholar
  10. 10.
    Li, Z., Chu, C.-H., Yao, W., Behr, R.A.. (2010) Ontology-driven event detection and indexing in smart spaces. In: IEEE International Conference on Semantic Computing (ICSC). 285–292.Google Scholar
  11. 11.
    Hasan, S., & Curry, E. (2014). Approximate Semantic Matching of Events for the Internet of Things. ACM Transactions on Internet Technology, 14, 1–23.CrossRefGoogle Scholar
  12. 12.
    Cugola, G., & Margara, A. (2012). Processing flows of information: From data stream to complex event processing. ACM Computing Surveys (CSUR), 44, 15.CrossRefGoogle Scholar
  13. 13.
    Dindar, N., Fischer, P.M., Soner, M., Tatbul, N.. (2011) Efficiently correlating complex events over live and archived data streams. In: 5th ACM international conference on Distributed event-based system. 243–254.Google Scholar
  14. 14.
    Peng, S., Li, Z., Li, Q., Chen, Q., Pan, W., Liu, H., Nie, Y.. (2011) Event detection over live and archived streams. Web-Age Information Management. 566–577. Springer.Google Scholar
  15. 15.
    Stankovic, J. A. (2014). Research directions for the internet of things. IEEE Internet of Things Journal, 1, 3–9.CrossRefGoogle Scholar
  16. 16.
    Singh, M.P., Hoque, M.A., Tarkoma, S.. (2016) A survey of systems for massive stream analytics. arXiv preprint arXiv:1605.09021.Google Scholar
  17. 17.
    Krempl, G., Žliobaite, I., Brzeziński, D., Hüllermeier, E., Last, M., Lemaire, V., Noack, T., Shaker, A., Sievi, S., & Spiliopoulou, M. (2014). Open challenges for data stream mining research. ACM SIGKDD explorations newsletter, 16, 1–10.CrossRefGoogle Scholar
  18. 18.
    Kim, H., Choi, B. S., & Huh, M. Y. (2016). Booster in High Dimensional Data Classification. IEEE Transactions on Knowledge and Data Engineering, 28, 29–40.CrossRefGoogle Scholar
  19. 19.
    Zámečníková, E., Kreslíková, J.. (2015) Comparison of platforms for high frequency data processing. In: IEEE 13th International Scientific Conference on Informatics. 296–301.Google Scholar
  20. 20.
    Mykland, P. A., & Zhang, L. (2012). The econometrics of high frequency data. Statistical Methods for Stochastic Differential Equations, 124, 109.MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Perera, C., Zaslavsky, A., Christen, P., & Georgakopoulos, D. (2014). Context aware computing for the internet of things: A survey. IEEE Communications Surveys & Tutorials, 16, 414–454.CrossRefGoogle Scholar
  22. 22.
    Ma, M., Wang, P., Chu, C.-H., & Liu, L. (2015). Efficient Multi-Pattern Event Processing over High-Speed Train Data Streams. IEEE Internet of Things Journal, 2(4), 295–309.CrossRefGoogle Scholar
  23. 23.
    Zhou, Q., Simmhan, Y., Prasanna, V.. (2013) Towards hybrid online on-demand querying of realtime data with stateful complex event processing. In: IEEE International Conference on Big Data. 199–205.Google Scholar
  24. 24.
    Zhang, Y., Qiu, M., Tsai, C.-W., Hassan, M.M., Alamri, A.. (2015) Health-CPS: Healthcare cyber-physical system assisted by cloud and big data. IEEE Systems Journal.Google Scholar
  25. 25.
    Leitão, P., Colombo, A. W., & Karnouskos, S. (2016). Industrial automation based on cyber-physical systems technologies: Prototype implementations and challenges. Computers in Industry, 81, 11–25.CrossRefGoogle Scholar
  26. 26.
    Chae, S., Yang, Y., Byun, J., Han, T.D.. (2016) Personal Smart Space: IoT Based User Recognition and Device Control. In: IEEE Tenth International Conference on Semantic Computing (ICSC). 181–182.Google Scholar
  27. 27.
    Allen, T.. (2016) The NIST Smart Space Project.Google Scholar
  28. 28.
    Chen, G., Wang, E., Sun, X., & Tang, Y. (2015). An intelligent analysis and mining system for urban lighting information. International Journal of Smart Home, 9, 253–262.CrossRefGoogle Scholar
  29. 29.
    Sim, J.M., Lee, Y., Kwon, O.. (2015) Acoustic sensor based recognition of human activity in everyday life for smart home services. International Journal of Distributed Sensor Networks.Google Scholar
  30. 30.
    Chetty, G., White, M., & Akther, F. (2015). Smart Phone Based Data Mining for Human Activity Recognition. Procedia Computer Science, 46, 1181–1187.CrossRefGoogle Scholar
  31. 31.
    Azzi, S., Bouzouane, A., Giroux, S., Dallaire, C., Bouchard, B.. (2014) Human activity recognition in big data smart home context. In: IEEE International Conference on Big Data. 1–8.Google Scholar
  32. 32.
    Moutacalli, M.T., Bouzouane, A., Bouchard, B.. (2014) New frequent pattern mining algorithm tested for activities models creation. In: IEEE Symposium on Computational Intelligence in Healthcare and e-health (CICARE). 69–76.Google Scholar
  33. 33.
    Chen, L., Cheung, W.K.. (2014) Recovering Human Mobility Flow Models and Daily Routine Patterns in a Smart Environment. In: IEEE International Conference on Data Mining Workshop (ICDMW). 541–548.Google Scholar
  34. 34.
    Chen, Y.-C., Peng, W.-C., Huang, J.-L., & Lee, W.-C. (2015). Significant correlation pattern mining in smart homes. ACM Transactions on Intelligent Systems and Technology (TIST), 6, 35.Google Scholar
  35. 35.
    Cook, D. J., & Krishnan, N. (2014). Mining the home environment. Journal of Intelligent Information Systems, 43, 503–519.CrossRefGoogle Scholar
  36. 36.
    Kulkarni, G., Gode, P., Reddy, J. P., & Deshmukh, M. (2015). Android Based Smart Home System. International Journal of Current Engineering and Technology, 5, 1022–1025.Google Scholar
  37. 37.
    Bourobou, S. T. M., & Yoo, Y. (2015). User activity recognition in smart homes using pattern clustering applied to temporal ANN algorithm. Sensors, 15, 11953–11971.CrossRefGoogle Scholar
  38. 38.
    Saranya, P., & Thara, L. (2015). Recongnition of Complex Human Activities using visual and Sequence Pattern Mining. International Journal of Research in Computer Applications and Robotics, 3(2), 22–29.Google Scholar
  39. 39.
    Ma, M., Wang, P., Chu, C.-H.. (2015) LTCEP: Efficient Long-Term Event Processing for Internet of Things Data Streams. In: IEEE International Conference on Internet of Things (iThings). 548–555.Google Scholar
  40. 40.
    Sussman, J.S.. (2008) Perspectives on intelligent transportation systems (ITS). Springer Science & Business Media.Google Scholar
  41. 41.
    El Faouzi, N.-E., Leung, H., Kurian, A.: Data fusion in intelligent transportation systems: Progress and challenges–A survey. Information Fusion 12, 4-10 (2011).Google Scholar
  42. 42.
    Gong, X., Liu, X.. (2003) A data mining based algorithm for traffic network flow forecasting. In: IEEE Intelligent Transportation Systems conference. 193–198.Google Scholar
  43. 43.
    Tan, H., Wu, Y., Shen, B., Jin, P. J., & Ran, B. (2016). Short-term traffic prediction based on dynamic tensor completion. IEEE Transactions on Intelligent Transportation Systems, 17, 2123–2133.CrossRefGoogle Scholar
  44. 44.
    Nunes, A. A., Dias, T. G., & e Cunha, J. F. (2016). Passenger Journey Destination Estimation From Automated Fare Collection System Data Using Spatial Validation. IEEE Transactions on Intelligent Transportation Systems, 17, 133–142.CrossRefGoogle Scholar
  45. 45.
    Shi, Q., & Abdel-Aty, M. (2015). Big data applications in real-time traffic operation and safety monitoring and improvement on urban expressways. Transportation Research Part C: Emerging Technologies, 58, 380–394.CrossRefGoogle Scholar
  46. 46.
    Ashokkumar, K., Sam, B., & Arshadprabhu, R. (2015). Cloud based intelligent transport system. Procedia Computer Science, 50, 58–63.CrossRefGoogle Scholar
  47. 47.
    Zhang, T., Xia, Y., Zhu, Q., Liu, Y., Shen, J.. (2014) Mining related information of traffic flows on lanes by k-medoids. In: 11th IEEE International Conference on Fuzzy Systems and Knowledge Discovery (FSKD). 390–396.Google Scholar
  48. 48.
    Necula, E.. (2014) Dynamic traffic flow prediction based on GPS Data. In: IEEE 26th International Conference on Tools with Artificial Intelligence (ICTAI). 922–929.Google Scholar
  49. 49.
    Shtern, M., Mian, R., Litoiu, M., Zareian, S., Abdelgawad, H., Tizghadam, A.. (2014) Towards a multi-cluster analytical engine for transportation data. In: International Conference on Cloud and Autonomic Computing (ICCAC). 249–257.Google Scholar
  50. 50.
    Ibrahim, H., Far, B.H.. (2014) Data-oriented intelligent transportation systems. In: IEEE 15th International Conference on Information Reuse and Integration (IRI). 322–329.Google Scholar
  51. 51.
    Rashid, S., Akram, U., Qaisar, S., Khan, S.A., Felemban, E.. (2014) Wireless sensor network for distributed event detection based on machine learning. In: IEEE International Conference on Internet of Things (iThings). 540–545.Google Scholar
  52. 52.
    Lin, G., Xin, L., Feng, H., Ying, L.. (2014) A new outlier detection algorithm and its application in intelligent transportation system. In: IEEE 7th Joint International Information Technology and Artificial Intelligence Conference (ITAIC). 442–445.Google Scholar
  53. 53.
    Foell, S., Phithakkitnukoon, S., Kortuem, G., Veloso, M., & Bento, C. (2015). Predictability of public transport usage: a study of bus rides in Lisbon, Portugal. IEEE Transactions on Intelligent Transportation Systems, 16, 2955–2960.CrossRefGoogle Scholar
  54. 54.
    Miyaji, M.. (2015) Data mining for safety transportation by means of using Internet survey. In: 31st IEEE International Conference on Data Engineering Workshops (ICDEW). 119–123.Google Scholar
  55. 55.
    Navale, S. A., & Gurav, Y. B. (2015). Crowdedness Spot Acquisition by Using Mobility Based Clustering. International Journal of Science and Research, 4, 171–174.Google Scholar
  56. 56.
    Lin, Y.X., Wang, P., Ma, M.. (2017) Intelligent Transportation System (ITS): Concept, Challenge and Opportunity. In: IEEE International Conference on High Performance and Smart Computing.Google Scholar
  57. 57.
    Qiu, M., Gao, W., Chen, M., Niu, J.-W., & Zhang, L. (2011). Energy efficient security algorithm for power grid wide area monitoring system. IEEE Transactions on Smart Grid, 2(4), 715–723.CrossRefGoogle Scholar
  58. 58.
    Gai, K., Qiu, M., Ming, Z., Zhao, H., and Qiu, L.. 2017 Spoofing-Jamming Attack Strategy Using Optimal Power Distributions in Wireless Smart Grid Networks. In: IEEE Transactions on Smart Grid.Google Scholar
  59. 59.
    Park, S., Ryu, S., Choi, Y., Kim, H.. (2014) A framework for baseline load estimation in demand response: Data mining approach. In: IEEE International Conference on Smart Grid Communications (SmartGridComm). 638–643.Google Scholar
  60. 60.
    ASGARI, V., Firozyan, M., RADMEHR, M.. (2015) Simultaneous price and demand forecasting in smart power distribution grid. Journal of Selcuk University Natural and Applied Science. 53–62.Google Scholar
  61. 61.
    Popeangă, J., & Lungu, I. (2014). Forecasting Final Energy Consumption using the Centered Moving Average Method and Time Series Analysis. Database Systems Journal, 5, 42–50.Google Scholar
  62. 62.
    Ford, V., Siraj, A., Eberle, W.. (2014) Smart grid energy fraud detection using artificial neural networks. In: IEEE Symposium on Computational Intelligence Applications in Smart Grid (CIASG). 1–6.Google Scholar
  63. 63.
    Tsai, J.C., Yen, N.Y., Hayashi, T.. (2014) Social network based smart grids analysis. In: IEEE International Symposium on Independent Computing (ISIC). 1–6.Google Scholar
  64. 64.
    Ploennigs, J., Chen, B., Palmes, P., Lloyd, R.. (2014) e2-Diagnoser: A System for Monitoring, Forecasting and Diagnosing Energy Usage. In: IEEE International Conference on Data Mining Workshop (ICDMW). 1231–1234.Google Scholar
  65. 65.
    Chen, H., Yang, H., Xu, A., Yuan, C.. (2014) A Decision Support System Using Two-Level Classifier for Smart Grid. In: Ninth International Conference on P2P, Parallel, Grid, Cloud and Internet Computing (3PGCIC). 42–45.Google Scholar
  66. 66.
    Kogo, T., Nakamura, S., Pravinraj, S., Arumugam, B.. (2014) A demand side prediction method for persistent scheduled power-cuts in developing countries. In: IEEE PES Innovative Smart Grid Technologies Conference Europe (ISGT-Europe). 1–6.Google Scholar
  67. 67.
    Gupta, P.K., Gibtner, A.K., Duchon, M., Koss, D., Schätz, B.. (2015) Using knowledge discovery for autonomous decision making in smart grid nodes. In: IEEE International Conference on Industrial Technology (ICIT). 3134–3139.Google Scholar
  68. 68.
    Zhen, Z., Wang, F., Sun, Y., Mi, Z., Liu, C., Wang, B., Lu, J.. (2015) SVM based cloud classification model using total sky images for PV power forecasting. In: IEEE Power & Energy Society Innovative Smart Grid Technologies Conference (ISGT). 1–5.Google Scholar
  69. 69.
    Meng Ma, W.L., Zhang, J., Wang, P., Zhou, Y., Liang, X.. (2017) Discover the Fingerprint of Electrical Appliance: Online Appliance Behavior Learning and Detection in Smart Homes. In: International Conference on Ubiquitous Intelligence and Computing.Google Scholar
  70. 70.
    Makonin, S., Popowich, F., Bartram, L., Gill, B., Bajic, I.V.. (2013) AMPds: A public dataset for load disaggregation and eco-feedback research. In: IEEE Electrical Power & Energy Conference (EPEC). 1–6.Google Scholar
  71. 71.
    Batra, N., Parson, O., Berges, M., Singh, A., Rogers, A.. (2014) A comparison of non-intrusive load monitoring methods for commercial and residential buildings. arXiv preprint arXiv:1408.6595.Google Scholar
  72. 72.
    Monacchi, A., Egarter, D., Elmenreich, W., D'Alessandro, S., Tonello, A.M.. (2014) GREEND: An energy consumption dataset of households in Italy and Austria. In: IEEE International Conference on Smart Grid Communications (SmartGridComm). 511–516.Google Scholar
  73. 73.
    Kolter, J.Z., Johnson, M.J.. (2011) REDD: A public data set for energy disaggregation research. In: Workshop on Data Mining Applications in Sustainability (SIGKDD), San Diego. 59–62.Google Scholar
  74. 74.
    Kleiminger, W., Beckel, C., Santini, S.. (2015) Household occupancy monitoring using electricity meters. In: ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp). 975–986.Google Scholar
  75. 75.
    Batra, N., Gulati, M., Singh, A., Srivastava, M.B.. (2013) It's Different: Insights into home energy consumption in India. In: 5th ACM Workshop on Embedded Systems For Energy-Efficient Buildings. 1–8.Google Scholar
  76. 76.
    Kelly, J., Knottenbelt, W.. (2015) The UK-DALE dataset, domestic appliance-level electricity demand and whole-house demand from five UK homes. Scientific Data. 2.Google Scholar
  77. 77.
    Barker, S., Mishra, A., Irwin, D., Cecchet, E., Shenoy, P., Albrecht, J.. (2012) Smart*: An open data set and tools for enabling research in sustainable homes. SustKDD.Google Scholar
  78. 78.
    Hu, F., Qiu, M., Li, J., Grant, T., Taylor, D., McCaleb, S., Butler, L., & Hamner, R. (2011). A review on cloud computing: Design challenges in architecture and security. Journal of Computing and Information Technology, 19(1), 25–55.CrossRefGoogle Scholar
  79. 79.
    Li, Y., Gai, K., Ming, Z., Zhao, H., and Qiu, M.. (2016) Intercrossed access controls for secure financial services on multimedia big data in cloud systems. In: ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), 12, (4s). 67.Google Scholar
  80. 80.
    Gai, K., Qiu, M., Zhao, H., Tao, L., & Zong, Z. (2016). Dynamic energy-aware cloudlet-based mobile cloud computing model for green computing. Journal of Network and Computer Applications, 59, 46–54.CrossRefGoogle Scholar
  81. 81.
    Zhu, X., Qin, X., & Qiu, M. (2011). QoS-aware fault-tolerant scheduling for real-time tasks on heterogeneous clusters. IEEE transactions on Computers, 60(6), 800–812.MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  • Meng Ma
    • 1
  • Weilan Lin
    • 2
  • Disheng Pan
    • 3
  • Yangxin Lin
    • 2
  • Ping Wang
    • 2
    • 4
    Email author
  • Yuchen Zhou
    • 5
  • Xiaoxing Liang
    • 5
  1. 1.School of Electronics Engineering and Computer SciencePeking University Peking UniversityBeijingChina
  2. 2.School of Software and MicroelectronicsPeking UniversityBeijingChina
  3. 3.School of Electronic and Computer EngineeringPeking University Shenzhen Graduate SchoolShenzhenChina
  4. 4.National Engineering Research Center for Software EngineeringPeking UniversityBeijingChina
  5. 5.IBM Research ChinaBeijingChina

Personalised recommendations