Practical Guidelines for Design of Human-in-the-Loop Systems: Lessons Learned

  • Vasily MoshnyagaEmail author
Part of the Studies in Systems, Decision and Control book series (SSDC, volume 140)


Technology has entered the age of smart systems, which not only implement traditional human functions of analyzing real-world data and making decisions but also employ humans within the feedback control loop. The development of such systems is not trivial, however, involving several challenges. In this chapter, we share our experience of building human-in-the-loop systems, such as a user-aware computer display, a viewer-conscious TV, a smart door, a smart carpet, a medication adherence control system and smart in-home system for monitoring people with cognitive impairment. We identify problems related to incorporating humans into the control loop and present guidelines for hardware and software designers.



The author thanks all members of the Computer Systems Lab. of Fukuoka University for their hard work in implementing the smart systems. Without their contribution, this manuscript would not be possible.


  1. 1.
    Worldwide Internet of Things Forecast Update: 2016–2020, IDC, Doc.# US40755516. Accessed May 2016
  2. 2.
    Nunes, D.S., Zhang, P., Silva, J.S.: A survey on human-in-the-loop applications towards an internet of all. IEEE Commun. Surv. Tutor 17(2), 944–965 (2015)CrossRefGoogle Scholar
  3. 3.
    Schirner, G., Erdogmus, D., Chowdhury, K., Padir, T.: The future of human-in-loop cyber-physical systems. Computer 46(1), 36–45 (2013)CrossRefGoogle Scholar
  4. 4.
    Munir, S., Stankovic, J.A., Liang, C.J.M., Lin, S.: New cyber physical system challenges for human-in-the loop control. In: Proceedings of the 8th Internationa Workshop Feedback Computer, June 2013Google Scholar
  5. 5.
    Reger, J.: Internet of Things—2017 trends, IOT Business News, 08 Dec 2016Google Scholar
  6. 6.
    Kawahara, Y., Minami, M., Saruwatari, S.: Challenges and lessons learned in building a practical smart space. In: Annual International Conference on Mobile and Ubiquitous Systems, pp. 213–222 (2004)Google Scholar
  7. 7.
    Linton, R.J., Schafeld, J., Padur, T.: Smart wheelchairs or not: lessons learned from discovery interviews. In: Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 5016–5019 (2015)Google Scholar
  8. 8.
    Bauer, L., et al.: Lessons learned from the deployment of a smart phone-based access-control system. In: SOUPS’07, pp. 64–75 (2007)Google Scholar
  9. 9.
    Bouchard, K., Bouchard, B., Bouzouane, A.: Practical guidelines to build smart homes: lessons learned. In: Opportunistic Networking, Smart Home, Smart City, Smart Systems, pp. 1–37. CRC press (2014)Google Scholar
  10. 10.
    Moshnyaga, V.G., Hashimoto, K., Suetsugu, T.: A camera-driven power management of computer display. IEEE Trans. CAS Video Technol. 22(11), 1542–1553 (2012)CrossRefGoogle Scholar
  11. 11.
    Lee, C., Moshnyaga, V.G.: Embedded system for camera-based TV power reduction. In: Euromicro Conference on DSD, pp. 764–768 (2011)Google Scholar
  12. 12.
    White, P., Sumi, N., Hayashida, A., Hashimoto, K., Moshnyaga, V.: Smart Door, Embedded System Symposium (2015)Google Scholar
  13. 13.
    Tanaka, O., Ryu, T., Hayashida, A., Moshnyaga, V.G., Hashimoto, K.: A smart carpet design for monitoring people with dementia. In: Salvaraj, H. et al. (eds.) Progress in System Engineering, Advances in Intelligent Systems, vol.330, pp. 653–659. Springer (2015)CrossRefGoogle Scholar
  14. 14.
    Moshnyaga, V., Tanaka, O., Ryu, T., et al.: An intelligent system for assisting family caregivers of dementia people. In: 2014 IEEE International Symposium on Series on Computational Intelligence, pp. 1–5 (2014)Google Scholar
  15. 15.
    Moshnyaga, V., Tanaka, O., Ryu, T.: Identification of basic behavioral activities by heterogeneous sensors of in-home monitoring system, in human behavior. In: Salah, A.A., et al. (eds.) Lecture Notes in Computer Science, vol. 9277, pp. 160–174. Springer (2015)CrossRefGoogle Scholar
  16. 16.
    Moshnyaga, V., Koyanagi, M., Hirayama, F., Takahama, A., Hashimoto, K.: A medication adherence monitoring system for people with dementia. In: Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, pp. 194–199 (2016)Google Scholar
  17. 17.
    Imaizumi, K., Moshnyaga, V.: Network-based face recognition on mobile devices. In: IEEE ICCE-Berlin, pp. 406–409 (2013)Google Scholar
  18. 18.
    Taigman, Y., Yang, M., Ranzato, M.A., Wolf, L.: DeepFace: closing the gap to human-level performance in face verification. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1701–1708 (2014)Google Scholar
  19. 19.
    Bombieri, N., Drogoudis, D., Gangemi, G., et al.: Addressing the smart systems design challenge: The SMAC platform, Microprocessors and Microsystems, MICRO 2228, Elsevier B.V., 4 June 2015, pp. 1–17Google Scholar
  20. 20.
    Liu, Y., Feyen, R., Tsimhoni, O.: Queueing network-model human processor (QN-MHP): A computational architecture for multitask performance in human-machine systems. ACM Trans. Comput. Human Interact. 13(1), 37–70 (2006)CrossRefGoogle Scholar
  21. 21.
    Booher, H.R.: Handbook of Human Systems Integration. Wiley, Hoboken (2003)CrossRefGoogle Scholar
  22. 22.
    Behrmann, G., David, A., larsen, K.G.:. A Tutorial on Uppaal. In: Formal Methods for design of real-time systems, vol.3185, Lecture Notes in Computer Science, pp. 200–236 (2004)Google Scholar
  23. 23.
    Cai, L., Gajski, D.: Transaction level modeling: and overview. In: International Conference on HW/SW Codesign & System Synthesis, Oct.2003, pp. 19–24Google Scholar
  24. 24.
    Wood, A., Selvo, L., Stankovic, J.A.: SenQ: an embedded query system for streaming data in heterogeneous interactive wireless sensor networks. In: Lecture Notes in Computer Science, Distributed Computing in Sensor Systems, vol.5067, pp 531–543 (2008)Google Scholar
  25. 25.
    Larsson, P.:. Energy-efficient software guidelines. Intel Software Solutions Group, Technical Report (2011)Google Scholar
  26. 26.
    Steigerwald, B., Chabukswar, R., Krishnan, K., De Vega, J.: Creating Energy—Efficient Software, Intel White Paper (2008)Google Scholar
  27. 27.
    Insup, L., Sokolsky, O.: Medical cyber physical systems. In: Proceedings of the 47th ACM/IEEE on Design Automation Conference (DAC), pp. 743–748 (2010)Google Scholar
  28. 28.
    Gulliksen, J., Goransson, B., Boivie, I., Blomkvist, S., Persson, J., Cajander, A.: Key principles for user-centered systems design. Behav. Inf. Technol. 22(6), 397–409 (2003)CrossRefGoogle Scholar
  29. 29.
    Rasmussen, J., et al.: Cognitive Systems Engineering. Wiley, NYGoogle Scholar
  30. 30.
    Rae, A.: Helping the operator in the loop: practical human machine iinterface principles for safe computer controlled systems. In: the Proceedings of the 12th Australian Workshop on Safety related Programmable Systems (SCS ’07) (2007)Google Scholar
  31. 31.
    Cranor, L.F.: A framework for reasoning about the human in the loop, CMU-Cylab-08–001, 24 Jan 2008Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Electronics Engineering and Computer ScienceFukuoka UniversityJonan-Ku, FukuokaJapan

Personalised recommendations