Motion synchronization for semi-autonomous robotic swarm with a passivity-short human operator

  • Made Widhi Surya AtmanEmail author
  • Takeshi Hatanaka
  • Zhihua Qu
  • Nikhil Chopra
  • Junya Yamauchi
  • Masayuki Fujita
Regular Paper


This paper investigates coordination between a human operator and robotic swarm. The objective is to guarantee human-enabled motion synchronization to desired position/velocity references. The presence of a human in the system could improve performance in completing complex missions and adaptation to changes in environment or mission goal. Although in some works the human is modeled or assumed as a passive system, this does not always hold and a systematic solution to deal with non-passive humans is still needed. To this end, this paper assumes the human operator’s process as a passivity-short system. Based on the positive feedback interconnection of passivity-short systems, we present a novel distributed control architecture interconnecting the human operator and the robotic swarm. The control goals are then proved to be achieved even in the presence of passivity shortage in the human operator. We finally demonstrate the proposed architecture through simulation studies and also implementation on an experimental testbed.


Semi-autonomous robotic swarm Distributed motion synchronization Interconnected passivity-short systems 



We acknowledge the financial support from JSPS KAKENHI Grant Nos. 15H04019 and 18H01459.

Supplementary material

Supplementary material 1 (wmv 4,067 KB)


  1. Atashzar, S.F., Shahbazi, M., Tavakoli, M., Patel, R.V.: A passivity-based approach for stable patient–robot interaction in haptics-enabled rehabilitation systems: modulated time-domain passivity control. IEEE Trans. Control Syst. Technol. 25–3, 991–1006 (2017)CrossRefGoogle Scholar
  2. Bai, H., Freeman, R.A., Lynch, K.M.: Robust dynamic average consensus of time-varying inputs. In: 49th IEEE Conf. decision and control, pp. 3104–3109 (2010)Google Scholar
  3. Colgate, J.E.: Coordinate transformations and logical operations for minimizing conservativeness in coupled stability criteria. Trans. ASME J. Dyn. Syst. Meas. Control 116–4, 643–649 (1994)CrossRefzbMATHGoogle Scholar
  4. Cummings, M.L.: Human supervisory control of swarming networks. In: 2nd annual swarming: autonomous intelligent networked system conference, pp. 1–9 (2004)Google Scholar
  5. Dyck, M., Jazayeri, A., Tavakoli, M.: Is the human operator in a teleoperation system passive? In: 2013 world haptics conf., pp. 683–688 (2013)Google Scholar
  6. Egerstedt, M.: Human interactions with complex networks. In: Samad T, Annaswamy AM (eds.) The impact of control technology, 2nd edition, IEEE Control Systems Society. (2014). Accessed 10 Oct 2017
  7. Franchi, A., Secchi, C., Son, H.I., Bulthoff, H.H., Giordano, P.R.: Bilateral teleoperation of groups of mobile robots with time-varying topology. IEEE Trans. Robot. 28–5, 1019–1033 (2012a)CrossRefGoogle Scholar
  8. Franchi, A., Secchi, C., Ryll, M., Bulthoff, H.H., Giordano, P.R.: Shared control: balancing autonomy and human assistance with a group of quadrotor UAVs. IEEE Robot. Autom. Mag. 19–3, 57–68 (2012b)CrossRefGoogle Scholar
  9. Freeman, R.A., Yang, P., Lynch, K.M.: Stability and convergence properties of dynamic average consensus estimators. In: 45th IEEE conf. decision and control, pp. 338–343 (2006)Google Scholar
  10. Giordano, P.R., Franchi, A., Secchi, C., Bulthoff, H.H.: A passivity-based decentralized strategy for generalized connectivity maintenance. Int. J. Robot. Res. 32–3, 299–323 (2013)CrossRefGoogle Scholar
  11. Hatanaka, T., Chopra, N., Fujita, M.: Passivity-based bilateral human-swarm-interactions for cooperative robotic networks and human passivity analysis. In: 54th IEEE Conf. decision and control, pp. 1033–1039 (2015a)Google Scholar
  12. Hatanaka, T., Chopra, N., Fujita, M., Spong, M.W.: Passivity-based Control and Estimation in Networked Robotics. Springer, Switzerland (2015b)CrossRefzbMATHGoogle Scholar
  13. Hatanaka, T., Chopra, N., Yamauchi, J., Doi, M., Kawai, Y., Fujita, M.: A passivity-based system design of semi-autonomous cooperative robotic swarm. ASME DSC Mag. 139–6, 14–18 (2017a)Google Scholar
  14. Hatanaka, T., Chopra, N., Yamauchi, J., Fujita, M.: A passivity-based approach to human-swarm collaborations and passivity analysis of human operators. In: Wang, Y., Zhang, F. (eds.) Trends in Control and Decision-Making for Human–Robot Collaboration Systems, pp. 325–355. Springer, Switzerland (2017b)CrossRefGoogle Scholar
  15. Hirche, S., Buss, M.: Human-oriented control for haptic teleoperation. Proc. IEEE 100–3, 623–647 (2012)CrossRefGoogle Scholar
  16. Hokayem, P.F., Spong, M.W.: Bilateral teleoperation: an historical survey. Automatica 42–12, 2035–2057 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  17. Kolling, A., Walker, P., Chakraborty, N., Sycara, K., Lewis, M.: Human interaction with robot swarms: a survey. IEEE Trans. Hum. Mach. Syst. 46–1, 9–26 (2016)CrossRefGoogle Scholar
  18. Lee, D., Spong, M.W.: Bilateral teleoperation of multiple cooperative robots over delayed communication networks: theory. In: Proc. IEEE Int. Conf. Robotics and Automation, pp. 360–365 (2005)Google Scholar
  19. Lee, D., Franchi, A., Son, H.I., Ha, C., Bulthoff, H.H., Giordano, P.R.: Semiautonomous haptic teleoperation control architecture of multiple unmanned aerial vehicles. IEEE/ASME Trans. Mechatron. 18–4, 1334–1345 (2013)CrossRefGoogle Scholar
  20. Liu, Y.C., Chopra, N.: Controlled synchronization of heterogeneous robotic manipulators in the task space. IEEE Trans. Robot. 28–1, 268–275 (2012)CrossRefGoogle Scholar
  21. McLurkin, J., Smith, J., Frankel, J., Sotkowitz, D., Blau, D., Schmidt, B.: Speaking swarmish: Human-robot interface design for large swarms of autonomous mobile robots. In: AAAI spring symposium, pp. 72–76 (2006)Google Scholar
  22. McRuer, D.: Human dynamics in man-machine systems. Automatica 16–3, 237–253 (1980)CrossRefzbMATHGoogle Scholar
  23. Mekdeci, B., Cummings, M.L.: Modeling multiple human operators in the supervisory control of heterogeneous unmanned vehicles. In: Proc. 9th workshop on performance metrics for intelligent systems, pp. 1–8 (2009)Google Scholar
  24. Music, S., Hirche, S.: Control sharing in human–robot team interaction. Annu. Rev. Control 44, 342–354 (2017)CrossRefGoogle Scholar
  25. Nuno, E., Basanez, L., Ortega, R.: Passivity-based control for bilateral teleoperation: a tutorial. Automatica 47, 485–495 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  26. Olsen, D.R., Wood, B.: Fan-out: Measuring human control of multiple robots. In: Proc. SIGHCI Conf. human factors in computing systems, pp. 231–238 (2004)Google Scholar
  27. Qu, Z.: An impact equivalence principle of separating control designs for networked heterogeneous affine systems. IFAC Proc. Vol. 45–26, 210–215 (2012)CrossRefGoogle Scholar
  28. Qu, Z., Simaan, M.A.: Modularized design for cooperative control and plug-and-play operation of networked heterogeneous systems. Automatica 50–9, 2405–2414 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  29. Rodriguez-Seda, E.J., Troy, J.J., Erignac, C.A., Murray, P., Stipanovic, D.M., Spong, M.W.: Bilateral teleoperation of multiple mobile agents: coordinated motion and collision avoidance. IEEE Trans. Control Syst. Technol. 18–4, 984–992 (2010)CrossRefGoogle Scholar
  30. Secchi, C., Franchi, A., Bulthoff, H.H., Giordano, P.R.: Bilateral teleoperation of a group of UAVs with communication delays and switching topology. In: Proc. IEEE Int. conf. robotics and automation, pp. 4307–4314 (2012)Google Scholar
  31. Stipanovic, D.M., Hokayem, P.F., Spong, M.W., Siljak, D.D.: Cooperative avoidance control for multiagent systems. Trans. ASME J. Dyn. Syst. Meas. Control 129–5, 699–707 (2007)CrossRefGoogle Scholar
  32. Wang, X., Wang, Y.: Co-design of control and scheduling for human-swarm collaboration systems based on mutual trust. In: Wang, Y., Zhang, F. (eds.) Trends in Control and Decision-Making for Human-Robot Collaboration Systems, pp. 387–413. Springer, Switzerland (2017)CrossRefGoogle Scholar
  33. Xia, M., Rahnama, A., Wang, S., Antsaklis, P.J.: On guaranteeing passivity and performance with a human controller. In: 2015 23th Mediterranean conf. control and automation, pp. 722–727 (2015)Google Scholar
  34. Yun X., Yamamoto, Y.: Internal dynamics of a wheeled mobile robot. In: Proc. 1993 IEEE/RSJ Int. conf. intelligent robots and systems, pp. 1288–1294 (1993)Google Scholar
  35. Wang, Y., Zhang, F. (eds.): Trends in Control and Decision-Making for Human–Robot Collaboration Systems. Springer, Switzerland (2017)zbMATHGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  1. 1.Department of Systems and Control Engineering, School of EngineeringTokyo Institute of TechnologyTokyoJapan
  2. 2.Division of Electrical, Electronic and Information Engineering, Graduate School of EngineeringOsaka UniversityOsakaJapan
  3. 3.Department of Electrical Engineering and Computer ScienceUniversity of Central FloridaOrlandoUSA
  4. 4.Department of Mechanical EngineeringUniversity of MarylandCollege ParkUSA

Personalised recommendations