Detection of Motion Patterns and Transition Conditions for Automatic Flow Diagram Generation of Robotic Tasks

  • Guilherme de Campos AffonsoEmail author
  • Kei Okada
  • Masayuki Inaba
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 867)


In this paper a method for detecting motion patterns and transitions between them in source code describing robotics tasks is proposed, being used to automatically generate flow diagrams of the corresponding task. This is done through the combination of both static and dynamic program analysis, first segmenting original code based on token matching and then using runtime data to judge importance of each segment and relations between them. Generation of flow diagrams is not only a simple way of visualizing overall procedure of complex tasks, but can also be used for current state identification and fail recovery systems. Proposed system is verified through experimentation using PR2 robot on the household task of cleaning up a table, being able to automatically generate a flow diagram with reasonable number of states.


Flow diagram Visualization of robotic tasks Dynamic programming analysis 


  1. 1.
    Protsko, L.B., Sorenson, P.G., Tremblay, J.P., Schaefer, D.A.: Towards the automatic generation of software diagrams. IEEE Trans. Softw. Eng. 17(1), 10–21 (1991)CrossRefGoogle Scholar
  2. 2.
    Furuta, Y., Inagaki, Y., Okada, K., Inaba, M.: Self-improving robot action management system with probabilistic graphical model based on task related memories. In: International Conference on Intelligent Autonomous Systems, pp. 811–823. Springer (2016)Google Scholar
  3. 3.
    Bryant, R.E.: Graph-based algorithms for Boolean function manipulation. IEEE Trans. Comput. 100(8), 677–691 (1986)CrossRefGoogle Scholar
  4. 4.
    Minsky, M.L.: Computation: Finite and Infinite Machines. Prentice-Hall Inc., Hoboken (1967)zbMATHGoogle Scholar
  5. 5.
    Böhm, C., Jacopini, G.: Flow diagrams, turing machines and languages with only two formation rules. Commun. ACM 9(5), 366–371 (1966)CrossRefGoogle Scholar
  6. 6.
    Cheng, K.-T., Krishnakumar, A.S.: Automatic functional test generation using the extended finite state machine model. In: 30th Conference on Design Automation, pp. 86–91. IEEE (1993)Google Scholar
  7. 7.
    Burch, J.R., Clarke, E.M., McMillan, K.L., Dill, D.L.: Sequential circuit verification using symbolic model checking. In: Proceedings of the 27th ACM/IEEE Design Automation Conference, pp. 46–51. ACM (1991)Google Scholar
  8. 8.
    Boyer, R.S., Strother Moore, J.: A mechanical proof of the unsolvability of the halting problem. J. ACM (JACM) 31(3), 441–458 (1984)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Landi, W.: Undecidability of static analysis. ACM Lett. Program. Lang. Syst. (LOPLAS) 1(4), 323–337 (1992)CrossRefGoogle Scholar
  10. 10.
    Cheng, K.-T., Krishnakumar, A.S.: Automatic generation of functional vectors using the extended finite state machine model. ACM Trans. Des. Autom. Electron. Syst. (TODAES) 1(1), 57–79 (1996)CrossRefGoogle Scholar
  11. 11.
    Jou, J.-Y., Rothweiler, S., Ernst, R., Sutarwala, S., Prabhu, A.: Bestmap: behavioral synthesis from c. In: Proceedings of the International Workshop on Logic Synthesis (1989)Google Scholar
  12. 12.
    Costagliola, G., Tortora, G., Orefice, S., De Lucia, A.: Automatic generation of visual programming environments. Computer 28(3), 56–66 (1995)CrossRefGoogle Scholar
  13. 13.
    Naebi, A., Khalegi, F., Hosseinpour, F., Zanjanab, A.G., Khoshravan, H., Kelishomi, A.E., Rahmatdoustbeilankouh, B.: A new flowchart and programming technique using bond graph for mechatronic systems. In: 2011 UkSim 13th International Conference on Computer Modelling and Simulation (UKSim), pp. 236–241. IEEE (2011)Google Scholar
  14. 14.
    Gusfield, D.: Algorithms on Strings, Trees and Sequences: Computer Science and Computational Biology. Cambridge University Press, New York (1997)CrossRefGoogle Scholar
  15. 15.
    Open Source Robotics Foundation. smach - ROS Wiki.

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Guilherme de Campos Affonso
    • 1
    Email author
  • Kei Okada
    • 1
  • Masayuki Inaba
    • 1
  1. 1.JSK LaboratoryThe University of TokyoTokyoJapan

Personalised recommendations