Journal of Intelligent & Robotic Systems

, Volume 94, Issue 2, pp 371–387 | Cite as

CS::APEX: A Framework for Algorithm Prototyping and Experimentation with Robotic Systems

Modeling Perception and High Level Robot Control with Activity Flow Graphs
  • Sebastian BuckEmail author
  • Andreas Zell


Robotic systems differ drastically in their sensory capabilities, their computational power and their designated tasks. For efficient algorithm development, however, we need to have a common modeling framework that enables us to generalize and re-use existing solutions. A modular approach, which is coherent across different platforms, also allows faster prototyping of new systems, given that existing functionality can be reused from already implemented modules. In this paper we develop a modeling framework based on data flow graphs that achieves the following goal: We first merge synchronous data flow and reactive programming into hybrid flow graphs, where we explicitly model synchronous and asynchronous data flow. Then we transfer concepts from finite-state machines to achieve a coherent framework which we call Activity Flow Graphs. The flow of activity enables us to model high level states directly in the data flow graph. The result is a single computation graph that can express both perception and high level control aspects of any robotic system. This theoretical foundation is the core of our open-source software framework CS::APEX, which allows the creation, manipulation and evaluation of Activity Flow Graphs and enables rapid prototyping and experimentation and can be used with any robot supporting the Robot Operating System (ROS). We then demonstrate the framework with two high level models for a fetch-and-delivery robot and a person following robot.


Data flow programming Activity flow graphs Framework Rapid prototyping Robotics Mobile robotics 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.



  1. 1.
    Berthold, M.R., Cebron, N., Dill, F., Gabriel, T.R., Kötter, T., Meinl, T., Ohl, P., Sieb, C., Thiel, K., Wiswedel, B.: KNIME: the Konstanz information miner. In: Studies in Classification, Data Analysis, and Knowledge Organization (GfKL 2007). Springer, Berlin (2007)Google Scholar
  2. 2.
    Biggs, G., Ando, N., Kotoku, T.: Rapid data processing pipeline development using openrtm-aist. In: 2011 IEEE/SICE International Symposium on System Integration (SII), pp 312–317 (2011),
  3. 3.
    Bitter, I., Van Uitert, R., Wolf, I., Ibanez, L., Kuhnigk, J.M.: Comparison of four freely available frameworks for image processing and visualization that use itk. IEEE Trans. Vis. Comput. Graph. 13(3), 483–493 (2007)CrossRefGoogle Scholar
  4. 4.
    Brunner, S.G., Steinmetz, F., Belder, R., Dömel, A.: Rafcon: a graphical tool for engineering complex, robotic tasks. In: 2016 IEEE International Conference on Intelligent Robots and Systems(IROS). IEEE (2016)Google Scholar
  5. 5.
    Buck, S., Hanten, R., Huskić, G., Rauscher, G., Kloss, A., Leininger, J., Ruff, E., Widmaier, F., Zell, A.: Conclusions from an object-delivery robotic competition: Sick robot day 2014. In: The 17th International Conference on Advanced Robotics (ICAR). Istanbul, pp 137–143 (2015).
  6. 6.
    Buck, S., Hanten, R., Pech, C.R., Zell, A.: Synchronous dataflow and visual programming for prototyping robotic algorithms. In: The 14th International Conference on Intelligent Autonomous Systems (IAS), pp 911–923, Shanghai (2016),
  7. 7.
    Buck, S., Hanten, R., Bohlmann, K., Zell, A.: Generic 3d obstacle detection for agvs using time-of-flight cameras. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) pp 4119–4124. Daejeon (2016).,
  8. 8.
    Buck, S., Hanten, R., Bohlmann, K., Zell, A.: Multi-sensor payload detection and acquisition for truck-trailer agvs. In: 017 IEEE International Conference on Robotics and Automation (ICRA), 2. Singapore (2017)Google Scholar
  9. 9.
    Demšar, J., Curk, T., Erjavec, A., Gorup, Č., Hočevar, T., Milutinovič, M., Možina, M., Polajnar, M., Toplak, M., Starič, A., Štajdohar, M., Umek, L., žagar, L., žbontar, J., žitnik, M., Zupan, B.: Orange: data mining toolbox in python. J. Mach. Learn. Res. 14, 2349–2353 (2013)zbMATHGoogle Scholar
  10. 10.
    Dumas, M., Ter Hofstede, A.H.: Uml activity diagrams as a workflow specification language. In: International Conference on the Unified Modeling Language, pp 76–90. Springer, Berlin (2001)Google Scholar
  11. 11.
    Eker, J., Janneck, J.W., Lee, E.A., Liu, J., Liu, X., Ludvig, J., Neuendorffer, S., Sachs, S., Xiong, Y.: Taming heterogeneity-the ptolemy approach. Proc. IEEE 91(1), 127–144 (2003)CrossRefGoogle Scholar
  12. 12.
    Ethan Rublee, V.R., et al.: Ecto - A C + + /Python computation graph framework. (2015)
  13. 13.
    Halbwachs, N., Caspi, P., Raymond, P., Pilaud, D.: The synchronous data flow programming language lustre. Proc. IEEE 79(9), 1305–1320 (1991)CrossRefGoogle Scholar
  14. 14.
    Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The weka data mining software: an update. SIGKDD Explor Newsl 11(1), 10–18 (2009). CrossRefGoogle Scholar
  15. 15.
    Harel, D., Lachover, H., Naamad, A., Pnueli, A., Politi, M., Sherman, R., Shtull-Trauring, A., Trakhtenbrot, M.: Statemate: a working environment for the development of complex reactive systems. IEEE Trans. Softw. Eng. 16(4), 403–414 (1990)CrossRefGoogle Scholar
  16. 16.
    Hart, S., Dinh, P., Yamokoski, J., Wightman, B., Radford, N.: Robot task commander: a framework and ide for robot application development. In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2014), pp 1547–1554 (2014),
  17. 17.
    Huskić, G., Buck, S., Ibargüen González, L.A., Zell, A.: Outdoor person following at higher speeds using a skid-steered mobile robot. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Vancouver. (accepted for publication) (2017)Google Scholar
  18. 18.
    Huskić, G., Buck, S., Zell, A.: Path following control of skid-steered wheeled mobile robots at higher speeds on different terrain types. In: IEEE International Conference on Robotics and Automation (ICRA). Singapore (2017)Google Scholar
  19. 19.
    Lee, E., Messerschmitt, D.G., et al.: Synchronous data flow. Proc. IEEE 75(9), 1235–1245 (1987)CrossRefGoogle Scholar
  20. 20.
    Liu, L., Pu, C.: Activity flow: towards incremental specification and flexible coordination of workflow activities. In: International Conference on Conceptual Modeling, pp 169–182. Springer (1997)Google Scholar
  21. 21.
    Ludäscher, B., Altintas, I., Berkley, C., Higgins, D., Jaeger, E., Jones, M., Lee, E.A., Tao, J., Zhao, Y.: Scientific workflow management and the kepler system. Concurr. Comput.: Pract. Exp. 18(10), 1039–1065 (2006)CrossRefGoogle Scholar
  22. 22.
    Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Wheeler, R., Ng, A.Y.: Ros: an open-source robot operating system. In: ICRA Workshop on Open Source Software, vol. 3, p 5 (2009)Google Scholar
  23. 23.
    Sadiq, S., Orlowska, M., Sadiq, W., Foulger, C.: Data flow and validation in workflow modelling. In: Proceedings of the 15th Australasian Database Conference, vol. 27, pp 207–214. Australian Computer Society, Inc (2004)Google Scholar
  24. 24.
    Trcka, N., van der Aalst, W., Sidorova, N.: Analyzing control-flow and data-flow in workflow processes in a unified way. Computer science report (08-31) (2008)Google Scholar
  25. 25.
    von Hanxleden, R., Duderstadt, B., Motika, C., Smyth, S., Mendler, M., Aguado, J., Mercer, S., O’Brien, O.: Sccharts: sequentially constructive statecharts for safety-critical applications. In: ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), pp 372–383 (2014)Google Scholar

Copyright information

© Springer Science+Business Media B.V., part of Springer Nature 2018

Authors and Affiliations

  1. 1.University of TübingenTübingenGermany

Personalised recommendations