Skip to main content

Towards On-site Collaborative Robotics

Voice Control, Co-speech Gesture and Context-Specific Object Recognition via Ad-hoc Communication


This work presents a novel set of accessible and unified hardware and software solutions that facilitate the implementation of natural human-machine interactions, as required by collaborative robotics in both indoor and outdoor environments. This extensible framework supports vocal control, co-speech gestures, and object recognition with feature tracking and adaptive resolution. The interactions are based on a new network messaging protocol that allows any device using TCP/IP to share variables with the full abstraction of the original machine software platform and can therefore be used synchronously by a vast array of equipment including CNC machines, industrial robots, construction equipment, mobile devices and PLCs. We conclude with the description of a testing scenario to be deployed during the conference workshop.


  • On-site worker assistance
  • Collaborative robotics
  • Multimodal communication
  • Advanced optical sensors

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-26378-6_31
  • Chapter length: 10 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
USD   139.00
Price excludes VAT (USA)
  • ISBN: 978-3-319-26378-6
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   179.99
Price excludes VAT (USA)
Hardcover Book
USD   179.99
Price excludes VAT (USA)
Fig. 1
Fig. 2
Fig. 3
Fig. 4


  • Andraos, S 2015, DMR: A Semantic Robot Control Language, Proceedings of the eCAADe Conference 2015, Vienna.

    Google Scholar 

  • Fellbaum, C 1998, WordNet: An Electronic Lexical Database. MIT Press, Cambridge, MA.

    Google Scholar 

  • Schwartz, T, 2012, HAL: Extension of a Visual Programming Language to Support Teaching and Research on Robotics Applied to Construction, Rob|Arch 2012, Springer, Vienna, pp. 92–101.

    Google Scholar 

  • Schwartz, T. 2015. A Generic Communication Library for Human-Robot Interaction on Construction Sites, Design Modelling Symposium 2015, Ed. by Thomsen, M.R. et al., Springer International Publishing, pp 341-351.

    Google Scholar 

  • Song, P, Fu, C W, and Cohen-Or, D 2012, Recursive Interlocking Puzzles, ACM Transactions on Graphics (TOG), vol. 31, no. 6, pp.128.

    Google Scholar 

  • Wahl, FM and Thomas U 2006, Robot Programming from Simple Moves to Complex Tasks, Institute for Robotics and Process Control, Technical University of Braunschweig, Braunschweig.

    Google Scholar 

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to Thibault Schwartz .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Schwartz, T., Andraos, S., Nelson, J., Knapp, C., Arnold, B. (2016). Towards On-site Collaborative Robotics. In: Reinhardt, D., Saunders, R., Burry, J. (eds) Robotic Fabrication in Architecture, Art and Design 2016. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-26376-2

  • Online ISBN: 978-3-319-26378-6

  • eBook Packages: EngineeringEngineering (R0)