Skip to main content

Towards On-site Collaborative Robotics

Voice Control, Co-speech Gesture and Context-Specific Object Recognition via Ad-hoc Communication

  • Chapter
  • First Online:
Robotic Fabrication in Architecture, Art and Design 2016


This work presents a novel set of accessible and unified hardware and software solutions that facilitate the implementation of natural human-machine interactions, as required by collaborative robotics in both indoor and outdoor environments. This extensible framework supports vocal control, co-speech gestures, and object recognition with feature tracking and adaptive resolution. The interactions are based on a new network messaging protocol that allows any device using TCP/IP to share variables with the full abstraction of the original machine software platform and can therefore be used synchronously by a vast array of equipment including CNC machines, industrial robots, construction equipment, mobile devices and PLCs. We conclude with the description of a testing scenario to be deployed during the conference workshop.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others


  • Andraos, S 2015, DMR: A Semantic Robot Control Language, Proceedings of the eCAADe Conference 2015, Vienna.

    Google Scholar 

  • Fellbaum, C 1998, WordNet: An Electronic Lexical Database. MIT Press, Cambridge, MA.

    Google Scholar 

  • Schwartz, T, 2012, HAL: Extension of a Visual Programming Language to Support Teaching and Research on Robotics Applied to Construction, Rob|Arch 2012, Springer, Vienna, pp. 92–101.

    Google Scholar 

  • Schwartz, T. 2015. A Generic Communication Library for Human-Robot Interaction on Construction Sites, Design Modelling Symposium 2015, Ed. by Thomsen, M.R. et al., Springer International Publishing, pp 341-351.

    Google Scholar 

  • Song, P, Fu, C W, and Cohen-Or, D 2012, Recursive Interlocking Puzzles, ACM Transactions on Graphics (TOG), vol. 31, no. 6, pp.128.

    Google Scholar 

  • Wahl, FM and Thomas U 2006, Robot Programming from Simple Moves to Complex Tasks, Institute for Robotics and Process Control, Technical University of Braunschweig, Braunschweig.

    Google Scholar 

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to Thibault Schwartz .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Schwartz, T., Andraos, S., Nelson, J., Knapp, C., Arnold, B. (2016). Towards On-site Collaborative Robotics. In: Reinhardt, D., Saunders, R., Burry, J. (eds) Robotic Fabrication in Architecture, Art and Design 2016. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-26376-2

  • Online ISBN: 978-3-319-26378-6

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics