Mobile Augmented Reality Based Annotation System: A Cyber-Physical Human System

  • Constantin Scheuermann
  • Felix Meissgeier
  • Bernd Bruegge
  • Stephan Verclas
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9768)


One goal of the Industry 4.0 initiative is to improve knowledge sharing among and within production sites. A fast and easy knowledge exchange can help to reduce costly down-times in factory environments. In the domain of automotive manufacturing, production line down-times cost in average about $1.3 million per hour. Saving seconds or minutes have a real business impact and the reduction of such down-time costs is of major interest.

In this paper we describe MARBAS, a Mobile Augmented Reality based Annotation System, which supports production line experts during their maintenance tasks. We developed MARBAS as Cyber-Physical Human System that enables experts to annotate a virtual representation of a real world scene. MARBAS uses a mobile depth sensor that can be attached to smart phones or tablets in combination with Instant Tracking. Experts can share information using our proposed system. We believe that such an annotation system can excel current maintenance processes by accelerating them.

To identify applicable mesh registration algorithms we conducted a practical simulation. We used a 6 axis joint-arm robot to evaluate 7 different ICP algorithms concerning time and accuracy. Our results show that PCL non-linear ICP offers best performance for our scenario. Additionally, we developed a vertical prototype using a mobile depth sensor in combination with a tablet. We could show the feasibility of our approach augmenting real world scenes with virtual information.


Augmented reality Maintenance Cyber-Physical Human System Annotation system 



The authors would like to thank Fortiss and especially Markus Rickert for providing the 6-axis joint robot arm used for the ICP algorithm evaluation.


  1. 1.
    Basu, M.: Gaussian-based edge-detection methods - a survey. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 32(3), 252–260 (2002)CrossRefGoogle Scholar
  2. 2.
    Besl, P.J., McKay, N.D.: A method for registration of 3-d shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14, 239–256 (1992)CrossRefGoogle Scholar
  3. 3.
    Bleser, G., Wuest, H., Stricker, D.: Online camera pose estimation in partially known and dynamic scenes. In: ISMAR, pp. 56–65. IEEE Computer Society (2006)Google Scholar
  4. 4.
    Davison, A.J., Reid, I.D., Molton, N.D., Stasse, O.: Monoslam: Real-time single camera slam. IEEE Trans. Pattern Anal. Mach. Intell. 29, 1052–1067 (2007)CrossRefGoogle Scholar
  5. 5.
    Drummond, T., Society, I.C., Cipolla, R.: Real-time visual tracking of complex structures. IEEE Trans. Pattern Anal. Mach. Intell. 24, 932–946 (2002)CrossRefGoogle Scholar
  6. 6.
    Durrant-Whyte, H., Bailey, T.: Simultaneous localisation and mapping (slam): Part i the essential algorithms. IEEE Robot. Autom. Mag. 13(2), 99–110 (2006)CrossRefGoogle Scholar
  7. 7.
    Endres, F., Hess, J., Engelhard, N., Sturm, J., Cremers, D., Burgard, W.: An evaluation of the rgb-d slam system, St. Paul, MA, USA (2012)Google Scholar
  8. 8.
    Jackson, T., Angermann, F., Meier, P.: Survey of use cases for mobile augmented reality browsers. In: Furht, B. (ed.) Handbook of Augmented Reality, pp. 409–431. Springer, New York (2011)CrossRefGoogle Scholar
  9. 9.
    Klopschitz, M., Schall, G., Schmalstieg, D., Reitmayr, G.: Visual tracking for augmented reality. In: 2010 International Conference on Indoor Positioning and Indoor Navigation (IPIN), pp. 1–4 (2010)Google Scholar
  10. 10.
    Lima, J., Simões, F., Figueiredo, L., Teichrieb, V., Kelner, J.: Model based markerless 3D tracking applied to augmented reality. J. 3D Interact. Syst. 1, 1–15 (2010)Google Scholar
  11. 11.
    Platonov, J., Heibel, H., Meier, P., Grollmann, B.: A mobile markerless ar system for maintenance and repair. In: IEEE/ACM International Symposium on Mixed and Augmented Reality 2006, pp. 105–108 (2006)Google Scholar
  12. 12.
    Reitmayr, G., Schmalstieg, D.: Location based applications for mobile augmented reality. In: Proceedings of the 4th Australasian User Interface Conference, pp. 65–73. Australian Computer Society (2003)Google Scholar
  13. 13.
    Rusinkiewicz, S., Levoy, M.: Efficient variants of the ICP algorithm. In: Third International Conference on 3D Digital Imaging and Modeling (3DIM), June 2001Google Scholar
  14. 14.
    Schall, G., Wagner, D., Reitmayr, G., Taichmann, E., Wieser, M., Schmalstieg, D., Hofmann-Wellenhof, B.: Global pose estimation using multi-sensor fusion for outdoor augmented reality. In: ISMAR, pp. 153–162. IEEE Computer Society (2009)Google Scholar
  15. 15.
    Scheuermann, C., Bruegge, B., Folmer, J., Verclas, S.: Incident localizationand assistance system: A case study of a cyber-physical human system. In: 2015 IEEE/CIC International Conference on Communications in China: 3rd IEEE ICCC International Workshop on Internet of Things (2015 ICCC IoT Workshop), Shenzhen, P.R. China, November 2015Google Scholar
  16. 16.
    Vadala, E., Graham, C.: Downtime costs auto industry $22k/minute - survey (2006).
  17. 17.
    You, S., Neumann, U.: Fusion of vision and gyro tracking for robust augmented reality registration. In: VR, pp. 71–78. IEEE Computer Society (2001)Google Scholar
  18. 18.
    Zhou, F., Duh, H., Billinghurst, M.: Trends in augmented reality tracking, interaction, display: A review of ten years of ISMAR. In: 7th IEEE/ACM International Symposium on Mixed and Augmented Reality (ISMAR). IEEE Computer Society (2008)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Constantin Scheuermann
    • 1
  • Felix Meissgeier
    • 1
  • Bernd Bruegge
    • 1
  • Stephan Verclas
    • 2
  1. 1.Department of Computer ScienceTechnical University MunichMunichGermany
  2. 2.T-Systems International GmbHFrankfurtGermany

Personalised recommendations