Dual-Arm Construction Robot with Remote-Control Function

Part of the Springer Tracts in Advanced Robotics book series (STAR, volume 128)


In disaster areas, operating heavy construction equipment remotely and autonomously is necessary, but conventional remote-controlled heavy equipment has problems such as insufficient operability, limited mobility on slopes and stairs, and low work efficiency because of difficult remote control. As part of the ImPACT-TRC Program, a group of Japanese researchers attempts to solve these problems by developing a construction robot for disaster relief tasks with a new mechanism and new control methods. This chapter presents the overview of construction robot and the details of main elemental technologies making up the robot. Section 5.1 describes the basic configuration of the robot and the teleoperation system. Section 5.2 is a tether powered drone which provides extra visual information. Sections 5.4 and 5.3 are force and tactile feedback for skillful teleoperation. Section 5.5 is visual information feedback which consists of an arbitrary viewpoint visualization system and a visible and LWIR camera system to observe surrounding of the robot in a dark night scene and/or a very foggy scene. These functions can dramatically increase construction equipment’s capacity to deal with large-scale disasters and accidents.


  1. 1.
    Araki, R., Okada, T., Tazaki, Y., Yokokohji, Y., Yoshinada, H., Nakamura, S., Kurashiki, K.: External force estimation of a hydraulically-driven robot in the disaster area for high fidelity teleoperation. In: 2018 JSME Conference on Robotics and Mechatronics (ROBOMECH 2018), 2A1-J05 (2018). (In Japanese)Google Scholar
  2. 2.
    Bensmaïa, S., Hollins, M.: Pacinian representations of fine surface texture. Percept. Psychophys. 67(5), 842–854 (2005)CrossRefGoogle Scholar
  3. 3.
    Bolanowski Jr., S.J., Gescheider, G.A., Verrillo, R.T., Checkosky, C.M.: Four channels mediate the mechanical aspects of touch. J. Acoust. Soc. Am. 84(5), 1680–1694 (1988)CrossRefGoogle Scholar
  4. 4.
    Bry, A., Bachrach, A., Roy, N.: State estimation for aggressive flight in GPS-denied environments using onboard sensing. In: 2012 IEEE International Conference on Robotics and Automation, pp. 1–8 (2012)Google Scholar
  5. 5.
    Chayama, K., Fujioka, A., Kawashima, K., Yamamoto, H., Nitta, Y., Ueki, C., Yamashita, A., Asama, H.: Technology of unmanned construction system in Japan. J. Robot. Mechatron. 26(4), 403–417 (2014)CrossRefGoogle Scholar
  6. 6.
    Choi, S.Y., Choi, B.H., Jeong, S.Y., Gu, B.W., Yoo, S.J., Rim, C.T.: Tethered aerial robots using contactless power systems for extended mission time and range. In: 2014 IEEE Energy Conversion Congress and Exposition (ECCE), pp. 912–916 (2014)Google Scholar
  7. 7.
    Culbertson, H., Unwin, J., Kuchenbecker, K.J.: Modeling and rendering realistic textures from unconstrained tool-surface interactions. IEEE Trans. Haptics 7(3), 381–393 (2014)CrossRefGoogle Scholar
  8. 8.
    CyPhy: The future of high-powered commercial drones. Accessed 8 Aug 2018
  9. 9.
    del Sol, E., et al.: External force estimation for teleoperation based on proprioceptive sensors. Int. J. Adv. Robot. Syst. 11(52) (2014)Google Scholar
  10. 10.
    DJI: Phantom4. Accessed 8 Aug 2018
  11. 11.
    Egawa, E., Kawamura, K., Ikuta, M., Eguchi, T.: Use of construction machinery in earthquake recovery work. Hitachi Rev. 62(2), 136–141 (2013)Google Scholar
  12. 12.
    He, S., Ye, J., Li, Z., Li, S., Wu, G., Wu, H.: A momentum-based collision detection algorithm for industrial robots. In: 2015 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 1253–1259 (2015)Google Scholar
  13. 13.
    Heuer, H., Owens, A.: Vertical gaze direction and the resting posture of the eyes. Perception 18(3), 363–377 (1989)CrossRefGoogle Scholar
  14. 14.
    Higashi, K., Okamoto, S., Yamada, Y.: What is the hardness perceived by tapping? In: International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, pp. 3–12. Springer, Berlin (2016)CrossRefGoogle Scholar
  15. 15.
    Hirabayashi, T., et al.: Teleoperation of construction machines with haptic information for underwater application. Autom. Constr. 15(5), 563–570 (2006)CrossRefGoogle Scholar
  16. 16.
    Hughes, C., Denny, P., Glavin, M., Jones, E.: Equidistant fish-eye calibration and rectification by vanishing point extraction. IEEE Trans. Pattern Anal. Mach. Intell. 32(12), 2289–2296 (2010)CrossRefGoogle Scholar
  17. 17.
    Ide, T., Nabae, H., Hirota, Y., Yamamoto, A., Suzumori, K.: Preliminary test results of hydraulic tough multi finger robot hand. In: 2018 JSME Conference on Robotics and Mechatronics (ROBOMECH 2018), 2P1-L01 (2018). (In Japanese)Google Scholar
  18. 18.
    Iwataki, S., Fujii, H., Moro, A., Yamashita, A., Asama, H., Yoshinada, H.: Visualization of the surrounding environment and operational part in a 3DCG model for the teleoperation of construction machines. In: 2015 IEEE/SICE International Symposium on System Integration (SII), pp. 81–87 (2015)Google Scholar
  19. 19.
    James, C.A., Bednarz, T.P., Haustein, K., Alem, L., Caris, C., Castleden, A.: Tele-operation of a mobile mining robot using a panoramic display: an exploration of operators sense of presence. In: 2011 IEEE International Conference on Automation Science and Engineering, pp. 279–284 (2011)Google Scholar
  20. 20.
    Kiribayashi, S., Yakushigawa, K., Nagatani, K.: Design and development of tether-powered multirotor micro unmanned aerial vehicle system for remote-controlled construction machine. In: Preprints of the 11th International Conference on Field and Service Robotics, p. # 24 (2017)Google Scholar
  21. 21.
    Kiribayashi, S., Yakushigawa, K., Nagatani, K.: Design and development of tether-powered multirotor micro unmanned aerial vehicle system for remote-controlled construction machine. In: Proceedings of Field and Service Robotics, pp. 637–648 (2018)Google Scholar
  22. 22.
    Kiribayashi, S., Yakushigawa, K., Nagatani, K.: Position estimation of tethered micro unmanned aerial vehicle by observing the slack tether. In: 2017 IEEE International Symposium on Safety, Security and Rescue Robotics (SSRR), pp. 159–165 (2017)Google Scholar
  23. 23.
    Komatsu, R., Fujii, H., Kono, H., Tamura, Y., Yamashita, A., Asama, H.: Bird’s-eye view image generation with camera malfunction in irradiation environment. In: 6th International Conference on Advanced Mechatronics (ICAM 2015), pp. 177–178 (2015)CrossRefGoogle Scholar
  24. 24.
    Kondo, D., Nakamura, S., Kurashiki, K., Yoshinada, H.: Immersive display for remote control of construction robot. In: Proceedings of the 62nd Annual Conference of System, Control and Information Engineers (ISCIE), pp. 131–135 (2018). (In Japanese)Google Scholar
  25. 25.
    Kontz, M.E., et al.: Pressure based exogenous force estimation. In: 2006 ASME International Mechanical Engineering Congress and Exposition, pp. 111–120 (2006)Google Scholar
  26. 26.
    Kuchenbecker, K.J., Fiene, J., Niemeyer, G.: Improving contact realism through event-based haptic feedback. IEEE Trans. Vis. Comput. Graph. 12(2), 219–230 (2006)CrossRefGoogle Scholar
  27. 27.
    Lamore, P., Muijser, H., Keemink, C.: Envelope detection of amplitude-modulated high-frequency sinusoidal signals by skin mechanoreceptors. J. Acoust. Soc. Am. 79(4), 1082–1085 (1986)CrossRefGoogle Scholar
  28. 28.
    Luh, J., Zheng, Y.F.: Computation of input generalized forces for robots with closed kinematic chain mechanisms. IEEE J. Robot. Autom. 1(2), 95–103 (1985)CrossRefGoogle Scholar
  29. 29.
    Makino, Y., Maeno, T., Shinoda, H.: Perceptual characteristic of multi-spectral vibrations beyond the human perceivable frequency range. In: 2011 IEEE World Haptics Conference (WHC), pp. 439–443. IEEE (2011)Google Scholar
  30. 30.
    McMahan, W., Kuchenbecker, K.J.: Spectral subtraction of robot motion noise for improved event detection in tactile acceleration signals. In: International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, pp. 326–337. Springer (2012)Google Scholar
  31. 31.
    McMahan, W., Gewirtz, J., Standish, D., Martin, P., Kunkel, J.A., Lilavois, M., Wedmid, A., Lee, D.I., Kuchenbecker, K.J.: Tool contact acceleration feedback for telerobotic surgery. IEEE Trans. Haptics 4(3), 210–220 (2011)CrossRefGoogle Scholar
  32. 32.
    Minamoto, M., Nakayama, K., Aokage, H., Sako, S.: Development of a Tele-earthwork system. Autom. Robot. Constr. XI 269–275 (1994)Google Scholar
  33. 33.
    Moteki, M., Nishiyama, A., Yuta, S., Ando, H., Ito, S., Fujino, K.: Work efficiency evaluation on the various remote control of the unmanned construction. In: 15th Symposium on Construction Robotics in Japan, pp. O–21 (2015). (In Japanese)Google Scholar
  34. 34.
    Nagano, H., Takenouchi, H., Konyo, M., Tadokoro, S.: Haptic transmission using high-frequency vibration generated on body of construction robot -performance evaluation of haptic transmission system under construction robot teleoperation-. In: 2018 JSME Conference on Robotics and Mechatronics (ROBOMECH 2018), 2A1-J04 (2018). (In Japanese)CrossRefGoogle Scholar
  35. 35.
    Ogino, Y., Shibata, T., Tanaka, M., Okutomi, M.: Coaxial visible and FIR camera system with accurate geometric calibration. In: SPIE Defense + Commercial Sensing (DCS 2017), vol. 10214, pp. 1021415–1–6 (2017)Google Scholar
  36. 36.
    Okamoto, S., Yamada, Y.: An objective index that substitutes for subjective quality of vibrotactile material-like textures. In: 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3060–3067. IEEE (2011)Google Scholar
  37. 37.
    Okamura, A.M., Dennerlein, J.T., Howe, R.D.: Vibration feedback models for virtual environments. In: Proceedings, 1998 IEEE International Conference on Robotics and Automation, vol. 1, pp. 674–679. IEEE (1998)Google Scholar
  38. 38.
    Okura, F., Ueda, Y., Sato, T., Yokoya, N.: Teleoperation of mobile robots by generating augmented free-viewpoint images. In: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 665–671 (2013)Google Scholar
  39. 39.
    Padgaonkar, A.J., et al.: Measurement of angular acceleration of a rigid body using linear accelerometers. ASME J. Appl. Mech. 42(3), 552–556 (1975)CrossRefGoogle Scholar
  40. 40.
    Papachristos, C., Tzes, A.: The power-tethered UAV-UGV team: a collaborative strategy for navigation in partially-mapped environments. In: 22nd Mediterranean Conference on Control and Automation, pp. 1153–1158 (2014)Google Scholar
  41. 41.
    Sato, T., Moro, A., Sugahara, A., Tasaki, T., Yamashita, A., Asama, H.: Spatio-temporal bird’s-eye view images using multiple fish-eye cameras. In: Proceedings of the 2013 IEEE/SICE International Symposium on System Integration, pp. 753–758 (2013)Google Scholar
  42. 42.
    Scaramuzza, D., Martinelli, A., Siegwart, R.: A toolbox for easily calibrating omnidirectional cameras. In: 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 5695–5701 (2006)Google Scholar
  43. 43.
    Shen, S., Michael, N., Kumar, V.: Autonomous multi-floor indoor navigation with a computationally constrained MAV. In: 2011 IEEE International Conference on Robotics and Automation, pp. 20–25 (2011)Google Scholar
  44. 44.
    Shibata, T., Tanaka, M., Okutomi, M.: Accurate joint geometric camera calibration of visible and far-infrared cameras. In: IS&T International Symposium on Electronic Imaging (EI 2017) (2017)CrossRefGoogle Scholar
  45. 45.
    Stephan, W., Davide, S., Roland, S.: Monocular-SLAM- based navigation for autonomous micro helicopters in gps-denied environments. J. Field Robot. 28(6), 854–874 (2011)CrossRefGoogle Scholar
  46. 46.
    Sun, W., Iwataki, S., Komatsu, R., Fujii, H., Yamashita, A., Asama, H.: Simultaneous tele-visualization of construction machine and environment using body mounted cameras. In: 2016 IEEE International Conference on Robotics and Biomimetics (ROBIO 2016), pp. 382–387 (2016).
  47. 47.
    Tachi, S.: Telexistence. J. Robot. Soc. Jpn. 33(4), 215–221 (2015). (In Japanese)CrossRefGoogle Scholar
  48. 48.
    Tian, Y., Chen, Z., Jia, T., Wang, A., Li, L.: Sensorless collision detection and contact force estimation for collaborative robots based on torque observer. In: 2016 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 946–951 (2016)Google Scholar
  49. 49.
    Yamaguchi, T., et al.: A survey on the man-machine interface in remote operation. In: 59th Annual Conference of Japan Society of Civil Engineers, vol. 59, pp. 373–374 (2004)Google Scholar
  50. 50.
    Yamauchi, T., Okamoto, S., Konyo, M., Hidaka, Y., Maeno, T., Tadokoro, S.: Real-time remote transmission of multiple tactile properties through master-slave robot system. In: 2010 IEEE International Conference on Robotics and Automation (ICRA), pp. 1753–1760. IEEE (2010)Google Scholar
  51. 51.
    Yoshikawa, T.: Foundation of Robotics. MIT Press, Cambridge (1990)Google Scholar
  52. 52.
    Zhang, Z.: Flexible camera calibration by viewing a plane from unknown orientations. In: 7th International Conference on Computer Vision (ICCV 1999), vol. 1, pp. 666–673 (1999)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Osaka UniversityOsakaJapan
  2. 2.Tohoku UniversitySendaiJapan
  3. 3.The University of TokyoTokyoJapan
  4. 4.Advanced Industrial Science and Technology (AIST)/Tokyo Institute of TechnologyTokyoJapan
  5. 5.University of TokyoTokyoJapan
  6. 6.NEC CorporationTokyoJapan
  7. 7.Tokyo Institute of TechnologyTokyoJapan
  8. 8.Advanced Industrial Science and Technology (AIST)TokyoJapan
  9. 9.Kobe UniversityKobeJapan
  10. 10.Keio UniversityTokyoJapan
  11. 11.Yokohama National UniversityYokohamaJapan
  12. 12.JPN CO. LTD.OtaJapan
  13. 13.Meijyo UniversityMeijyoJapan
  14. 14.International Rescue System Institute (IRS)KobeJapan
  15. 15.Nagaoka University of TechnologyNagaokaJapan
  16. 16.Texas A&M UniversityTexasUSA

Personalised recommendations