Precise planar motion measurement of a swimming multi-joint robotic fish

  • Jun Yuan
  • Junzhi YuEmail author
  • Zhengxing Wu
  • Min Tan
Research Paper


This paper presents a method for planar motion measurement of a swimming multi-joint robotic fish. The motion of the robotic fish is captured via image sequences and a proposed tracking scheme is employed to continuously detect and track the robotic fish. The tracking scheme initially acquires a rough scope of the robotic fish and thereafter precisely locates it. Historical motion information is utilized to determine the rough scope, which can speed up the tracking process and avoid possible ambient interference. A combination of adaptive bilateral filtering and k-means clustering is then applied to segment out color markers accurately. The pose of the robotic fish is calculated in accordance with the centers of these markers. Further, we address the problem of time synchronization between the on-board motion control system of the robotic fish and the motion measurement system. To the best of our knowledge, this problem has not been tackled in previous research on robotic fish. With information about both the multi-link structure and motion law of the robotic fish, we convert the problem to a nonlinear optimization problem, which we then solve using the particle swarm optimization (PSO) algorithm. Further, smoothing splines are adopted to fit curves of poses versus time, in order to obtain a continuous motion state and alleviate the impact of noise. Velocity is acquired via a temporal derivative operation. The results of experiments conducted verify the efficacy of the proposed method.


motion measurement robotic fish time synchronization visual tracking localization 




  1. (1)


  2. (2)

    提出了一种机器鱼板载控制系统与运动测量系统之间的时间同步算法, 利用机器鱼的多连杆结构和关节运动规律解决时间同步问题;

  3. (3)

    采用平滑样条函数拟合位姿曲线, 以抑制噪声的影响, 并基于此获取运动速度。



运动测量 机器鱼 时间同步 视觉跟踪 定位 


  1. 1.
    Su Z S, Yu J Z, TanM, et al. Implementing flexible and fast turning maneuvers of a multijoint robotic fish. IEEE/ASME Trans Mechatron, 2014, 19: 329–338CrossRefGoogle Scholar
  2. 2.
    Kopman V, Laut J, Acquaviva F, et al. Dynamic modeling of a robotic fish propelled by a compliant tail. IEEE J Oceanic Eng, 2014, 40: 209–221CrossRefGoogle Scholar
  3. 3.
    Swain D T, Couzin I D, Leonard N E. Real-time feedback-controlled robotic fish for behavioral experiments with fish schools. Proc IEEE, 2012, 100: 150–163CrossRefGoogle Scholar
  4. 4.
    Takada Y, Koyama K, Usami T. Position estimation of small robotic fish based on camera information and Gyro sensors. Robotics, 2014, 3: 149–162CrossRefGoogle Scholar
  5. 5.
    Wang W, Xie G M. Online high-precision probabilistic localization of robotic fish using visual and inertial cues. IEEE Trans Ind Electron, 2014, 62: 1113–1124CrossRefGoogle Scholar
  6. 6.
    Fontaine E, Lentink D, Kranenbarg S, et al. Automated visual tracking for studying the ontogeny of zebrafish swimming. J Exp Biol, 2008, 211: 1305–1316CrossRefGoogle Scholar
  7. 7.
    Butail S, Paley D A. Three-dimensional reconstruction of the fast-start swimming kinematics of densely schooling fish. J R Soc Interface, 2012, 9: 77–88CrossRefGoogle Scholar
  8. 8.
    Porez M, Boyer F, Ijspeert A J. Improved Lighthill fish swimming model for bio-inspired robots: modeling, computational aspects and experimental comparisons. Int J Robot Res, 2014, 33: 1322–1341CrossRefGoogle Scholar
  9. 9.
    Marchese A D, Onal C D, Rus D. Autonomous soft robotic fish capable of escape maneuvers using fluidic elastomer actuators. Soft Robot, 2014, 1: 75–87CrossRefGoogle Scholar
  10. 10.
    Yu J Z, Wang S, Tan M. A parallel algorithm for visual tracking of multiple free-swimming robot fishes based on color information. In: Proceedings of IEEE International Conference on Intelligent Systems and Signal Processing, Changsha, 2003. 359–364Google Scholar
  11. 11.
    Kopman V, Laut J, Polverino G, et al. Closed-loop control of zebrafish response using a bioinspired robotic-fish in a preference test. J R Soc Interface, 2013, 10: 20120540CrossRefGoogle Scholar
  12. 12.
    Butail S, Bartolini T, Porfiri M. Collective response of zebrafish shoals to a free-swimming robotic fish. PLoS ONE, 2013, 8: e76123CrossRefGoogle Scholar
  13. 13.
    Huck T, Westenberger A, Fritzsche M, et al. Precise timestamping and temporal synchronization in multi-sensor fusion. In: Proceedings of IEEE Intelligent Vehicles Symposium, Baden, 2011. 242–247Google Scholar
  14. 14.
    Romer K. Time synchronization and localization in sensor networks. Dissertation for Ph.D. Degree. Zurich: Swiss Federal Institute of Technology Zurich, 2005. 97–137Google Scholar
  15. 15.
    Liu T, Hu Y, Hua Y, et al. Study on autonomous and distributed time synchronization method for formation UAVs. In: Proceedings of IEEE International Joint Conference of FCS, Denver, 2015. 293–296Google Scholar
  16. 16.
    Nakadai K, Nakajima H, Murase M, et al. Real-time tracking of multiple sound sources by integration of in-room and robot-embedded microphone arrays. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, Beijing, 2006. 852–859Google Scholar
  17. 17.
    Lim A, Mizumoto T, Cahier L K, et al. Robot musical accompaniment: Integrating audio and visual cues for real-time synchronization with a human flutist. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, Taipei, 2010. 1964–1969Google Scholar
  18. 18.
    Nilsson O, Handel P. Time synchronization and temporal ordering of asynchronous sensor measurements of a multisensor navigation system. In: Proceedings of IEEE/ION Position Location and Navigation Symposium, Indian Wells, 2010. 897–902CrossRefGoogle Scholar
  19. 19.
    Wu Z X, Yu J Z, Su Z S, et al. Towards an Esox lucius inspired multimodal robotic fish. Sci China Inf Sci, 2015, 58: 052203Google Scholar
  20. 20.
    Tomasi C, Manduchi R. Bilateral filtering for gray and color images. In: Proceedings of IEEE International Conference on Computer Vision, Bombay, 1998. 839–846Google Scholar
  21. 21.
    Hartigan J A, Wong M A. A k-means clustering algorithm. J Roy Stat Soc C-App, 1979, 28: 100–108zbMATHGoogle Scholar
  22. 22.
    Wu Z X, Yu J Z, Tan M, et al. Kinematic comparison of forward and backward swimming and maneuvering in a self-propelled sub-carangiform robotic fish. J Bionic Eng, 2014, 11: 199–212CrossRefGoogle Scholar
  23. 23.
    Kennedy J. Particle swarm optimization. In: Sammut C, Webb G, eds. Encyclopedia of Machine Learning. Berlin: Springer, 2010. 760–766Google Scholar
  24. 24.
    Ebbesen S, Kiwitz P, Guzzella L. A generic particle swarm optimization Matlab function. In: Proceedings of American Control Conference, Montreal, 2012. 1519–1524Google Scholar
  25. 25.
    Reinsch C H. Smoothing by spline functions. Numer Math, 1967, 10: 177–183MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Comaniciu D, Meer P. Mean shift: a robust approach toward feature space analysis. IEEE Trans Patt Anal and Mach Intell, 2002, 24: 603–619CrossRefGoogle Scholar
  27. 27.
    Rother C, Kolmogorov V, Blake A. Grabcut: interactive foreground extraction using iterated graph cuts. ACM Trans Graph, 2004, 23: 309–314CrossRefGoogle Scholar

Copyright information

© Science China Press and Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  1. 1.State Key Laboratory of Management and Control for Complex Systems, Institute of AutomationChinese Academy of SciencesBeijingChina

Personalised recommendations