Precise planar motion measurement of a swimming multi-joint robotic fish



This paper presents a method for planar motion measurement of a swimming multi-joint robotic fish. The motion of the robotic fish is captured via image sequences and a proposed tracking scheme is employed to continuously detect and track the robotic fish. The tracking scheme initially acquires a rough scope of the robotic fish and thereafter precisely locates it. Historical motion information is utilized to determine the rough scope, which can speed up the tracking process and avoid possible ambient interference. A combination of adaptive bilateral filtering and k-means clustering is then applied to segment out color markers accurately. The pose of the robotic fish is calculated in accordance with the centers of these markers. Further, we address the problem of time synchronization between the on-board motion control system of the robotic fish and the motion measurement system. To the best of our knowledge, this problem has not been tackled in previous research on robotic fish. With information about both the multi-link structure and motion law of the robotic fish, we convert the problem to a nonlinear optimization problem, which we then solve using the particle swarm optimization (PSO) algorithm. Further, smoothing splines are adopted to fit curves of poses versus time, in order to obtain a continuous motion state and alleviate the impact of noise. Velocity is acquired via a temporal derivative operation. The results of experiments conducted verify the efficacy of the proposed method.




  1. (1)


  2. (2)

    提出了一种机器鱼板载控制系统与运动测量系统之间的时间同步算法, 利用机器鱼的多连杆结构和关节运动规律解决时间同步问题;

  3. (3)

    采用平滑样条函数拟合位姿曲线, 以抑制噪声的影响, 并基于此获取运动速度。

This is a preview of subscription content, access via your institution.


  1. 1

    Su Z S, Yu J Z, TanM, et al. Implementing flexible and fast turning maneuvers of a multijoint robotic fish. IEEE/ASME Trans Mechatron, 2014, 19: 329–338

    Article  Google Scholar 

  2. 2

    Kopman V, Laut J, Acquaviva F, et al. Dynamic modeling of a robotic fish propelled by a compliant tail. IEEE J Oceanic Eng, 2014, 40: 209–221

    Article  Google Scholar 

  3. 3

    Swain D T, Couzin I D, Leonard N E. Real-time feedback-controlled robotic fish for behavioral experiments with fish schools. Proc IEEE, 2012, 100: 150–163

    Article  Google Scholar 

  4. 4

    Takada Y, Koyama K, Usami T. Position estimation of small robotic fish based on camera information and Gyro sensors. Robotics, 2014, 3: 149–162

    Article  Google Scholar 

  5. 5

    Wang W, Xie G M. Online high-precision probabilistic localization of robotic fish using visual and inertial cues. IEEE Trans Ind Electron, 2014, 62: 1113–1124

    Article  Google Scholar 

  6. 6

    Fontaine E, Lentink D, Kranenbarg S, et al. Automated visual tracking for studying the ontogeny of zebrafish swimming. J Exp Biol, 2008, 211: 1305–1316

    Article  Google Scholar 

  7. 7

    Butail S, Paley D A. Three-dimensional reconstruction of the fast-start swimming kinematics of densely schooling fish. J R Soc Interface, 2012, 9: 77–88

    Article  Google Scholar 

  8. 8

    Porez M, Boyer F, Ijspeert A J. Improved Lighthill fish swimming model for bio-inspired robots: modeling, computational aspects and experimental comparisons. Int J Robot Res, 2014, 33: 1322–1341

    Article  Google Scholar 

  9. 9

    Marchese A D, Onal C D, Rus D. Autonomous soft robotic fish capable of escape maneuvers using fluidic elastomer actuators. Soft Robot, 2014, 1: 75–87

    Article  Google Scholar 

  10. 10

    Yu J Z, Wang S, Tan M. A parallel algorithm for visual tracking of multiple free-swimming robot fishes based on color information. In: Proceedings of IEEE International Conference on Intelligent Systems and Signal Processing, Changsha, 2003. 359–364

    Google Scholar 

  11. 11

    Kopman V, Laut J, Polverino G, et al. Closed-loop control of zebrafish response using a bioinspired robotic-fish in a preference test. J R Soc Interface, 2013, 10: 20120540

    Article  Google Scholar 

  12. 12

    Butail S, Bartolini T, Porfiri M. Collective response of zebrafish shoals to a free-swimming robotic fish. PLoS ONE, 2013, 8: e76123

    Article  Google Scholar 

  13. 13

    Huck T, Westenberger A, Fritzsche M, et al. Precise timestamping and temporal synchronization in multi-sensor fusion. In: Proceedings of IEEE Intelligent Vehicles Symposium, Baden, 2011. 242–247

    Google Scholar 

  14. 14

    Romer K. Time synchronization and localization in sensor networks. Dissertation for Ph.D. Degree. Zurich: Swiss Federal Institute of Technology Zurich, 2005. 97–137

    Google Scholar 

  15. 15

    Liu T, Hu Y, Hua Y, et al. Study on autonomous and distributed time synchronization method for formation UAVs. In: Proceedings of IEEE International Joint Conference of FCS, Denver, 2015. 293–296

    Google Scholar 

  16. 16

    Nakadai K, Nakajima H, Murase M, et al. Real-time tracking of multiple sound sources by integration of in-room and robot-embedded microphone arrays. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, Beijing, 2006. 852–859

    Google Scholar 

  17. 17

    Lim A, Mizumoto T, Cahier L K, et al. Robot musical accompaniment: Integrating audio and visual cues for real-time synchronization with a human flutist. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, Taipei, 2010. 1964–1969

    Google Scholar 

  18. 18

    Nilsson O, Handel P. Time synchronization and temporal ordering of asynchronous sensor measurements of a multisensor navigation system. In: Proceedings of IEEE/ION Position Location and Navigation Symposium, Indian Wells, 2010. 897–902

    Google Scholar 

  19. 19

    Wu Z X, Yu J Z, Su Z S, et al. Towards an Esox lucius inspired multimodal robotic fish. Sci China Inf Sci, 2015, 58: 052203

    Google Scholar 

  20. 20

    Tomasi C, Manduchi R. Bilateral filtering for gray and color images. In: Proceedings of IEEE International Conference on Computer Vision, Bombay, 1998. 839–846

    Google Scholar 

  21. 21

    Hartigan J A, Wong M A. A k-means clustering algorithm. J Roy Stat Soc C-App, 1979, 28: 100–108

    MATH  Google Scholar 

  22. 22

    Wu Z X, Yu J Z, Tan M, et al. Kinematic comparison of forward and backward swimming and maneuvering in a self-propelled sub-carangiform robotic fish. J Bionic Eng, 2014, 11: 199–212

    Article  Google Scholar 

  23. 23

    Kennedy J. Particle swarm optimization. In: Sammut C, Webb G, eds. Encyclopedia of Machine Learning. Berlin: Springer, 2010. 760–766

    Google Scholar 

  24. 24

    Ebbesen S, Kiwitz P, Guzzella L. A generic particle swarm optimization Matlab function. In: Proceedings of American Control Conference, Montreal, 2012. 1519–1524

    Google Scholar 

  25. 25

    Reinsch C H. Smoothing by spline functions. Numer Math, 1967, 10: 177–183

    MathSciNet  Article  MATH  Google Scholar 

  26. 26

    Comaniciu D, Meer P. Mean shift: a robust approach toward feature space analysis. IEEE Trans Patt Anal and Mach Intell, 2002, 24: 603–619

    Article  Google Scholar 

  27. 27

    Rother C, Kolmogorov V, Blake A. Grabcut: interactive foreground extraction using iterated graph cuts. ACM Trans Graph, 2004, 23: 309–314

    Article  Google Scholar 

Download references

Author information



Corresponding author

Correspondence to Junzhi Yu.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Yuan, J., Yu, J., Wu, Z. et al. Precise planar motion measurement of a swimming multi-joint robotic fish. Sci. China Inf. Sci. 59, 92208 (2016).

Download citation


  • motion measurement
  • robotic fish
  • time synchronization
  • visual tracking
  • localization


  • 运动测量
  • 机器鱼
  • 时间同步
  • 视觉跟踪
  • 定位