Abstract
This article concerns tracking of floating objects using fixed-wing UAVs with a monocular thermal camera. Target tracking from an agile aerial vehicle is challenging because uncertainty in the UAV pose negatively affects the accuracy of the measurements obtained through thermal images. Consequently, the accuracy of the tracking estimates is degraded if navigation uncertainty is neglected. This is especially relevant for the estimated target covariance since inconsistency is a likely consequence. A tracking system based on the Schmidt-Kalman filter is proposed to mitigate navigation uncertainty. Images gathered with an uncertain UAV pose are weighted less than images captured with a reliable pose. The UAV pose is estimated independently in a multiplicative extended Kalman filter where the estimated covariance matrix is a measure of the uncertainty. The method is compared experimentally with two traditional alternatives based on the extended Kalman filter. The results show that the proposed method performs better with respect to consistency and accuracy.
Article PDF
Data Availability
Data available upon request.
References
Al-Isawi, M.M., Sasiadek, J.Z.: Guidance and control of a robot capturing an uncooperative space target. J. Intell. Robot. Syst. 93(3), 713–721 (2019)
Albrektsen, S.M., Johansen, T.A.: Syncboard - a high accuracy sensor timing board for uav payloads. In: The International Conference on Unmanned Aircraft Systems. https://doi.org/10.1109/ICUAS.2017.7991410, pp 1706–1715 (2017)
Bar-Shalom, Y., Li, X.R., Kirubarajan, T.: Estimation with Applications to Tracking and Navigation: Theory Algorithms and Software. Wiley, New York (2004)
Bar-Shalom, Y., Willett, P.K., Tian, X.: Tracking and Data Fusion: A handbook of Algorithms. YBS Publishing Storrs, USA (2011)
Beard, R.W., McLain, T.W.: Small Unmanned Aircraft: Theory and Practice. Princeton University Press, Princeton (2012)
Brekke, E.F., Wilthil, E.F.: Suboptimal kalman filters for target tracking with navigation uncertainty in one dimension. In: 2017 IEEE Aerospace Conference. https://doi.org/10.1109/AERO.2017.7943601, pp 1–11 (2017)
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I., Leonard, J.J.: Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age. IEEE Trans. Robot. 32(6), 1309–1332 (2016). https://doi.org/10.1109/TRO.2016.2624754
Farrell, J.: Aided Navigation: GPS with High Rate Sensors, 1st edn. McGraw-Hill Inc, New York (2008)
Fossen, T.: Handbook of Marine Craft Hydrodynamics and Motion Control. Wiley, New York (2011)
Fossen, T.: Handbook of Marine Craft Hydrodynamics and Motion Control, 2nd edn. Wiley, New York (2021)
Heikkila, J., Silven, O., et al.: A four-step camera calibration procedure with implicit image correction. In: CVPR, vol. 97, p 1106. Citeseer (1997)
Helgesen, H.H., Leira, F.S., Bryne, T.H., Albrektsen, S.M., Johansen, T.A.: Real-time georeferencing of thermal images using small fixed-wing uavs in maritime environments. ISPRS J. Photogramm. Remote. Sens. 154, 84–97 (2019). https://doi.org/10.1016/j.isprsjprs.2019.05.009
Helgesen, H.H., Leira, F.S., Fossen, T.I., Johansen, T.A.: Tracking of ocean surface objects from unmanned aerial vehicles with a pan/tilt unit using a thermal camera. J. Intell. Robot. Syst. https://doi.org/10.1007/s10846-017-0722-3 (2017)
Helgesen, H.H., Leira, F.S., Johansen, T.A.: Colored-noise tracking of floating objects using uavs with thermal cameras. In: The International Conference on Unmanned Aircraft Systems (2019)
Helgesen, H.H., Leira, F.S., Johansen, T.A., Fossen, T.I., Pettersen, K.Y., Nijmeijer, H.: Detection and tracking of floating objects using a uav with thermal camera. In: Fossen, T.I. (ed.) Sensing and Control for Autonomous Vehicles: Applications to Land, Water and Air Vehicles. https://doi.org/10.1007/978-3-319-55372-6∖_14, pp 289–316. Springer International Publishing (2017)
Kanellakis, C., Nikolakopoulos, G.: Survey on computer vision for uavs: Current developments and trends. J. Intell. Robot. Syst. 87(1), 141–168 (2017)
Leira, F.S., Johansen, T.A., Fossen, T.I.: Automatic detection, classification and tracking of objects in the ocean surface from uavs using a thermal camera. In: 2015 IEEE Aerospace Conference. https://doi.org/10.1109/AERO.2015.7119238, pp 1–10 (2015)
Mallick, M., La Scala, B.: Comparison of single-point and two-point difference track initiation algorithms using position measurements. Acta Autom. Sin. 34(3), 258–265 (2008)
Markley, F.L.: Attitude error representations for kalman filtering. J. Guid. Control Dyn. 26(2), 311–317 (2003). https://doi.org/10.2514/2.5048
Mullane, J., Vo, B., Adams, M.D., Vo, B.: A random-finite-set approach to bayesian slam. IEEE Trans. Robot. 27(2), 268–282 (2011)
Novoselov, R.Y., Herman, S.M., Gadaleta, S.M., Poore, A.B.: Mitigating the effects of residual biases with schmidt-kalman filtering. In: 2005 7th International Conference on Information Fusion. https://doi.org/10.1109/ICIF.2005.1591877 (2005)
Pajares, G.: Overview and current status of remote sensing applications based on unmanned aerial vehicles (uavs). Photogramm. Eng. Remote Sens. 81(4), 281–329 (2015). https://doi.org/10.14358/PERS.81.4.281
Prasad, D.K., Rajan, D., Rachmawati, L., Rajabally, E., Quek, C.: Video processing from electro-optical sensors for object detection and tracking in a maritime environment: A survey. IEEE Trans. Intell. Transp. Syst. 18(8), 1993–2016 (2017). https://doi.org/10.1109/TITS.2016.2634580
Li, R.X., Jilkov, V.P.: Survey of maneuvering target tracking. part i. dynamic models. IEEE Trans. Aerosp. Electron. Syst. 39(4), 1333–1364 (2003). https://doi.org/10.1109/TAES.2003.1261132
Schmidt, S.F.: Applications of state space methods to navigation problems. Adv. Control Syst. 3, 293–340 (1966)
Silva, A., Mendonça, R., Santana, P.: Monocular trail detection and tracking aided by visual slam for small unmanned aerial vehicles. J. Intell. Robot. Syst. 97(3), 531–551 (2020)
Sola, J., Quaternion kinematics for the error-state kf. http://www.iri.upc.edu/people/jsola/JoanSola/objectes/notes/kinematics.pdf. Accessed 02 Feb 2018 (2017)
Urzua, S., Munguía, R, Grau, A.: Monocular slam system for mavs aided with altitude and range measurements: a gps-free approach. J. Intell. Robot. Syst. 94(1), 203–217 (2019)
Wang, C.C., Thorpe, C., Thrun, S., Hebert, M., Durrant-Whyte, H.: Simultaneous localization, mapping and moving object tracking. Int. J. Robot. Res. 26(9), 889–916 (2007). https://doi.org/10.1177/0278364907081229
Wang, D., Xu, C., Yuan, P., Huang, D.: A revised monte carlo method for target location with uav. J. Intell. Robot. Syst. 97(2), 373–386 (2020)
Wilthil, E.F., Brekke, E.F.: Compensation of navigation uncertainty for target tracking on a moving platform. In: 2016 19th International Conference on Information Fusion (FUSION), pp 1616–1621 (2016)
Yang, C., Blasch, E., Douville, P.: Design of schmidt-kalman filter for target tracking with navigation errors. In: 2010 IEEE Aerospace Conference. https://doi.org/10.1109/AERO.2010.5446689, pp 1–12 (2010)
Acknowledgements
The authors are grateful for the excellent assistance from UAV operators Lars Semb and Pål Kvaløy.
Funding
Open access funding provided by NTNU Norwegian University of Science and Technology (incl St. Olavs Hospital - Trondheim University Hospital). This work has been carried out at the NTNU Centre for Autonomous Marine Operations and Systems. This work was supported by the Research Council of Norway through the Centres of Excellence funding scheme, Project number 223254. It has also been supported by NFR/ MAROFF with project number 269480.
Author information
Authors and Affiliations
Contributions
This article has four authors, and their names and contributions are: Dr. Håkon Hagen Helgesen wrote the first version of the manuscript, planned, prepared and participated in the experiments (data collection), implemented and derived the tracking system, and performed the analysis of the results. Associate prof. Torleiv H. Bryne prepared the navigation sensors, participated in the experiments, and implemented the navigation system. Dr. Erik F. Wilthil derived the theoretical foundation of the tracking system together with Håkon. Prof. Tor Arne Johansen led the research effort and provided valuable suggestions and discussions before conducting experiments and in the process of writing the manuscript. All authors have commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
This paper does not report research that requires ethical approval. Consent to participate or consent to publish statements are accordingly also not required.
Competing interests
The authors have no conflicts of interest to declare that are relevant to the content of this article.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
All authors are with the NTNU Centre for Autonomous Marine Operations and Systems
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Helgesen, H.H., Bryne, T.H., Wilthil, E.F. et al. Camera-Based Tracking of Floating Objects using Fixed-wing UAVs. J Intell Robot Syst 102, 80 (2021). https://doi.org/10.1007/s10846-021-01432-z
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10846-021-01432-z