Skip to main content

Advertisement

Log in

Deep learning for automated analysis of fish abundance: the benefits of training across multiple habitats

  • Published:
Environmental Monitoring and Assessment Aims and scope Submit manuscript

Abstract

Environmental monitoring guides conservation and is particularly important for aquatic habitats which are heavily impacted by human activities. Underwater cameras and uncrewed devices monitor aquatic wildlife, but manual processing of footage is a significant bottleneck to rapid data processing and dissemination of results. Deep learning has emerged as a solution, but its ability to accurately detect animals across habitat types and locations is largely untested for coastal environments. Here, we produce five deep learning models using an object detection framework to detect an ecologically important fish, luderick (Girella tricuspidata). We trained two models on footage from single habitats (seagrass or reef) and three on footage from both habitats. All models were subjected to tests from both habitat types. Models performed well on test data from the same habitat type (object detection measure: mAP50: 91.7 and 86.9% performance for seagrass and reef, respectively) but poorly on test sets from a different habitat type (73.3 and 58.4%, respectively). The model trained on a combination of both habitats produced the highest object detection results for both tests (an average of 92.4 and 87.8%, respectively). The ability of the combination trained models to correctly estimate the ecological abundance metric, MaxN, showed similar patterns. The findings demonstrate that deep learning models extract ecologically useful information from video footage accurately and consistently and can perform across habitat types when trained on footage from the variety of habitat types.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2

References

  • Abrantes, K. G., Barnett, A., Baker, R., & Sheaves, M. (2015). Habitat-specific food webs and trophic interactions supporting coastal-dependent fishery species: An Australian case study. Reviews in Fish Biology and Fisheries, 25(2), 337–363.

    Article  Google Scholar 

  • Alom, M. Z., Taha, T. M., Yakopcic, C., Westberg, S., Sidike, P., Nasrin, M. S., Hasan, M., van Essen, B. C., Awwal, A. A. S., & Asari, V. K. (2019). A state-of-the-art survey on deep learning theory and architectures. Electronics, 8(3), 292.

    Article  Google Scholar 

  • Beijbom, O., Edmunds, P. J., Kline, D. I., Mitchell, B. G., & Kriegman, D. (2012). Automated annotation of coral reef survey images. IEEE Conference on Computer Vision and Pattern Recognition, 2012, 1170–1177.

  • Buckland, M., & Gey, F. (1994). The relationship between recall and precision. Journal of the American Society for Information Science, 45(1), 12–19.

    Article  Google Scholar 

  • Christin, S., Hervet, E., & Lecomte, N. (2019). Applications for deep learning in ecology. Methods in Ecology and Evolution, 10(10), 1632–1644.

    Article  Google Scholar 

  • Davidson, N. C. (2014). How much wetland has the world lost? Long-term and recent trends in global wetland area. Marine and Freshwater Research, 65(10), 934–941.

    Article  Google Scholar 

  • Ditria, E. M., Lopez-Marcano, S., Sievers, M., Jinks, E. L., Brown, C. J., & Connolly, R. M. (2020). Automating the analysis of fish abundance using object detection: Optimizing animal ecology with deep learning. Frontiers in Marine Science, 7, 429.

    Article  Google Scholar 

  • dos Santos, A. A., & Goncalves, W. N. (2019). Improving Pantanal fish species recognition through taxonomic ranks in convolutional neural networks. Ecological Informatics, 53, 100977. https://doi.org/10.1016/j.ecoinf.2019.100977.

    Article  Google Scholar 

  • Everingham, M., Van Gool, L., Williams, C. K., Winn, J., & Zisserman, A. (2010). The pascal visual object classes (voc) challenge. International Journal of Computer Vision, 88(2), 303–338.

    Article  Google Scholar 

  • Ferguson, A., Harvey, E. S., Rees, M., & Knott, N. A. (2015). Does the abundance of girellids and kyphosids correlate with cover of the palatable green algae, Ulva spp.? A test on temperate rocky intertidal reefs. Journal of Fish Biology, 86(1), 375–384.

    Article  CAS  Google Scholar 

  • Frid, A., & Dill, L. (2002). Human-caused disturbance stimuli as a form of predation risk. Conservation Ecology, 6(1).

  • Goldsmith, F. B. (2012). Monitoring for conservation and ecology (Vol. 3). Springer Science & Business Media.

  • Goutte, C., & Gaussier, E. (2005). A probabilistic interpretation of precision, recall and F-score, with implication for evaluation. Paper presented at the European Conference on Information Retrieval.

    Book  Google Scholar 

  • Hobday, A. J., & Pecl, G. T. (2014). Identification of global marine hotspots: Sentinels for change and vanguards for adaptation action. Reviews in Fish Biology and Fisheries, 24(2), 415–425.

    Article  Google Scholar 

  • Igulu, M. M., Nagelkerken, I., Dorenbosch, M., Grol, M. G., Harborne, A. R., Kimirei, I. A., et al. (2014). Mangrove habitat use by juvenile reef fish: Meta-analysis reveals that tidal regime matters more than biogeographic region. PLoS One, 9(12), e114715.

    Article  Google Scholar 

  • Kalogeiton, V., Ferrari, V., & Schmid, C. (2016). Analysing domain shift factors between videos and images for object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(11), 2327–2334.

    Article  Google Scholar 

  • Lecchini, D., & Galzin, R. (2005). Spatial repartition and ontogenetic shifts in habitat use by coral reef fishes (Moorea, French Polynesia). Marine Biology, 147(1), 47–58.

    Article  Google Scholar 

  • LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444.

    Article  CAS  Google Scholar 

  • Mandal, R., Connolly, R. M., Schlacher, T. A., & Stantic, B. (2018). Assessing fish abundance from underwater video using deep neural networks. Paper presented at the 2018 international joint conference on neural networks (IJCNN).

  • Massa, F., & Girshick, R. (2018). Maskrcnn-benchmark: Fast, modular reference implementation of instance segmentation and object detection algorithms in PyTorch.

  • Maxwell, S. L., Fuller, R. A., Brooks, T. M., & Watson, J. E. (2016). Biodiversity: The ravages of guns, nets and bulldozers. Nature News, 536(7615), 143–145.

    Article  CAS  Google Scholar 

  • Moniruzzaman, M., Islam, S. M. S., Bennamoun, M., & Lavery, P. (2017). Deep learning on underwater marine object detection: A survey. Paper presented at the International Conference on Advanced Concepts for Intelligent Vision Systems.

    Book  Google Scholar 

  • Podder, T. K., Sibenac, M., & Bellingham, J. G. (2019). Applications and challenges of AUV docking systems deployed for long-term science missions. Monterey Bay Aquarium Research Institute.

  • Pollock, B. (2017). Latitudinal change in the distribution of luderick Girella tricuspidata (Pisces: Girellidae) associated with increasing coastal water temperature in eastern Australia. Marine and Freshwater Research, 68(6), 1187–1192.

    Article  Google Scholar 

  • Prechelt, L. (1998). Early stopping-but when? In Neural Networks: Tricks of the trade (pp. 55-69): Springer.

  • Rawat, W., & Wang, Z. (2017). Deep convolutional neural networks for image classification: A comprehensive review. Neural Computation, 29(9), 2352–2449.

    Article  Google Scholar 

  • Ridgway, K. (2007). Long-term trend and decadal variability of the southward penetration of the east Australian current. Geophysical Research Letters, 34(13).

  • Salman, A., Maqbool, S., Khan, A. H., Jalal, A., & Shafait, F. (2019a). Real-time fish detection in complex backgrounds using probabilistic background modelling. Ecological Informatics, 51, 44–51.

    Article  Google Scholar 

  • Salman, A., Siddiqui, S. A., Shafait, F., Mian, A., Shortis, M. R., Khurshid, K., et al. (2019b). Automatic fish detection in underwater videos by a deep neural network-based hybrid motion learning system. ICES Journal of Marine Science.

  • Sarwar, S. S., Ankit, A., & Roy, K. (2019). Incremental learning in deep convolutional neural networks using partial network sharing. IEEE Access.

  • Sheaves, M., Bradley, M., Herrera, C., Mattone, C., Lennard, C., Sheaves, J., & Konovalov, D. A. (2020). Optimizing video sampling for juvenile fish surveys: Using deep learning and evaluation of assumptions to produce critical fisheries parameters. Fish and Fisheries.

  • Sievers, M., Brown, C. J., Tulloch, V. J., Pearson, R. M., Haig, J. A., Turschwell, M. P., et al. (2019). The role of vegetated coastal wetlands for marine megafauna conservation. Trends in Ecology & Evolution, 34, 807–817.

    Article  Google Scholar 

  • Silliman, B. R., He, Q., Angelini, C., Smith, C. S., Kirwan, M. L., Daleo, P., et al. (2019). Field experiments and meta-analysis reveal wetland vegetation as a crucial element in the coastal protection paradigm. Current Biology, 29(11), 1800–1806. e1803.

    Article  CAS  Google Scholar 

  • Spampinato, C., Giordano, D., Di Salvo, R., Chen-Burger, Y.-H. J., Fisher, R. B., & Nadarajan, G. (2010). Automatic fish classification for underwater species behavior understanding. In Proceedings of the first ACM international workshop on Analysis and retrieval of tracked events and motion in imagery streams (pp. 45–50).

  • Tao, Y., Tu, Y., & Shyu, M.-L. (2019). Efficient Incremental Training for Deep Convolutional Neural Networks. In 2019 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR) (pp. 286–291): IEEE.

  • Townhill, B. L., Radford, Z., Pecl, G., van Putten, I., Pinnegar, J. K., & Hyder, K. (2019). Marine recreational fishing and the implications of climate change. Fish and Fisheries, 20(5), 977–992.

    Article  Google Scholar 

  • Tulloch, V. J., Turschwell, M. P., Giffin, A. L., Halpern, B. S., Connolly, R., Griffiths, L., et al. (2020). Linking threat maps with management to guide conservation investment. Biological Conservation, 245, 108527.

    Article  Google Scholar 

  • Vergés, A., Doropoulos, C., Czarnik, R., McMahon, K., Llonch, N., & Poore, A. G. (2018). Latitudinal variation in seagrass herbivory: Global patterns and explanatory mechanisms. Global Ecology and Biogeography, 27(9), 1068–1079.

    Article  Google Scholar 

  • Villon, S., Chaumont, M., Subsol, G., Villéger, S., Claverie, T., & Mouillot, D. (2016). Coral reef fish detection and recognition in underwater videos by supervised machine learning: Comparison between Deep Learning and HOG+ SVM methods. In International Conference on Advanced Concepts for Intelligent Vision Systems (pp. 160–171). Springer.

  • Villon, S., Mouillot, D., Chaumont, M., Darling, E. S., Subsol, G., Claverie, T., & Villéger, S. (2018). A deep learning method for accurate and fast identification of coral reef fishes in underwater images. Ecological Informatics, 48, 238–244.

    Article  Google Scholar 

  • Weinstein, B. G. (2018). A computer vision for animal ecology. Journal of Animal Ecology, 87(3), 533–545.

    Article  Google Scholar 

  • Wendländer, N. S., Lange, T., Connolly, R. M., Kristensen, E., Pearson, R. M., Valdemarsen, T., & Flindt, M. R. (2020). Assessing methods for restoring seagrass (Zostera muelleri) in Australia’s subtropical waters. Marine and Freshwater Research, 71(8), 996–1005.

    Article  Google Scholar 

  • Whitmarsh, S. K., Fairweather, P. G., & Huveneers, C. (2017). What is big BRUVver up to? Methods and uses of baited underwater video. Reviews in Fish Biology and Fisheries, 27(1), 53–73.

    Article  Google Scholar 

  • Xu, W., & Matzner, S. (2018). Underwater fish detection using deep learning for water power applications. Paper presented at the 2018 international conference on computational science and computational intelligence (CSCI),

Download references

Acknowledgements

We thank the volunteers who assisted in manual training of the deep learning algorithm, M. Turner and A. Shand.

Funding

RC was supported by a Discovery Project from the Australian Research Council (DP180103124). All authors were supported by the Global Wetlands Project, with support by a charitable organisation which neither seeks nor permits publicity for its efforts.

Author information

Authors and Affiliations

Authors

Contributions

ED and RC designed the study. ED and SL conducted the fieldwork. ED and EJ developed the deep learning architecture and user interface. RC provided resources. All authors helped interpret results. ED led the writing of the manuscript, with input from all authors.

Corresponding author

Correspondence to Ellen M. Ditria.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

ESM 1

(PDF 314 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ditria, E.M., Sievers, M., Lopez-Marcano, S. et al. Deep learning for automated analysis of fish abundance: the benefits of training across multiple habitats. Environ Monit Assess 192, 698 (2020). https://doi.org/10.1007/s10661-020-08653-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10661-020-08653-z

Keywords

Navigation