Hierarchical Classification in AUV Imagery

  • M. S. Bewley
  • N. Nourani-Vatani
  • D. Rao
  • B. Douillard
  • O. Pizarro
  • S. B. Williams
Part of the Springer Tracts in Advanced Robotics book series (STAR, volume 105)


In recent years, Autonomous Underwater Vehicles (AUVs) have been used extensively to gather imagery and other environmental data for ocean monitoring. Processing of this vast amount of collected imagery to label content is difficult, expensive and time consuming. Because of this, typically only a small subset of images are labelled, and only at a small number of points. In order to make full use of the raw data returned from the AUV, this labelling process needs to be automated. In this work the single species classification problem of [1] is extended to a multi-species classification problem following a taxonomical hierarchy. We demonstrate the application of techniques used in areas such as computer vision, text classification and medical diagnosis to the supervised hierarchical classification of benthic images. After making a comparison to flat multi-class classification, we also discuss critical aspects such as training topology and various prediction and scoring methodologies. An interesting aspect of the presented work is that the ground truth labels are sparse and incomplete, i.e. not all labels go to the leaf node, which brings with it other interesting challenges.We find that the best classification results are obtained using Local Binary Patterns (LBP), training a network of binary classifiers with probabilistic output, and applying “one-vs-rest” classification at each level of the hierarchy for prediction. This work presents a working solution that allows AUV images to be automatically labelled with the most appropriate node in a hierarchy of 19 biological groupings and morphologies. The result is that the output of the AUV system can include a semantic map using the taxonomy prescribed by marine scientists. This has the potential to not only reduce the manual labelling workload, but also to reduce the current dependence that marine scientists have on extrapolating information from a relatively small number of sparsely labelled points.


Leaf Node Principle Component Analysis Local Binary Pattern Image Patch Autonomous Underwater Vehicle 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bewley, M., Douillard, B., Nourani-Vatani, N., Friedman, A., Pizarro, O., Williams, S.: Automated species detection: An experimental approach to kelp detection from sea-floor AUV images (December 2012)Google Scholar
  2. 2.
    Williams, S., Pizarro, O., Jakuba, M., Johnson, C., Barrett, N., Babcock, R., Kendrick, G., Steinberg, P., Heyward, A., Doherty, P., Mahon, I., Johnson-Roberson, M., Steinberg, D., Friedman, A.: Monitoring of benthic reference sites: Using an autonomous underwater vehicle. IEEE Robotics and Automation Magazine 19(1), 73–84 (2012)CrossRefGoogle Scholar
  3. 3.
    Smith, D., Dunbabin, M.: Automated counting of the northern pacific sea star in the derwent using shape recognition. In: The Australasian Conference on Robotics & Automation (ACRA), pp. 500–507 (December 2007)Google Scholar
  4. 4.
    Steinberg, D., Friedman, A., Pizarro, O., Williams, S.: A bayesian nonparametric approach to clustering data from underwater robotic surveys. In: International Symposium on Robotics Research (ISRR) (August 2011)Google Scholar
  5. 5.
    National environmental research program (NERP) marine biodiversity hub, scored autonomous underwater vehicle imagery, east tasmania 2008-2010 (2010),
  6. 6.
    Kohler, K.E., Gill, S.M.: Coral point count with excel extensions (CPCe): a visual basic program for the determination of coral and substrate coverage using random point count methodology. Computers & Geosciences 32(9), 1259–1269 (2006), CrossRefGoogle Scholar
  7. 7.
    Denuelle, A., Dunbabin, M.: Kelp detection in highly dynamic environments using texture recognition. In: The Australasian Conference on Robotics & Automation (ACRA) (December 2010)Google Scholar
  8. 8.
    Beijbom, O., Edmunds, P., Kline, D., Mitchell, B., Kriegman, D.: Automated annotation of coral reef survey images. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), p. 1170 (2012)Google Scholar
  9. 9.
    Mehta, A., Ribeiro, E., Gilner, J., van Woesik, R.: Coral reef texture classification using support vector machines. VISAPP (2), 302–310 (2007)Google Scholar
  10. 10.
    Silla Jr., C.N., Freitas, A.A.: A survey of hierarchical classification across different application domains. Data Mining and Knowledge Discovery 22(1-2), 31–72 (2011)CrossRefzbMATHMathSciNetGoogle Scholar
  11. 11.
    Fagni, T., Sebastiani, F.: On the selection of negative examples for hierarchical text categorization. In: Proceedings of the 3rd Language & Technology Conference (LTC 2007), pp. 24–28 (2007)Google Scholar
  12. 12.
    Kiritchenko, S., Matwin, S., Nock, R., Famili, A.F.: Learning and evaluation in the presence of class hierarchies: application to text categorization. In: Lamontagne, L., Marchand, M. (eds.) Canadian AI 2006. LNCS (LNAI), vol. 4013, pp. 395–406. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  13. 13.
    Ojala, T., Pietikaeinen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(7), 971–987 (2002)CrossRefGoogle Scholar
  14. 14.
    Ahonen, T., Matas, J., He, C., Pietikäinen, M.: Rotation invariant image description with local binary pattern histogram fourier features. In: Salberg, A.-B., Hardeberg, J.Y., Jenssen, R. (eds.) SCIA 2009. LNCS, vol. 5575, pp. 61–70. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  15. 15.
    Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images. Master’s thesis, Department of Computer Science, University of Toronto (2009)Google Scholar
  16. 16.
    Coates, A., Lee, H., Ng, A.Y.: An analysis of single-layer networks in unsupervised feature learning. In: Proceedings of the 14th International Conference on Artificial Intelligence and Statistics (AISTATS), vol. 15 (2011)Google Scholar
  17. 17.
    Barrett, N., Meyer, L., Hill, N., Walsh, P.: Methods for the processing and scoring of AUV digital imagery from south eastern tasmania. University of Tasmania, Technical Report 11560 (August 2011),
  18. 18.
    Edwards, L.: Release of CATAMI classification scheme (February 2013),
  19. 19.
    IMOS: integrated marine observing system (September 2013),

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • M. S. Bewley
    • 1
  • N. Nourani-Vatani
    • 1
  • D. Rao
    • 1
  • B. Douillard
    • 1
  • O. Pizarro
    • 1
  • S. B. Williams
    • 1
  1. 1.Australian Centre for Field RoboticsThe University of SydneySydneyAustralia

Personalised recommendations