Structured Output Prediction with Hierarchical Loss Functions for Seafloor Imagery Taxonomic Categorization

  • Navid Nourani-Vatani
  • Roberto López-Sastre
  • Stefan Williams
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9117)


In this paper we study the challenging problem of seafloor imagery taxonomic categorization. Our contribution is threefold. First, we demonstrate that this task can be elegantly translated into a Structured SVM learning framework. Second, we introduce a taxonomic loss function in the structured output classification objective during learning that is shown to improve the performance over other loss functions. And third, we show how the Structured SVM can naturally deal with the problem of learning from data imbalance by scaling the cost of misclassification during the optimization. We present a thorough experimental evaluation using the challenging and publicly available Tasmania Coral Point Count dataset, where our models drastically outperform the state-of-the-art-results reported.


Seafloor imagery Categorization Recognition Structured prediction 



The authors acknowledge the Australian National Research Program (NERP) Marine Biodiversity Hub for the taxonomical labeling and the Australian Centre for Field Robotics for gathering the image data.


  1. 1.
    Williams, S., Pizarro, O., Jakuba, M., Johnson, C., Barrett, N., Babcock, R., Kendrick, G., Steinberg, P., Heyward, A., Doherty, P., et al.: Monitoring of benthic reference sites: using an autonomous underwater vehicle. IEEE Robot. Autom. Mag. 19(1), 73–84 (2012)CrossRefGoogle Scholar
  2. 2.
    Johnson-Roberson, M., Kumar, S., Williams, S.: Segmentation and classification of coral for oceanographic surveys: a semi-supervised machine learning approach. In: IEEE OCEANS, pp. 1–6 (2006)Google Scholar
  3. 3.
    Beijbom, O., Edmunds, P.J., Kline, D., Mitchell, B., Kriegman, D.: Automated annotation of coral reef survey images. In: IEEE CVPR, pp. 1170–1177 (2012)Google Scholar
  4. 4.
    Bewley, M., Douillard, B., Nourani-Vatani, N., Friedman, A., Pizarro, O., Williams, S.: Automated species detection: an experimental approach to kelp detection from sea-floor AUV images. In: ACRA (2012)Google Scholar
  5. 5.
    Tsochantaridis, I., Hofmann, T., Joachims, T., Altun, Y.: Support vector machine learning for interdependent and structured output spaces. In: ICML, p. 104 (2004)Google Scholar
  6. 6.
    Bewley, M., Nourani-Vatani, N., Rao, D., Douillard, B., Pizarro, O., Williams, S.: Hierarchical classification in AUV imagery. In: Mejias, L., Corke, P., Roberts, J. (eds.) Field and Service Robotics. Springer, Switzerland (2015)Google Scholar
  7. 7.
    Smith, D., Dunbabin, M.: Automated counting of the northern pacific sea star in the derwent using shape recognition. In: DICTA, pp. 500–507, September 2007Google Scholar
  8. 8.
    Soriano, M., Marcos, S., Saloma, C., Quibilan, M., Alino, P.: Image classification of coral reef components from underwater color video. In: MTS/IEEE Conference and Exhibition OCEANS, pp. 1008–1013 (2001)Google Scholar
  9. 9.
    Cai, L., Hofmann, T.: Hierarchical document categorization with support vector machines. In: ACM CIKM, pp. 78–87 (2004)Google Scholar
  10. 10.
    Tuia, D., Muñoz-Marí, J., Kanevski, M., Camps-Valls, G.: Structured output SVM for remote sensing image classification. J. Signal Process. Syst. 65(3), 301–310 (2011)CrossRefGoogle Scholar
  11. 11.
    Binder, A., Müller, K., Kawanabe, M.: On taxonomies for multi-class image categorization. Int. J. Comput. Vision (IJCV) 99, 281–301 (2012)CrossRefGoogle Scholar
  12. 12.
    Joachims, T.: Multi-class support vector machine (2008).
  13. 13.
    Wang, K., Zhou, S., Liew, S.: Building hierarchical classifiers using class proximity. In: International Conference on Very Large Data Bases (VLDB) (1999)Google Scholar
  14. 14.
    Veropoulos, K., Campbell, C., Cristianini, N.: Controlling the sensitivity of support vector machines. In: IJCAI, vol. 1999, pp. 55–60 (1999)Google Scholar
  15. 15.
    Japkowicz, N., Stephen, S.: The class imbalance problem: a systematic study. Intell. Data Anal. 6(5), 429–449 (2002)zbMATHGoogle Scholar
  16. 16.
    Ahonen, T., Matas, J., He, C., Pietikäinen, M.: Rotation invariant image description with local binary pattern histogram fourier features. In: Salberg, A.-B., Hardeberg, J.Y., Jenssen, R. (eds.) SCIA 2009. LNCS, vol. 5575, pp. 61–70. Springer, Heidelberg (2009) CrossRefGoogle Scholar
  17. 17.
    Chang, C., Lin, C.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2, 27:1–27:27 (2011)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Navid Nourani-Vatani
    • 1
  • Roberto López-Sastre
    • 2
  • Stefan Williams
    • 3
  1. 1.EREDMAN Truck and Bus AGMunichGermany
  2. 2.GRAMUniversity of AlcaláAlcalá de HenaresSpain
  3. 3.ACFRUniversity of SydneySydneyAustralia

Personalised recommendations