Skip to main content
SpringerLink
Log in

Hierarchical classification with reject option for live fish recognition

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

A live fish recognition system is needed in application scenarios where manual annotation is too expensive, i.e. too many underwater videos. We present a novel balance-enforced optimized tree with reject option (BEOTR) for live fish recognition. It recognizes the top 15 common species of fish and detects new species in an unrestricted natural environment recorded by underwater cameras. The three main contributions of the paper are: (1) a novel hierarchical classification method suited for greatly unbalanced classes, (2) a novel classification-rejection method to clear up decisions and reject unknown classes, (3) an application of the classification method to free swimming fish. This system assists ecological surveillance research, e.g. fish population statistics in the open sea. BEOTR is automatically constructed based on inter-class similarities. Afterwards, trajectory voting is used to eliminate accumulated errors during hierarchical classification and, therefore, achieves better performance. We apply a Gaussian mixture model and Bayes rule as a reject option after the hierarchical classification to evaluate the posterior probability of being a certain species to filter less confident decisions. The proposed BEOTR-based hierarchical classification method achieves significant improvements compared to state-of-the-art techniques on a live fish image dataset of 24,150 manually labelled images from South Taiwan Sea.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Lee, D., Schoenberger, R.B., Shiozawa, D., Xu, X.Q., Zhan, P.C.: Contour matching for a fish recognition and migration-monitoring system. Proc. SPIE 5606(1), 37–48 (2004)

    Article  Google Scholar 

  2. Ruff, B.P., Marchant, J.A., Frost, A.R.: Fish sizing and monitoring using a stereo image analysis system applied to fish farming. Aquac. Eng. 14(2), 155–173 (1995)

    Article  Google Scholar 

  3. Strachan, N.J.C., Nesvadba, P., Allen, A.R.: Fish species recognition by shape analysis of images. Pattern Recognit. 23(5), 539–544 (1990)

    Article  Google Scholar 

  4. Okamoto, M., Morita, S., Sato, T.: Fundamental study to estimate fish biomass around coral reef using 3-dimensional underwater video system. In: Proceedings of the OCEANS 2000 MTS/IEEE Conference and Exhibition, vol. 2, pp. 1389–1392 (2000)

  5. Walther, D., Edgington, D.R., Koch, C.: Detection and tracking of objects in underwater video. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 544–549 (2004)

  6. Rova, A., Mori, G., Dill, L.M.: One fish, two fish, butterfish, trumpeter: recognizing fish in underwater video. In: IAPR Conference on Machine Vision Applications, pp. 404–407 (2007)

  7. Zion, B., Shklyar, A., Karplus, I.: In-vivo fish sorting by computer vision. Aquac. Eng. 22, 165–179 (June 2000)

  8. Heithaus, M.R., Dill, L.M.: Food availability and tiger shark predation risk influence bottlenose dolphin habitat use. Ecology 83(2), 480–491 (2002)

    Article  Google Scholar 

  9. Strachan, N.J.C.: Length measurement of fish by computer vision. Comput. Electron. Agric. 8(2), 93–104 (1993)

    Article  Google Scholar 

  10. Toh, Y.H., Ng, T.M., Liew, B.K.: Automated fish counting using image processing. In: Proceedings of the International Conference on Computational Intelligence and Software Engineering, pp. 1–5 (2009)

  11. Schettini, R., Corchs, S.: Underwater image processing: state of the art of restoration and image enhancement methods. EURASIP J. Adv. Signal Process 2010, 1–7 (Jan. 2010)

  12. Strachan, N.J.C.: Recognition of fish species by colour and shape. Image Vis. Comput. 11, 2–10 (Jan. 1993)

  13. Spampinato, C., Giordano, D., Salvo, R.D., Chen-Burger, Y.H., Fisher, R.B., Nadarajan, G.: Automatic fish classification for underwater species behavior understanding. In: Proceedings of the First ACM International Workshop on Analysis And Retrieval of Tracked Events and Motion in Imagery Streams, pp. 45–50 (2010)

  14. Larsen, R., Ólafsdóttir, H., Ersbøll, B.: Shape and texture based classification of fish species. In: Proceedings of SCIA, pp. 745–749 (2009)

  15. Caley, M.J., Carr, M.H., Hixon, M.A., Hughes, T.P., Jones, G.P., Menge, B.A.: Recruitment and the local dynamics of open marine populations. Annu. Rev. Ecol. Syst. 27, 477–500 (Jan. 1996)

  16. Brehmer, P., Chi, T.D., Mouillot, D.: Amphidromous fish school migration revealed by combining fixed sonar monitoring (horizontal beaming) with fishing data. J. Exp. Mar. Biol. Ecol. 334, 139–150 (June 2006)

  17. Nadarajan, G., Chen-Burger, Y., Fisher, R., Spampinato, C.: A flexible system for automated composition of intelligent video analysis. In: Proceedings of ISPA, pp. 259–264 (2011)

  18. Chih-Chung, C., Chih-Jen, L.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2(3), 1–27 (2011)

    Article  Google Scholar 

  19. Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20, 273–297 (Sept. 1995)

  20. Duan, K.-B., Keerthi, S.S.: Which is the best multiclass SVM method? An empirical study. In: Proceedings of the 6th International Conference on Multiple Classifier Systems (MCS’05), pp. 278–285. Springer, New York (2005)

  21. Carlos, S., Alex, F.: A survey of hierarchical classification across different application domains. Data Min. Knowl. Discov. 22(1–2), 31–72 (2010)

    Google Scholar 

  22. Deng, J., Berg, A., Li, K., Fei-Fei, L.: What does classifying more than 10,000 image categories tell us? ECCV 6315, 71–84 (2010)

    Google Scholar 

  23. Gordon, A.D.: A review of hierarchical classification. J. R. Stat. Soc. 150(2), 119–137 (1987)

    MATH  Google Scholar 

  24. Mathis, C.: Classification using a hierarchical bayesian approach. In: Proceedings of ICPR, vol. 4, pp. 103–106 (2002)

  25. Deng, J., Dong, W., Socher, R., Li, L., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: Proceedings of CVPR, pp. 248–255 (2009)

  26. Platt, J.C.: Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In: Advances in Large Margin Classifiers, pp. 61–74. MIT Press, USA (1999)

  27. Wang, Y.-C.F., Casasent, D.: A support vector hierarchical method for multi-class classification and rejection. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN’09), pp. 3281–3288 (2009)

  28. Rother, C., Kolmogorov, V., Blake, A.: GrabCut: interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. (SIGGRAPH), 23, 309–314 (2004)

  29. Mokhtarian, F., Suomela, R.: Robust image corner detection through curvature scale space. IEEE Trans. PAMI 20(12), 1376–1381 (1998)

    Article  Google Scholar 

  30. He, X.C., Yung, N.H.C.: Curvature scale space corner detector with adaptive threshold and dynamic region of support. In: Proceedings of the International Conference on Pattern Recognition, vol. 2, pp. 791–794. IEEE Computer Society (2004)

  31. Saeys, Y., Inza, In, Larrañaga, P.: A review of feature selection techniques in bioinformatics. Bioinformatics 23(19), 2507–2517 (2007)

    Article  Google Scholar 

  32. Flusser, J., Zitova, B., Suk, T.: Moments and moment invariants in pattern recognition. Wiley (2009)

  33. Flusser, J., Suk, T., Zitova, B.: Affine moment invariants. In: Moments and Moment Invariants in Pattern Recognition, p. 49C112, Wiley, New York (2009)

  34. Bosch, A., Zisserman, A., Munoz, X.: Representing shape with a spatial pyramid kernel. In: Proceedings of the 6th ACM International Conference on Image and Video Retrieval (CIVR’07), p. 401C408. ACM, New York (2007)

  35. Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press Inc, Oxford (1995)

    Google Scholar 

  36. Mckenna, S.J., Gong, S., Raja, Y.: Modelling facial colour and identity with Gaussian mixtures. Pattern Recognit. 31, 1883–1892 (1998)

    Article  Google Scholar 

  37. Shental, N., Bar-hillel, A., Hertz, T., Weinshall, D.: Computing Gaussian mixture models with EM using equivalence constraints. In: Advances in Neural Information Processing Systems, vol. 16, MIT Press, USA (2003)

  38. Figueiredo, M.A.T., Jain, A.: Unsupervised learning of finite mixture models. IEEE Trans. Pattern Anal. Mach. Intell. 24(3), 381–396 (2002)

    Article  Google Scholar 

  39. Zhao, Y., Karypis, G., Du, D.-Z.: Criterion functions for document clustering. University of Minnesota (2005)

  40. Rissanen, J.: Stochastic complexity and modeling. Ann. Stat. 14, 1080–1100 (1986)

  41. Boom, B., Huang, P., He, J., Fisher, R.: Supporting ground-truth annotation of image datasets using clustering. In: Proceedings of the 21st International Conference on Pattern Recognition (ICPR), pp. 1542–1545 (2012)

  42. Hastie, T., Tibshirani, R., Friedman, J.J.H.: The Elements of Statistical Learning, vol. 1. Springer, New York (2001)

    Book  MATH  Google Scholar 

  43. Chib, S.: Marginal likelihood from the Gibbs output. J. Am. Stat. Assoc. 90(432), 1313–1321 (1995)

    Article  MATH  MathSciNet  Google Scholar 

  44. Huang, P.X., Boom, B.J., Fisher, R.B.: Underwater live fish recognition using a balance-guaranteed optimized tree. In: Computer Vision-ACCV 2012, pp. 422–433 (2013)

Download references

Author information

Authors and Affiliations

  1. Room G.12, 10 Crichton street, Edinburgh, EH8 9AB, UK

    Phoenix X. Huang & Bastiaan J. Boom

  2. Room 1.26, 10 Crichton street, Edinburgh, EH8 9AB, UK

    Robert B. Fisher

Authors
  1. Phoenix X. Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bastiaan J. Boom
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Robert B. Fisher
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Phoenix X. Huang.

Electronic supplementary material

Below is the link to the electronic supplementary material.

ESM 1 (PDF 244 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, P.X., Boom, B.J. & Fisher, R.B. Hierarchical classification with reject option for live fish recognition. Machine Vision and Applications 26, 89–102 (2015). https://doi.org/10.1007/s00138-014-0641-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-014-0641-2

Keywords

Search

Navigation