NOD-CC: A Hybrid CBR-CNN Architecture for Novel Object Discovery

  • J. T. TurnerEmail author
  • Michael W. FloydEmail author
  • Kalyan GuptaEmail author
  • Tim OatesEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11680)


Deep Learning methods have shown a rapid increase in popularity due to their state-of-the-art performance on many machine learning tasks. However, these methods often rely on extremely large datasets to accurately train the underlying machine learning models. For supervised learning techniques, the human effort required to acquire, encode, and label a sufficiently large dataset may add such a high cost that deploying the algorithms is infeasible. Even if a sufficient workforce exists to create such a dataset, the human annotators may differ in the quality, consistency, and level of granularity of their labels. Any impact this has on the overall dataset quality will ultimately impact the potential performance of an algorithm trained on it. This paper partially addresses this issue by providing an approach, called NOD-CC, for discovering novel object types in images using a combination of Convolutional Neural Networks (CNNs) and Case-Based Reasoning (CBR). The CNN component labels instances of known object types while deferring to the CBR component to identify and label novel, or poorly understood, object types. Thus, our approach leverages the state-of-the-art performance of CNNs in situations where sufficient high-quality training data exists, while minimizing its limitations in data-poor situations. We empirically evaluate our approach on a popular computer vision dataset and show significant improvements to object classification performance when full knowledge of potential class labels is not known in advance.


Deep learning Novel object discovery Computer vision Convolutional Neural Networks 


  1. 1.
    Begum, S., Ahmed, M.U., Funk, P., Xiong, N., Folke, M.: Case-based reasoning systems in the health sciences: a survey of recent trends and developments. IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.) 41(4), 421–434 (2011)CrossRefGoogle Scholar
  2. 2.
    Chen, X., Mottaghi, R., Liu, X., Fidler, S., Urtasun, R., Yuille, A.: Detect what you can: detecting and representing objects using holistic models and body parts. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1971–1978 (2014)Google Scholar
  3. 3.
    Everingham, M., Van Gool, L.J., Williams, C.K.I., Winn, J.M., Zisserman, A.: The Pascal Visual Object Classes (VOC) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010)CrossRefGoogle Scholar
  4. 4.
    Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 580–587. IEEE Computer Society (2014)Google Scholar
  5. 5.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
  6. 6.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Proceedings of the Conference on Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  7. 7.
    Kumar, A., Kim, J., Cai, W., Fulham, M., Feng, D.: Content-based medical image retrieval: a survey of applications to multidimensional and multimodality data. J. Digit. Imaging 26(6), 1025–1039 (2013)CrossRefGoogle Scholar
  8. 8.
    Kurth, T., et al.: Exascale deep learning for climate analytics. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis, pp. 51:1–51:12. IEEE Press (2018)Google Scholar
  9. 9.
    Kuznetsova, A., et al.: The open images dataset V4: unified image classification, object detection, and visual relationship detection at scale (2018). arXiv preprint arXiv:1811.00982
  10. 10.
    Lenz, M., Bartsch-Spörl, B., Burkhard, H.D., Wess, S.: Case-Based Reasoning Technology: From Foundations to Applications. Springer, Heidelberg (1998). Scholar
  11. 11.
    López-Sánchez, D., Corchado, J.M., González Arrieta, A.: A CBR system for efficient face recognition under partial occlusion. In: Aha, D.W., Lieber, J. (eds.) ICCBR 2017. LNCS (LNAI), vol. 10339, pp. 170–184. Springer, Cham (2017). Scholar
  12. 12.
    Macura, R.T., Macura, K.J.: MacRad: radiology image resource with a case-based retrieval system. In: Veloso, M., Aamodt, A. (eds.) ICCBR 1995. LNCS, vol. 1010, pp. 43–54. Springer, Heidelberg (1995). Scholar
  13. 13.
    Micarelli, A., Neri, A., Sansonetti, G.: A case-based approach to image recognition. In: Blanzieri, E., Portinale, L. (eds.) EWCBR 2000. LNCS, vol. 1898, pp. 443–454. Springer, Heidelberg (2000). Scholar
  14. 14.
    Minsky, M., Papert, S.: Perceptrons: An Introduction to Computational Geometry. MIT Press, Cambridge (1969)zbMATHGoogle Scholar
  15. 15.
    Page, A., Turner, J., Mohsenin, T., Oates, T.: Comparing raw data and feature extraction for seizure detection with deep learning methods. In: Proceedings of the International Florida Artificial Intelligence Research Society Conference, pp. 284–287 (2014)Google Scholar
  16. 16.
    Patrini, G., Rozza, A., Menon, A., Nock, R., Qu, L.: Making deep neural networks robust to label noise: a loss correction approach. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2233–2241 (2017)Google Scholar
  17. 17.
    Perner, P., Holt, A., Richter, M.: Image processing in case-based reasoning. Knowl. Eng. Rev. 20(3), 311–314 (2005)CrossRefGoogle Scholar
  18. 18.
    Ravi, D., Wong, C., Lo, B., Yang, G.Z.: A deep learning approach to on-node sensor data analytics for mobile or wearable devices. IEEE J. Biomed. Health Inform. 21(1), 56–64 (2017)CrossRefGoogle Scholar
  19. 19.
    Sani, S., Massie, S., Wiratunga, N., Cooper, K.: Learning deep and shallow features for human activity recognition. In: Li, G., Ge, Y., Zhang, Z., Jin, Z., Blumenstein, M. (eds.) KSEM 2017. LNCS (LNAI), vol. 10412, pp. 469–482. Springer, Cham (2017). Scholar
  20. 20.
    Sani, S., Wiratunga, N., Massie, S.: Learning deep features for kNN-based human activity recognition. In: Proceedings of the ICCBR Workshops: Case-Based Reasoning and Deep Learning Workshop, pp. 95–103 (2017)Google Scholar
  21. 21.
    Sathyanarayana, A., et al.: Sleep quality prediction from wearable data using deep learning. JMIR Mhealth Uhealth 4(4), e125 (2016)CrossRefGoogle Scholar
  22. 22.
    Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818–2826 (2016)Google Scholar
  23. 23.
    Turner, J.T., Floyd, M.W., Gupta, K.M., Aha, D.W.: Novel object discovery using case-based reasoning and convolutional neural networks. In: Cox, M.T., Funk, P., Begum, S. (eds.) ICCBR 2018. LNCS (LNAI), vol. 11156, pp. 399–414. Springer, Cham (2018). Scholar
  24. 24.
    Turner, J., Page, A., Mohsenin, T., Oates, T.: Deep belief networks used on high resolution multichannel electroencephalography data for seizure detection. In: Proceedings of the AAAI Spring Symposium Series: Big Data Becomes Personal: Knowledge into Meaning (2014)Google Scholar
  25. 25.
    Xie, S., Girshick, R.B., Dollár, P., Tu, Z., He, K.: Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5987–5995 (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.University of Maryland Baltimore CountyBaltimoreUSA
  2. 2.Knexus Research CorporationNational HarborUSA

Personalised recommendations