Skip to main content
SpringerLink
Log in

A Nearest Neighbor Classifier-Based Automated On-Line Novel Visual Percept Detection Method

  • Chapter
  • First Online:
New Developments in Unsupervised Outlier Detection

Abstract

The k-nearest neighbor classifier is a very effective and simple nonparametric technique in pattern classification. However, it only can classify new data items into existing categories, but not the data items coming from any new categories that have not been identified beforehand. Also, its classification performance is highly influenced by the efficiency of the k-nearest neighbors search structure employed. In this chapter, we present a fast approximate nearest neighbor search tree-based novelty filter for the multiple percept detection and incremental learning tasks in image sequences. We begin with an introduction to the concept of novelty detection in general and an image patch-based perceptual learning system as a basis for visual novelty detection in specific. The proposed on-line novel visual percept detection method is next presented. Finally, the performance of the proposed filter is compared with that of the well-known Grow-When-Required neural network approach for a novelty detection task in an indoor environment and with that of efficient support vector data description method for a novelty detection task in an outdoor environment.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Cover, T., & Hart, P. (1967). Nearest neighbor pattern classification. IEEE Transactions on Information Theory, 13(1), 21–27.

    Article  MATH  Google Scholar 

  2. Liu, Z. G., Pan, Q., & Dezert, J. (2013). A new belief-based k-nearest neighbor classification method. Pattern Recognition, 46, 834–844.

    Article  Google Scholar 

  3. Mitani, Y., & Hamamoto, Y. (2006). A local mean-based nonparametric classifier. Pattern Recognition Letters, 27(10), 1151–1159.

    Article  Google Scholar 

  4. Gou, J., Yi, Z., Du, L., & Xiong, T. (2011). A local mean-based k-nearest centroid neighbor classifier. Computer Journal, 55(9), 1058–1071.

    Article  Google Scholar 

  5. Sánchez, J. S., Pla, F., & Ferri, F. J. (1997). On the use of neighbourhood-based non-parametric classifiers. Pattern Recognition Letters, 18(11–13), 1179–1186.

    Article  Google Scholar 

  6. Samsudin, N. A., & Bradley, A. P. (2010). Nearest neighbour group-based classification. Pattern Recognition, 43(10), 3458–3467.

    Article  MATH  Google Scholar 

  7. Shanableh, T., Assaleh, K., & Al-Rousan, M. (2007). Spatio-temporal feature-extraction techniques for isolated gesture recognition in Arabic sign language. IEEE Transactions on Systems, Man, and Cybernetics. Part B, Cybernetics, 37(3), 641–650.

    Article  Google Scholar 

  8. Xu, J., Yang, J., & Lai, Z. (2013). K-local hyperplane distance nearest neighbor classifier oriented local discriminant analysis. Information Sciences, 232, 11–26.

    Article  MathSciNet  MATH  Google Scholar 

  9. Maji, P. (2011). Fuzzy–rough supervised attribute clustering algorithm and classification of microarray data. IEEE Transactions on Systems, Man, and Cybernetics. Part B, Cybernetics, 41(1), 222–233.

    Article  Google Scholar 

  10. Raymer, M. L., Doom, T. E., Kuhn, L. A., & Punch, W. F. (2003). Knowledge discovery in medical and biological datasets using a hybrid Bayes classifier/evolutionary algorithm. IEEE Transactions on Systems, Man, and Cybernetics. Part B, Cybernetics, 33(5), 802–813.

    Article  Google Scholar 

  11. Dudczyk, J., Kawalec, A., & Owczarek, R. (2008). An application of iterated function system attractor for specific radar source identification. In Proceedings of the 2008 17th International Conference on Microwaves, Radar and Wireless Communications (MIKON’08), Wroclaw, Poland.

    Google Scholar 

  12. Dudczyk, J., Kawalec, A., & Cyrek, J. (2008). Applying the distance and similarity functions to radar signals identification. In Proceedings of the 2008 International Radar Symposium (IRS’08), Wroclaw, Poland.

    Google Scholar 

  13. Dudczyk, J., & Wnuk, M. (2004). The utilization of unintentional radiation for identification of the radiation sources. In Proceedings of the 34th European Microwave Conference, Amsterdam, The Netherlands (Vol. 2, Issue 2, pp. 777–780).

    Google Scholar 

  14. Dudczyk, J. (2017). A method of feature selection in the aspect of specific identification of radar signals. Bulletin of the Polish Academy of Sciences: Technical Sciences, 65(1), 113–119.

    Article  Google Scholar 

  15. Mensink, T., Verbeek, J., Perronnin, F., & Csurka, G. (2013). Distance-based image classification: Generalizing to new classes at near-zero cost. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(11), 2624–2637.

    Article  Google Scholar 

  16. Frigui, H., & Gader, P. (2009). Detection and discrimination of land mines in ground-penetrating radar based on edge histogram descriptors and a possibilistic k-nearest neighbor classifier. IEEE Transactions on Fuzzy Systems, 17(1), 185–199.

    Article  Google Scholar 

  17. Ma, L., Crawford, M. M., & Tian, J. (2010). Local manifold learning-based k-nearest-neighbor for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 48(11), 4099–4109.

    Google Scholar 

  18. Manavalan, B., Shin, T. H., & Lee, G. (2018). PVP-SVM: Sequence-based prediction of phage virion proteins using a support vector machine. Frontiers in Microbiology, 9, 1–10.

    Article  Google Scholar 

  19. Manavalan, B., Subramaniyam, S., Shin, T. H., & Kim, M. O. (2018). Machine-learning-based prediction of cell-penetrating peptides and their uptake efficiency with improved accuracy. Journal of Proteome Research, 17(8), 2715–2726.

    Article  Google Scholar 

  20. Manavalan, B., Shin, T. H., & Lee, G. (2018). DHSpred: Support-vector-machine-based human DNase I hypersensitive sites prediction using the optimal features selected by random forest. Oncotarget, 9(2), 1944–1956.

    Article  Google Scholar 

  21. Manavalan, B., Shin, T. H., Kim, M. O., & Lee, G. (2018). AIPpred: Sequence-based prediction of anti-inflammatory peptides using random forest. Frontiers in Pharmacology, 9, 1–12.

    Article  Google Scholar 

  22. L. Tarassenko, P. Hayton, N. Cerneaz, M. Brady. Novelty detection for the identification of masses in mammograms. In Proceedings of the 4th International Conference on Artificial Neural Networks, Cambridge, UK (pp. 442–447).

    Google Scholar 

  23. Quinn, J., & Williams, C. (2007). Known unknowns: Novelty detection in condition monitoring. In Proceedings of the 3rd Iberian Conference on Pattern Recognition and Image Analysis (IbPRIA’07), Girona, Spain, LNCS (Vol. 4477, Isusue 1, pp. 1–6).

    Google Scholar 

  24. Clifton, L., Clifton, D., Watkinson, P., & Tarassenko, L. (2011). Identification of patient deterioration in vital-sign data using one-class support vector machines. In Proceedings of the 2001 Federated Conference on Computer Science and Information Systems (FedCSIS’11), Szczecin, Poland (pp. 125–131).

    Google Scholar 

  25. Tarassenko, L., Clifton, D., Bannister, P., King, S., & King, D. (2009). Novelty detection. Wiley.

    Google Scholar 

  26. Surace, C., & Worden, K. (2010). Novelty detection in a changing environment: A negative selection approach. Mechanical Systems and Signal Processing, 24(4), 1114–1128.

    Article  Google Scholar 

  27. Patcha, A., & Park, J. (2007). An overview of anomaly detection techniques: Existing solutions and latest technological trends. Computer Networks, 51(12), 3448–3470.

    Article  Google Scholar 

  28. Jyothsna, V., Prasad, V. V. R., & Prasad, K. M. (2011). A review of anomaly based intrusion detection systems. International Journal of Computer Applications, 28(7), 26–35.

    Article  Google Scholar 

  29. Diehl, C., & Hampshire, J. (2002). Real-time object classification and novelty detection for collaborative video surveillance. In Proceedings of the International Joint Conference on Neural Networks (IJCNN’02) (Vol. 3, Issue 3, pp. 2620–2625).

    Google Scholar 

  30. Markou, M., & Singh, S. (2006). A neural network-based novelty detector for image sequence analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(10), 1664–1677.

    Article  Google Scholar 

  31. Vieira Neto, H., & Nehmzow, U. (2007). Real-time automated visual inspection using mobile robots. Journal of Intelligent and Robotic Systems, 49(3), 293–307.

    Article  Google Scholar 

  32. Sofman, B., Neuman, B., Stentz, A., & Bagnell, J. (2011). Anytime online novelty and change detection for mobile robots. Journal of Field Robotics, 28(4), 589–618.

    Article  MATH  Google Scholar 

  33. Zhang, Y., Meratnia, N., & Havinga, P. (2010). Outlier detection techniques for wireless sensor networks: A survey. IEEE Communications Surveys & Tutorials, 12(2), 159–170.

    Article  Google Scholar 

  34. Dutta, H., Giannella, C., Borne, K., & Kargupta, H. (2007). Distributed top-k outlier detection from astronomy catalogs using the DEMAC system. In Proceedings of the 7th SIAM International Conference on Data Mining, Minneapolis, Minnesota, USA (pp. 473–478).

    Google Scholar 

  35. Escalante, H. (2005). A comparison of outlier detection algorithms for machine learning. In Proceedings of the International Conference on Communications in Computing (pp. 10–15).

    Google Scholar 

  36. Basu, S., Bilenko, M., & Mooney, R. (2004). A probabilistic framework for semi- supervised clustering. In Proceedings of the 10th ACM International Conference on Knowledge Discovery and Data Mining (SIGKDD’04), Seattle, WA, USA (pp. 59–68).

    Google Scholar 

  37. Markou, M., & Singh, S. (2003). Novelty detection: A review-part 1: Statistical approaches. Signal Processing, 83(12), 2481–2497.

    Article  MATH  Google Scholar 

  38. Markou, M., & Singh, S. (2003). Novelty detection: A review-part 2: Neural network based approaches. Signal Processing, 83(12), 2499–2521.

    Article  MATH  Google Scholar 

  39. Duda, R. O., Hart, P. E., & Stork, D. G. (2001). Pattern classification. NY, USA: Wiley.

    MATH  Google Scholar 

  40. Bishop, C. (2006). Pattern recognition and machine learning. New York: Springer.

    MATH  Google Scholar 

  41. Marsland, S., Nehmzow, U., & Shapiro, J. (2001). Vision-based environmental novelty detection on a mobile robot. In Proceedings of the 8th International Conference on Neural Information Processing (ICONIP’01), Shanghai, China (Vol. 1–3, pp. 69–74).

    Google Scholar 

  42. Marsland, S., Nehmzow, U., & Shapiro, J. (2002). Environment-specific novelty detection. In From Animals to Animates: Proceedings of the 7th International Conference on the Simulation of Adaptive Behavior (SAB’02), MIT, Edinburgh, UK (pp. 36–45).

    Google Scholar 

  43. Neto, H. V., & Nehmzow, U. (2004). Visual novelty detection for inspection tasks using mobile robots. In Proceedings of the 8th Brazilian Symposium on Neural Networks (SBRN’04), Sao Luis, Brazil.

    Google Scholar 

  44. Nehmzow, U., & Neto, H. V. (2004). Novelty-based visual inspection using mobile robots. In Towards Autonomous Robotic Systems: Proceedings of the 5th British Conference on Mobile Robotics (TAROS’04), Colchester, UK.

    Google Scholar 

  45. Neto, H. V., & Nehmzow, U. (2007). Visual novelty detection with automatic scale selection. Robotics and Autonomous Systems, 55(9), 693–701.

    Article  Google Scholar 

  46. Tax, D. M. J., & Duin, R. P. W. (2004). Support vector data description. Machine Learning, 54(1), 45–66.

    Article  MATH  Google Scholar 

  47. Liu, Y.-H., Liu, Y.-C., & Chen, Y.-J. (2010). Fast support vector data descriptions for novelty detection. IEEE Transactions on Neural Networks, 21(8), 1296–1313.

    Article  Google Scholar 

  48. Peng, X., & Xu, D. (2010). Efficient support vector data descriptions for novelty detection. Neural Computing and Applications, 21(8), 2023–2032.

    Article  Google Scholar 

  49. Nister, D., & Stewenius, H. (2006). Scalable recognition with a vocabulary tree. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR’06), New York, NY, United states (Vol. 2, pp. 2161–2168).

    Google Scholar 

  50. Meeds, E. (2005). Novelty detection model selection using volume estimation. UTML-TR-2005–004, Technical Report, University of Toronto.

    Google Scholar 

  51. Olshausen, B. A., & Field, D. J. (1996). Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 381(6583), 607–609.

    Article  Google Scholar 

  52. Goldstone, R. (1998). Perceptual learning. Annual Review of Psychology, 49, 585–612.

    Article  Google Scholar 

  53. Buhmann, J. M., Lange, T., & Ramacher, U. (2005). Image segmentation by networks of spiking neurons. Neural Computation, 17(5), 1010–1031.

    Article  MATH  Google Scholar 

  54. Tugcu, M., Wang, X., Hunter, J. E., Phillips, J., Noelle, D., & Wilkes, D. M. (2007). A computational neuroscience model of working memory with application to robot perceptual learning. In Proceedings of the 3rd IASTED International Conference on Computational Intelligence (CI’07), Banff, AB, Canada (pp. 120–127).

    Google Scholar 

  55. Wang, X., Tugcu, M., Hunter, J. E., & Wilkes, D. M. (2009). Exploration of configural representation in landmark learning using working memory toolkit. Pattern Recognition Letters, 30(1), 66–79.

    Article  Google Scholar 

  56. Swain, M. J., & Ballard, D. (1991). Color indexing. International Journal of Computer Vision, 7(1), 11–32.

    Article  Google Scholar 

  57. Hunter, J. E. (2005). Human motion segmentation and object recognition using Fuzzy rules. In Proceedings of 14th Annual IEEE International Workshop on Robot and Human Interactive Communication (RO-MAN’05), Nashville, TN (pp. 210–216).

    Google Scholar 

  58. Gabor, D. (1946). Theory of communications. Journal of Institute of Electrical Engineering, 93(3), 429–457.

    Google Scholar 

  59. Hunter, J. E., Tugcu, M., Wang, X., Costello, C., & Wilkes, D. M. (2011). Exploiting sparse representations in very high-dimensional feature spaces obtained from patch-based processing. Machine Vision and Applications, 22(3), 449–460.

    Google Scholar 

  60. Shepard, R. N. (1964). Attention and the metric structure of the stimulus space. Journal of Mathematical Psychology, 1(1), 54–87.

    Article  Google Scholar 

  61. Garner, W. R. (1974). The processing of information and structure. New York, USA: Wiley.

    Google Scholar 

  62. Shepard, R. N. (1987). Toward a universal law of generalization for psychological science. Science, 237(4820), 1317–1323.

    Article  MathSciNet  MATH  Google Scholar 

  63. Wang, X., Wang, X. L., & Wilkes, D. M. (2009). A divide-and-conquer approach for minimum spanning tree-based clustering. IEEE Transactions on Knowledge and Data Engineering, 21(7), 945–958.

    Article  Google Scholar 

  64. Lowe, D. (2004). Distinctive image features from scale invariant keypoints. International Journal of Computer Vision, 60(2), 91–110.

    Article  Google Scholar 

  65. Wang, X., Wang, X. L., & Wilkes, D. M. (2012). An automated vision based on-line novel percept detection method for mobile robot. Robotics and Autonomous Systems, 60(10), 1279–1294.

    Article  Google Scholar 

Download references

Acknowledgements

This chapter was modified from the paper published by our group in Robotics and Autonomous Systems [65]. The related contents are reused with permission.

Author information

Authors and Affiliations

  1. School of Software Engineering, Xi’an Jiaotong University, Xi’an, Shaanxi, China

    Xiaochun Wang

  2. School of Information Engineering, Chang’an University, Xi’an, Shaanxi, China

    Xiali Wang

  3. Department of Electrical Engineering and Computer Science, Vanderbilt University, Nashville, TN, USA

    Mitch Wilkes

Authors
  1. Xiaochun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiali Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mitch Wilkes
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaochun Wang .

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Xi'an Jiaotong University Press

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Wang, X., Wang, X., Wilkes, M. (2021). A Nearest Neighbor Classifier-Based Automated On-Line Novel Visual Percept Detection Method. In: New Developments in Unsupervised Outlier Detection. Springer, Singapore. https://doi.org/10.1007/978-981-15-9519-6_9

Download citation

Publish with us

Policies and ethics

Search

Navigation