Skip to main content
SpringerLink
Log in

\(N^4\)-Fields: Neural Network Nearest Neighbor Fields for Image Transforms

  • Conference paper
  • First Online:
Computer Vision -- ACCV 2014 (ACCV 2014)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 9004))

Included in the following conference series:

Abstract

We propose a new architecture for difficult image processing operations, such as natural edge detection or thin object segmentation. The architecture is based on a simple combination of convolutional neural networks with the nearest neighbor search.

We focus our attention on the situations when the desired image transformation is too hard for a neural network to learn explicitly. We show that in such situations the use of the nearest neighbor search on top of the network output allows to improve the results considerably and to account for the underfitting effect during the neural network training. The approach is validated on three challenging benchmarks, where the performance of the proposed architecture matches or exceeds the state-of-the-art.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://code.google.com/p/cuda-convnet/.

  2. 2.

    http://sites.skoltech.ru/compvision/projects/n4/ at the moment of publication.

References

  1. LeCun, Y., Boser, B.E., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W.E., Jackel, L.D.: Handwritten digit recognition with a back-propagation network. In: NIPS, pp. 396–404 (1989)

    Google Scholar 

  2. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: NIPS (2012)

    Google Scholar 

  3. Ciresan, D.C., Giusti, A., Gambardella, L.M., Schmidhuber, J.: Deep neural networks segment neuronal membranes in electron microscopy images. In: NIPS, pp. 2852–2860 (2012)

    Google Scholar 

  4. Arbeláez, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 898–916 (2011)

    Article  Google Scholar 

  5. Silberman, N., Fergus, R.: Indoor scene segmentation using a structured light sensor. In: ICCV Workshops, pp. 601–608. IEEE (2011)

    Google Scholar 

  6. Staal, J., Abrmoff, M.D., Niemeijer, M., Viergever, M.A., van Ginneken, B.: Ridge-based vessel segmentation in color images of the retina. IEEE Trans. Med. Imaging 23(4), 501–509 (2004)

    Article  Google Scholar 

  7. Egmont-Petersen, M., de Ridder, D., Handels, H.: Image processing with neural networks - a review. Pattern Recognit. 35(10), 2279–2301 (2002)

    Article  MATH  Google Scholar 

  8. Mnih, V., Hinton, G.E.: Learning to detect roads in high-resolution aerial images. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part VI. LNCS, vol. 6316, pp. 210–223. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  9. Schulz, H., Behnke, S.: Learning object-class segmentation with convolutional neural networks. In: ESANN, vol. 3 (2012)

    Google Scholar 

  10. Kivinen, J.J., Williams, C.K.I., Heess, N.: Visual boundary prediction: A deep neural prediction network and quality dissection. In: AISTATS, pp. 512–521 (2014)

    Google Scholar 

  11. Ranzato, M., Hinton, G.E.: Modeling pixel means and covariances using factorized third-order boltzmann machines. In: CVPR, pp. 2551–2558 (2010)

    Google Scholar 

  12. Farabet, C., Couprie, C., Najman, L., LeCun, Y.: Learning hierarchical features for scene labeling. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1915–1929 (2013)

    Article  Google Scholar 

  13. Jain, V., Murray, J.F., Roth, F., Turaga, S.C., Zhigulin, V.P., Briggman, K.L., Helmstaedter, M., Denk, W., Seung, H.S.: Supervised learning of image restoration with convolutional networks. In: ICCV, pp. 1–8 (2007)

    Google Scholar 

  14. Jain, V., Seung, H.S.: Natural image denoising with convolutional networks. In: NIPS, pp. 769–776 (2008)

    Google Scholar 

  15. Burger, H.C., Schuler, C.J., Harmeling, S.: Image denoising: can plain neural networks compete with bm3d? In: CVPR, pp. 2392–2399 (2012)

    Google Scholar 

  16. Chopra, S., Hadsell, R., LeCun, Y.: Learning a similarity metric discriminatively, with application to face verification.In: CVPR, vol. 1, pp. 539–546 (2005)

    Google Scholar 

  17. Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image restoration by sparse 3d transform-domain collaborative filtering. In: Electronic Imaging 2008, International Society for Optics and Photonics, pp. 681207–681207 (2008)

    Google Scholar 

  18. Criminisi, A., Pérez, P., Toyama, K.: Region filling and object removal by exemplar-based image inpainting. IEEE Trans. Image Process. 13(9), 1200–1212 (2004)

    Article  Google Scholar 

  19. Freeman, W.T., Pasztor, E.C., Carmichael, O.T.: Learning low-level vision. Int. J. Comput. Vis. 40(1), 25–47 (2000)

    Article  MATH  Google Scholar 

  20. Dollár, P., Zitnick, C.L.: Structured forests for fast edge detection. In: ICCV (2013)

    Google Scholar 

  21. Hinton, G.E., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Improving neural networks by preventing co-adaptation of feature detectors. CoRR abs/1207.0580 (2012)

    Google Scholar 

  22. Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. CoRR abs/1311.2901 (2013)

    Google Scholar 

  23. Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., LeCun, Y.: Overfeat: Integrated recognition, localization and detection using convolutional networks. CoRR abs/1312.6229 (2013)

    Google Scholar 

  24. Vedaldi, A., Fulkerson, B.: VLFeat: An open and portable library of computer vision algorithms (2008). http://www.vlfeat.org/

  25. Xiaofeng, R., Bo, L.: Discriminatively trained sparse code gradients for contour detection. In: NIPS, pp. 593–601 (2012)

    Google Scholar 

  26. Isola, P., Zoran, D., Krishnan, D., Adelson, E.H.: Crisp boundary detection using pointwise mutual information. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part III. LNCS, vol. 8691, pp. 799–814. Springer, Heidelberg (2014)

    Chapter  Google Scholar 

  27. Hou, X., Yuille, A., Koch, C.: Boundary detection benchmarking: Beyond f-measures. In: CVPR, vol. 2013, pp. 1–8. IEEE (2013)

    Google Scholar 

  28. Becker, C., Rigamonti, R., Lepetit, V., Fua, P.: Supervised feature learning for curvilinear structure segmentation. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013, Part I. LNCS, vol. 8149, pp. 526–533. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Skolkovo Institute of Science and Technology (Skoltech), Moscow, Russia

    Yaroslav Ganin & Victor Lempitsky

Authors
  1. Yaroslav Ganin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Victor Lempitsky
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yaroslav Ganin .

Editor information

Editors and Affiliations

  1. Technische Universität München, Garching, Bayern, Germany

    Daniel Cremers

  2. University of Adelaide, Adelaide, South Australia, Australia

    Ian Reid

  3. Keio University, Yokohama, Kanagawa, Japan

    Hideo Saito

  4. University of California at Merced, Merced, California, USA

    Ming-Hsuan Yang

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Ganin, Y., Lempitsky, V. (2015). \(N^4\)-Fields: Neural Network Nearest Neighbor Fields for Image Transforms. In: Cremers, D., Reid, I., Saito, H., Yang, MH. (eds) Computer Vision -- ACCV 2014. ACCV 2014. Lecture Notes in Computer Science(), vol 9004. Springer, Cham. https://doi.org/10.1007/978-3-319-16808-1_36

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-16808-1_36

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-16807-4

  • Online ISBN: 978-3-319-16808-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation