Skip to main content
SpringerLink
Log in

Why over-parameterization of deep neural networks does not overfit?

  • Perspective
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

References

  1. Neyshabur B, Tomioka R, Srebro N. Norm-based capacity control in neural networks. In: Proceedings of the 28th Conference on Learing Theory, Paris, 2015. 1376–1401

  2. Zhang C Y, Bengio S, Hardt M, et al. Understanding deep learning requires rethinking generalization. In: Proceedings of the 5th International Conference on Learning Representation, Toulon, 2017

  3. Nagarajan V, Kolter J Z. Uniform convergence may be unable to explain generalization in deep learning. In: Proceedins of Advances in Neural Information Processing Systems, 2019. 11615–11626

  4. Lawrence S, Giles C L, Tsoi A C. Lessons in neural network training: overfitting may be harder than expected. In: Proceedings of the 14th National Conference on Artificial Intelligence, Providence, 1997. 540–545

  5. Liu Y Y, Starzyk J A, Zhu Z. Optimized approximation algorithm in neural networks without overfitting. IEEE Trans Neural Netw, 2008, 19: 983–995

    Article  Google Scholar 

  6. Kulis B. Metric learning: a survey. Found Trends Mach Learn, 2013, 5: 287–363

    Article  Google Scholar 

  7. Davis J V, Kulis B, Jain P, et al. Information-theoretic metric learning. In: Proceedings of the 24th International Conference on Machine Learning, Corvalis, 2007. 209–216

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China (NSFC) (Grant Nos. 61751306, 61921006). The author wants to thank Shen-Huan LYU and Zhi-Hao TAN for discussion and help in figures.

Author information

Authors and Affiliations

  1. National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, 210023, China

    Zhi-Hua Zhou

Authors
  1. Zhi-Hua Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhi-Hua Zhou.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, ZH. Why over-parameterization of deep neural networks does not overfit?. Sci. China Inf. Sci. 64, 116101 (2021). https://doi.org/10.1007/s11432-020-2885-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11432-020-2885-6

Search

Navigation