Skip to main content
SpringerLink
Log in

Toward actionable testing of deep learning models

  • Perspective
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

References

  1. Wang Z, Yan M, Liu S, et al. Survey on testing deep learning neural networks (in Chinese). J Software, 2020, 31: 1255–1275

    Google Scholar 

  2. Huang X, Kroening D, Ruan W, et al. A survey of safety and trustworthiness of deep neural networks: Verification, testing, adversarial attack and defence, and interpretability. Comput Sci Rev, 2020, 37: 100270

    Article  MathSciNet  MATH  Google Scholar 

  3. Ma L, Juefei-Xu F, Zhang F, et al. DeepGauge: multigranularity testing criteria for deep learning systems. In: Proceedings of ACM/IEEE International Conference on Automated Software Engineering, 2018. 120–131

  4. Pei K, Cao Y, Yang J, et al. DeepXplore: automated white-box testing of deep learning systems. In: Proceedings of the ACM Symposium on Operating Systems Principles, 2017. 1–18

  5. Raghunathan A, Xie S M, Yang F, et al. Adversarial training can hurt generalization. In: Proceedings of ICML Deep Phenomena, 2019

  6. Zhang Y, Ren L, Chen L, et al. Detecting numerical bugs in neural network architectures. In: Proceedings of the ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 2020. 826–837

  7. Tian Y, Ma S, Wen M, et al. To what extent do DNN-based image classification models make unreliable inferences? Empir Software Eng, 2021, 26: 84

    Article  Google Scholar 

  8. Zhou Z Q, Sun L. Metamorphic testing for machine translations: MT4MT. In: Proceedings of Australian Software Engineering Conference, 2018. 96–100

  9. He P, Meister C, Su Z. Structure-invariant testing for machine translation. In: Proceedings of International Conference on Software Engineering, 2020. 961–973

  10. Gupta S, He P, Meister C, et al. Machine translation testing via pathological invariance. In: Proceedings of the ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 2020. 863–875

  11. He P, Meister C, Su Z. Testing machine translation via referential transparency. In: Proceedings of International Conference on Software Engineering, 2021. 410–422

  12. Sun Z, Zhang J M, Harman M, et al. Automatic testing and improvement of machine translation. In: Proceedings of International Conference on Software Engineering, 2020. 974–985

  13. Sun Z, Zhang J M, Xiong Y, et al. Improving machine translation systems via isotopic replacement. In: Proceedings of International Conference on Software Engineering, 2020. 1181–1192

Download references

Acknowledgements

This work was supported by National Key Research and Development Program of China (Grant No. 2019YFE0198100) and Innovation and Technology Commission of HKSAR (Grant No. MHP/055/19).

Author information

Authors and Affiliations

  1. Key Laboratory of High Confidence Software Technologies (Peking University), Ministry of Education, Beijing, 100871, China

    Yingfei Xiong

  2. School of Computer Science, Peking University, Beijing, 100871, China

    Yingfei Xiong

  3. Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong, 999077, China

    Yongqiang Tian & Shing-Chi Cheung

  4. School of Computer Science, University of Waterloo, Waterloo, ON, N2L 3G1, Canada

    Yongqiang Tian

  5. Research Institute of Trustworthy Autonomous Systems, Southern University of Science and Technology, Shenzhen, 518055, China

    Yepang Liu

  6. Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, 518055, China

    Yepang Liu

Authors
  1. Yingfei Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yongqiang Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yepang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shing-Chi Cheung
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Yingfei Xiong, Yongqiang Tian, Yepang Liu or Shing-Chi Cheung.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xiong, Y., Tian, Y., Liu, Y. et al. Toward actionable testing of deep learning models. Sci. China Inf. Sci. 66, 176101 (2023). https://doi.org/10.1007/s11432-022-3580-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11432-022-3580-5

Search

Navigation