Skip to main content
SpringerLink
Log in

STAMP 4 NLP – An Agile Framework for Rapid Quality-Driven NLP Applications Development

  • Conference paper
  • First Online:
Quality of Information and Communications Technology (QUATIC 2021)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1439))

Abstract

The progress in natural language processing (NLP) research over the last years, offers novel business opportunities for companies, as automated user interaction or improved data analysis. Building sophisticated NLP applications requires dealing with modern machine learning (ML) technologies, which impedes enterprises from establishing successful NLP projects. Our experience in applied NLP research projects shows that the continuous integration of research prototypes in production-like environments with quality assurance builds trust in the software and shows convenience and usefulness regarding the business goal. We introduce STAMP 4 NLP as an iterative and incremental process model for developing NLP applications. With STAMP 4 NLP, we merge software engineering principles with best practices from data science. Instantiating our process model allows efficiently creating prototypes by utilizing templates, conventions, and implementations, enabling developers and data scientists to focus on the business goals. Due to our iterative-incremental approach, businesses can deploy an enhanced version of the prototype to their software environment after every iteration, maximizing potential business value and trust early and avoiding the cost of successful yet never deployed experiments.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://github.com/philipp-kohl/stamp4nlp.

  2. 2.

    Similar to https://drivendata.github.io/cookiecutter-data-science/.

  3. 3.

    https://spacy.io/.

  4. 4.

    https://pytorch.org/.

References

  1. Adadi, A., Berrada, M.: Peeking inside the black-box: a survey on explainable artificial intelligence (XAI). IEEE Access 6, 52138–52160 (2018). https://doi.org/10.1109/ACCESS.2018.2870052

    Article  Google Scholar 

  2. Amershi, S., et al.: Software engineering for machine learning: a case study. In: Proceedings of the 41st International Conference on Software Engineering: Software Engineering in Practice, ICSE-SEIP 2019, pp. 291–300. IEEE Press (2019). https://doi.org/10.1109/ICSE-SEIP.2019.00042

  3. Baechle, M., Kirchberg, P.: Ruby on rails. Softw. IEEE 24, 105–108 (2007). https://doi.org/10.1109/MS.2007.176

    Article  Google Scholar 

  4. Chapman, P., et al.: CRISP-DM 1.0 step-by-step data mining guide. In: SPSS (2000)

    Google Scholar 

  5. Cohn, M.: Succeeding with Agile: Software Development Using Scrum. Pearson Education Limited (2009). ISBN 978-0-32166-056-5

    Google Scholar 

  6. Costello, K.: Gartner survey shows 37 percent of organizations have implemented AI in some form (2019). https://www.gartner.com/en/newsroom/press-releases/2019-01-21-gartner-survey-shows-37-percent-of-organizations-have. Accessed 24 Apr 2021

  7. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 1 (Long and Short Papers), pp. 4171–4186. Association for Computational Linguistics, Minneapolis (2019). https://doi.org/10.18653/v1/N19-1423. https://www.aclweb.org/anthology/N19-1423

  8. Duvall, P., Matyas, S.M., Glover, A.: Continuous Integration: Improving Software Quality and Reducing Risk. Addison-Wesley Professional, Boston (2007).ISBN 0-32133-638-0

    Google Scholar 

  9. Fayyad, U., Piatetsky-Shapiro, G., Smyth, P.: From data mining to knowledge discovery in databases. AI Mag. 17(3), 37 (1996)

    Google Scholar 

  10. Goasduff, L.: 3 barriers to AI adoption (2019). https://www.gartner.com/smarterwithgartner/3-barriers-to-ai-adoption/. Accessed 24 May 2021

  11. Jyoti, R., Shirer, M.: Idc survey finds artificial intelligence adoption being driven by improved customer experience, greater employee efficiency, and accelerated innovation (2020). https://www.idc.com/getdoc.jsp?containerId=prUS46534820. Accessed 24 May 2021

  12. Li, J., Sun, A., Han, J., Li, C.: A survey on deep learning for named entity recognition. IEEE Trans. Knowl. Data Eng, p. 1 (2020). https://doi.org/10.1109/TKDE.2020.2981314

  13. Liu, S.: Global natural language processing market 2017–2025 (2020). https://www.statista.com/statistics/607891/worldwide-natural-language-processing-market-revenues/. Accessed 24 May 2021

  14. Mane, D., Chitnis, K., Ojha, N.: The spring framework: an open source java platform for developing robust java applications. Int. J. Innov. Technol. Explor. Eng. (IJITEE) (2013)

    Google Scholar 

  15. Peters, M.E., et al.: Deep contextualized word representations. CoRR. abs/1802.05365 (2018). http://arxiv.org/abs/1802.05365

  16. Pineau, J., et al.: Improving reproducibility in machine learning research (a report from the neurips 2019 reproducibility program) (2020). arXiv:2003.12206. Version 3

  17. Reuters: Amazon ditched AI recruiting tool that favored men for technical jobs (2018). https://www.theguardian.com/technology/2018/oct/10/amazon-hiring-ai-gender-bias-recruiting-engine. Accessed 24 May 2021

  18. Ribeiro, M.T., Wu, T., Guestrin, C., Singh, S.: Beyond accuracy: behavioral testing of NLP models with CheckList. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4902–4912. Association for Computational Linguistics, Online (2020). https://doi.org/10.18653/v1/2020.acl-main.442. https://www.aclweb.org/anthology/2020.acl-main.442

  19. Schreiber, M., Kraft, B., Zündorf, A.: Metrics driven research collaboration: focusing on common project goals continuously. In: 2017 IEEE/ACM 4th International Workshop on Software Engineering Research and Industrial Practice (SER IP), pp. 41–47 (2017). https://doi.org/10.1109/SER-IP.2017.6

  20. Schreiber, M.: Towards effective natural language application development. Dissertation, University of Kassel (2019). https://doi.org/10.17170/kobra-20190529539

  21. Schreiber, M., Kraft, B., Zündorf, A.: Cost-efficient quality assurance of natural language processing tools through continuous monitoring with continuous integration. In: Proceedings of the 3rd International Workshop on Software Engineering Research and Industrial Practice, SER&IP 2016, pp. 46–52. Association for Computing Machinery, New York (2016). https://doi.org/10.1145/2897022.2897029

  22. Sildatke, M., Karwanni, H., Kraft, B., Schmidts, O., Zündorf, A.: Automated software quality monitoring in research collaboration projects. In: Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops, ICSEW’20, pp. 603–610. Association for Computing Machinery, New York (2020). https://doi.org/10.1145/3387940.3391478

  23. Staff, V.: Why do 87% of data science projects never make it into production? (2019). https://venturebeat.com/2019/07/19/why-do-87-of-data-science-projects-never-make-it-into-production/. Accessed 24 May 2021

  24. Studer, S., et al.: Towards CRISP-ML(Q): a machine learning process model with quality assurance methodology (2020). arXiv:2003.05155

  25. Varanasi, B., Belida, S.: Maven archetypes. In: Introducing Maven. Apress, Berkeley (2014). https://doi.org/10.1007/978-1-4842-0841-0_6

  26. Vijayan, V.K., Bindu, K.R., Parameswaran, L.: A comprehensive study of text classification algorithms. In: 2017 International Conference on Advances in Computing, Communications and Informatics (ICACCI), pp. 1109–1113 (2017). https://doi.org/10.1109/ICACCI.2017.8125990

  27. Vincent, J.: Twitter taught microsoft’s AI chatbot to be a racist asshole in less than a day (2016). https://www.theverge.com/2016/3/24/11297050/tay-microsoft-chatbot-racist. Accessed 24 May 2021

  28. Weber, C., Hirmer, P., Reimann, P., Schwarz, H.: A new process model for the comprehensive management of machine learning models. In: Proceedings of the 21st International Conference on Enterprise Information Systems, vol. 1: ICEIS, pp. 415–422. INSTICC, SciTePress (2019). https://doi.org/10.5220/0007725304150422

Download references

Author information

Authors and Affiliations

  1. FH Aachen - University of Applied Sciences, 52428, Jülich, Germany

    Philipp Kohl, Oliver Schmidts, Lars Klöser, Henri Werth & Bodo Kraft

  2. University of Kassel, 34127, Kassel, Germany

    Albert Zündorf

Authors
  1. Philipp Kohl
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Oliver Schmidts
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lars Klöser
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Henri Werth
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bodo Kraft
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Albert Zündorf
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Philipp Kohl .

Editor information

Editors and Affiliations

  1. Faculty of Engineering of the University of Porto, Porto, Portugal

    Ana C. R. Paiva

  2. Institut Polytechnique de Paris, Paris, France

    Ana Rosa Cavalli

  3. University of Algarve, Faro, Portugal

    Paula Ventura Martins

  4. University of Castila-La Mancha, Ciudad Real, Ciudad Real, Spain

    Ricardo Pérez-Castillo

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kohl, P., Schmidts, O., Klöser, L., Werth, H., Kraft, B., Zündorf, A. (2021). STAMP 4 NLP – An Agile Framework for Rapid Quality-Driven NLP Applications Development. In: Paiva, A.C.R., Cavalli, A.R., Ventura Martins, P., Pérez-Castillo, R. (eds) Quality of Information and Communications Technology. QUATIC 2021. Communications in Computer and Information Science, vol 1439. Springer, Cham. https://doi.org/10.1007/978-3-030-85347-1_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-85347-1_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-85346-4

  • Online ISBN: 978-3-030-85347-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation