Skip to main content
SpringerLink
Log in

Natural Language Processing: Challenges and Future Directions

  • Conference paper
  • First Online:
Artificial Intelligence and Industrial Applications (A2IA 2020)

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 144))

Abstract

Natural language processing (NLP) is a well-known sub-field of artificial intelligence that is having huge success and attention in recent years, its applications are also exploding in terms of innovation and consumer adoption, personal voice assistants and chatbots are two examples among many others, despite this recent success, NLP still has huge challenges and open issues. In this paper, we provide a short overview of NLP, then we dive into the different challenges that are facing it, finally, we conclude by presenting recent trends and future research directions that are speculated by the research community.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 219.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 279.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://gluebenchmark.com/leaderboard.

  2. 2.

    https://gluebenchmark.com.

  3. 3.

    https://super.gluebenchmark.com.

  4. 4.

    https://decanlp.com.

References

  1. Abdul-Mageed, M., Ungar, L.H.: EmoNet: fine-grained emotion detection with gated recurrent neural networks. In: ACL (2017)

    Google Scholar 

  2. Agić, Ž., Vulić, I.: JW300: a wide-coverage parallel corpus for low-resource languages. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, July 2019, pp. 3204–3210. Association for Computational Linguistics (2019). https://doi.org/10.18653/v1/P19-1310

  3. Amac, M.S., Yagcioglu, S., Erdem, A., Erdem, E.: Procedural reasoning networks for understanding multimodal procedures (2019)

    Google Scholar 

  4. Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.R., Samek, W.: On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE 10(7), 1–46 (2015). https://doi.org/10.1371/journal.pone.0130140

    Article  Google Scholar 

  5. Baehrens, D., Schroeter, T., Harmeling, S., Kawanabe, M., Hansen, K., Mueller, K.R.: How to explain individual classification decisions (2009)

    Google Scholar 

  6. Chevalier-Boisvert, M., Bahdanau, D., Lahlou, S., Willems, L., Saharia, C., Nguyen, T.H., Bengio, Y.: BabyAI: a platform to study the sample efficiency of grounded language learning (2018)

    Google Scholar 

  7. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding (2018)

    Google Scholar 

  8. Duong, L., Hoang, V.C.D., Pham, T.Q., Hong, Y.H., Dovgalecs, V., Bashkansky, G., Black, J., Bleeker, A., Huitouze, S.L., Johnson, M.: An adaptable task-oriented dialog system for stand-alone embedded devices. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, Florence, Italy, July 2019, pp. 49–57. Association for Computational Linguistics (2019). https://doi.org/10.18653/v1/P19-3009

  9. Dvorak, A., Novak, V.: Formalization of commonsense reasoning in fuzzy logic in broader sense. Appl. Comput. Math. 10, 106–121 (2011)

    MathSciNet  MATH  Google Scholar 

  10. Geva, M., Goldberg, Y., Berant, J.: Are we modeling the task or the annotator? an investigation of annotator bias in natural language understanding datasets (2019)

    Google Scholar 

  11. Gu, J., Hassan, H., Devlin, J., Li, V.O.: Universal neural machine translation for extremely low resource languages. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), New Orleans, Louisiana, June 2018, pp. 344–354. Association for Computational Linguistics (2018). https://doi.org/10.18653/v1/N18-1032

  12. Gu, J., Wang, Y., Chen, Y., Li, V.O.K., Cho, K.: Meta-learning for low-resource neural machine translation. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October—November 2018, pp. 3622–3631. Association for Computational Linguistics (2018). https://doi.org/10.18653/v1/D18-1398

  13. Guo, C., Cao, J., Zhang, X., Shu, K., Liu, H.: DEAN: learning dual emotion for fake news detection on social media (2019)

    Google Scholar 

  14. Gururangan, S., Swayamdipta, S., Levy, O., Schwartz, R., Bowman, S., Smith, N.A.: Annotation artifacts in natural language inference data. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), New Orleans, Louisiana, June 2018, pp. 107–112. Association for Computational Linguistics (2018). https://doi.org/10.18653/v1/N18-2017

  15. Jiang, C., Yu, H.F., Hsieh, C.J., Chang, K.W.: Learning word embeddings for low-resource languages by PU learning. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), New Orleans, Louisiana, June 2018, pp. 1024–1034. Association for Computational Linguistics (2018). https://doi.org/10.18653/v1/N18-1093

  16. Kordjamshidi, P., Rahgooy, T., Manzoor, U.: Spatial language understanding with multimodal graphs using declarative learning based programming. In: Proceedings of the 2nd Workshop on Structured Prediction for Natural Language Processing, Copenhagen, Denmark, September 2017, pp. 33–43. Association for Computational Linguistics (2017). https://doi.org/10.18653/v1/W17-4306

  17. Lample, G., Charton, F.: Deep learning for symbolic mathematics (2019)

    Google Scholar 

  18. Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., Soricut, R.: ALBERT: A lite BERT for self-supervised learning of language representations (2019)

    Google Scholar 

  19. Liu, C.W., Lowe, R., Serban, I., Noseworthy, M., Charlin, L., Pineau, J.: How NOT to evaluate your dialogue system: an empirical study of unsupervised evaluation metrics for dialogue response generation. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Texas, November 2016, pp. 2122–2132. Association for Computational Linguistics (2016). https://doi.org/10.18653/v1/D16-1230

  20. Liu, H., Yin, Q., Wang, W.Y.: Towards explainable NLP: a generative explanation framework for text classification. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, July 2019, pp. 5570–5581. Association for Computational Linguistics (2019). https://doi.org/10.18653/v1/P19-1560

  21. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., Stoyanov, V.: RoBERTa: a robustly optimized BERT pretraining approach (2019)

    Google Scholar 

  22. McCoy, T., Pavlick, E., Linzen, T.: Right for the wrong reasons: diagnosing syntactic heuristics in natural language inference. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, July 2019, pp. 3428–3448. Association for Computational Linguistics (2019). https://doi.org/10.18653/v1/P19-1334

  23. Murinová, P., Novák, V.: A formal theory of generalized intermediate syllogisms. Fuzzy Sets Syst. 186(1), 47–80 (2012). https://doi.org/10.1016/j.fss.2011.07.004

    Article  MathSciNet  MATH  Google Scholar 

  24. Murinová, P., Novák, V.: Analysis of generalized square of opposition with intermediate quantifiers. Fuzzy Sets Syst. 242, 89–113 (2014). https://doi.org/10.1016/j.fss.2013.05.006

    Article  MathSciNet  MATH  Google Scholar 

  25. Nie, Y., Williams, A., Dinan, E., Bansal, M., Weston, J., Kiela, D.: Adversarial NLI: a new benchmark for natural language understanding (2019)

    Google Scholar 

  26. Novák, V.: A comprehensive theory of trichotomous evaluative linguistic expressions. Fuzzy Sets Syst. 159(22), 2939–2969 (2008). https://doi.org/10.1016/j.fss.2008.02.023

    Article  MathSciNet  MATH  Google Scholar 

  27. Novikova, J., Dušek, O., Cercas Curry, A., Rieser, V.: Why we need new evaluation metrics for NLG. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Denmark, September 2017, pp. 2241–2252. Association for Computational Linguistics (2017). https://doi.org/10.18653/v1/D17-1238

  28. Novák, V.: Linguistic characterization of time series. Fuzzy Sets Syst. 285, 52–72 (2016). https://doi.org/10.1016/j.fss.2015.07.017

    Article  MathSciNet  MATH  Google Scholar 

  29. Peyrard, M.: Studying summarization evaluation metrics in the appropriate scoring range. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, July 2019, pp. 5093–5100. Association for Computational Linguistics (2019). https://doi.org/10.18653/v1/P19-1502

  30. Poliak, A., Naradowsky, J., Haldar, A., Rudinger, R., Durme, B.V.: Hypothesis only baselines in natural language inference (2018)

    Google Scholar 

  31. Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I.: Language models are unsupervised multitask learners (2019)

    Google Scholar 

  32. Rashkin, H., Smith, E.M., Li, M., Boureau, Y.L.: Towards empathetic open-domain conversation models: a new benchmark and dataset. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, July 2019, pp. 5370–5381. Association for Computational Linguistics (2019). https://doi.org/10.18653/v1/P19-1534

  33. Şahin, G.G., Steedman, M.: Data augmentation via dependency tree morphing for low-resource languages. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October–November 2018, pp. 5004–5009. Association for Computational Linguistics (2018). https://doi.org/10.18653/v1/D18-1545

  34. Sanabria, R., Caglayan, O., Palaskar, S., Elliott, D., Barrault, L., Specia, L., Metze, F.: How2: A large-scale dataset for multimodal language understanding (2018)

    Google Scholar 

  35. Seyeditabari, A., Tabari, N., Gholizadeh, S., Zadrozny, W.: Emotion detection in text: focusing on latent representation (2019)

    Google Scholar 

  36. Simonyan, K., Vedaldi, A., Zisserman, A.: Deep inside convolutional networks: visualising image classification models and saliency maps (2013)

    Google Scholar 

  37. Srivastava, Y., Murali, V., Dubey, S.R., Mukherjee, S.: Visual question answering using deep learning: a survey and performance analysis (2019)

    Google Scholar 

  38. Sun, Y., Wang, S., Li, Y., Feng, S., Tian, H., Wu, H., Wang, H.: Ernie 2.0: a continual pre-training framework for language understanding (2019)

    Google Scholar 

  39. Wang, Y., Yao, Q., Kwok, J., Ni, L.M.: Generalizing from a few examples: a survey on few-shot learning (2019)

    Google Scholar 

  40. Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., Le, Q.V.: XLNet: generalized autoregressive pretraining for language understanding (2019)

    Google Scholar 

  41. Young, T., Hazarika, D., Poria, S., Cambria, E.: Recent trends in deep learning based natural language processing [review article]. IEEE Comput. Intell. Mag. 13(3), 55–75 (2018). https://doi.org/10.1109/MCI.2018.2840738. cited By 209

    Article  Google Scholar 

  42. Yu, Z., Cui, Y., Yu, J., Tao, D., Tian, Q.: Multimodal unified attention networks for vision-and-language interactions (2019)

    Google Scholar 

  43. Zhang, Y., Yang, Q.: A survey on multi-task learning (2017)

    Google Scholar 

  44. Zhang, Z., Han, X., Liu, Z., Jiang, X., Sun, M., Liu, Q.: ERNIE: enhanced language representation with informative entities (2019)

    Google Scholar 

  45. Zhong, P., Wang, D., Miao, C.: Knowledge-enriched transformer for emotion detection in textual conversations (2019)

    Google Scholar 

  46. Zhou, X., Zhang, Y., Cui, L., Huang, D.: Evaluating commonsense in pre-trained language models (2019)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. LSE2I Laboratory of National School of Applied Sciences, Oujda, Morocco

    Zakaria Kaddari, Youssef Mellah, Jamal Berrich, Mohammed G. Belkasmi & Toumi Bouchentouf

Authors
  1. Zakaria Kaddari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Youssef Mellah
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jamal Berrich
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mohammed G. Belkasmi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Toumi Bouchentouf
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Zakaria Kaddari , Youssef Mellah , Jamal Berrich , Mohammed G. Belkasmi or Toumi Bouchentouf .

Editor information

Editors and Affiliations

  1. Department of Mathematics and Computer Science, National Graduate School for Arts and Crafts, Meknes, Morocco

    Tawfik Masrour

  2. Department of Industrial and Manufacturing Engineering, National Graduate School for Arts and Crafts, Meknes, Morocco

    Ibtissam El Hassani

  3. Department of Industrial and Manufacturing Engineering, National Graduate School for Arts and Crafts, Meknes, Morocco

    Anass Cherrafi

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kaddari, Z., Mellah, Y., Berrich, J., Belkasmi, M.G., Bouchentouf, T. (2021). Natural Language Processing: Challenges and Future Directions. In: Masrour, T., El Hassani, I., Cherrafi, A. (eds) Artificial Intelligence and Industrial Applications. A2IA 2020. Lecture Notes in Networks and Systems, vol 144. Springer, Cham. https://doi.org/10.1007/978-3-030-53970-2_22

Download citation

Publish with us

Policies and ethics

Search

Navigation