Advertisement

Deep Learning for French Legal Data Categorization

  • Eya HammamiEmail author
  • Imen Akermi
  • Rim Faiz
  • Mohand Boughanem
Conference paper
First Online:
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11815)

Abstract

In current years, deep learning has showed promising results when used in the field of natural language processing (NLP). Neural Networks (NNs) such as convolutional neural network (CNN) and recurrent neural network (RNN) have been utilized for different NLP tasks like information retrieval, sentiment analysis and document classification. In this paper, we explore the use of NNs-based method for legal text classification. In our case, the results show that NN models with a fixed input length outperforms baseline methods.

Keywords

Natural Language Processing Deep learning Convolutional Neural Networks Document categorization Legal domain 
This is a preview of subscription content, log in to check access.

References

  1. 1.
    Joachims, T.: Text categorization with support vector machines: learning with many relevant features. In: Nédellec, C., Rouveirol, C. (eds.) ECML 1998. LNCS, vol. 1398, pp. 137–142. Springer, Heidelberg (1998).  https://doi.org/10.1007/BFb0026683CrossRefGoogle Scholar
  2. 2.
    McCallum, A., Nigam, K., et al.: A comparison of event models for naive bayes text classification. In: AAAI-1998 Workshop on Learning for Text Categorization, vol. 752, pp. 41–48. Citeseer (1998)Google Scholar
  3. 3.
    Joulin, A., Grave, E., Bojanowski, P., Mikolov, T.: Bag of Tricks for Efficient Text Classification. CoRR, abs/1607.01759 (2016). http://arxiv.org/abs/1607.01759, arXiv:1607.01759. https://dblp.org/rec/bib/journals/corr/JoulinGBM16, dblp computer science bibliography, https://dblp.org. Accessed 13 Aug 2018
  4. 4.
    Collobert, R., Weston, J.: A unified architecture for natural language processing: deep neural networks with multitask learning. In: Proceedings of the 25th International Conference on Machine Learning, pp. 160–167. ACM (2008) Google Scholar
  5. 5.
    Yogatama, D., Dyer, C., Ling, W., Blunsom, P.: Generative and discriminative text classification with recurrent neural networks. arXiv preprint arXiv:1703.01898 (2017)
  6. 6.
    Xiao, Y., Cho, K.: Efficient Character-Level Document Classification by Combining Convolution and Recurrent Layers, CoRR, abs/1602.00367 (2016). http://arxiv.org/abs/1602.00367, arXiv:1602.00367, https://dblp.org/rec/bib/journals/corr/XiaoC16, dblp computer science bibliography, https://dblp.org. Accessed 13 Aug 2018
  7. 7.
    Kim, Y.: Convolutional Neural Networks for Sentence Classification, CoRR, abs/1408.5882 (2014). http://arxiv.org/abs/1408.5882, arXiv:1408.5882, https://dblp.org/rec/bib/journals/corr/Kim14f, dblp computer science bibliography, https://dblp.org. Accessed 13 Aug 2018
  8. 8.
    Zhang, X., Zhao, J.J., LeCun, Y.: Character-Level Convolutional Networks for Text Classification, CoRR, abs/1509.01626 (2015). http://arxiv.org/abs/1509.01626, arXiv:1509.01626, https://dblp.org/rec/bib/journals/corr/ZhangZL15, dblp computer science bibliography, https://dblp.org. Accessed 13 Aug 2018
  9. 9.
    Conneau, A., Schwenk, H., Barrault, L., LeCun, Y.: Very Deep Convolutional Networks for Natural Language Processing, CoRR, abs/1606.01781 (2016). http://arxiv.org/abs/1606.01781, arXiv:1606.01781, https://dblp.org/rec/bib/journals/corr/ConneauSBL16, dblp computer science bibliography, https://dblp.org. Accessed 13 Aug 2018
  10. 10.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P., et al.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  11. 11.
    Collobert, R., Weston, J.: A unified architecture for natural language processing: deep neural networks with multitask learning. In: Proceedings of the 25th International Conference on Machine Learning, ICML 2008, Helsinki, Finland, vol. 8, pp. 160–167. ACM, New York (2008). http://doi.acm.org/10.1145/1390156.1390177,  https://doi.org/10.1145/1390156.1390177.1390177. ISBN: 978–1-60558-205-4
  12. 12.
    Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)
  13. 13.
    MKalchbrenner, N., Grefenstette, E., Blunsom, P.: A convolutional neural network for modelling sentences. arXiv preprint arXiv:1404.2188 (2014)
  14. 14.
    Zhang, X., Zhao, J., LeCun, Y.: Character-level convolutional networks for text classification. In: Advances in Neural Information Processing Systems, pp. 649–657 (2015)Google Scholar
  15. 15.
    Koomsubha, T., Vateekul, P.: A character-level convolutional neural network with dynamic input length for Thai text categorization. In: 2017 9th International Conference on Knowledge and Smart Technology (KST), pp. 101–105. IEEE (2017)Google Scholar
  16. 16.
    Kim, Y., Jernite, Y., Sontag, D., Rush, A.M.: Character-aware neural language models. In: Thirtieth AAAI Conference on Artificial Intelligence (2016)Google Scholar
  17. 17.
    Radford, A., Jozefowicz, R., Sutskever, I.: Learning to generate reviews and discovering sentiment. arXiv preprint arXiv:1704.01444 (2017)
  18. 18.
    Socher, R., et al.: Recursive deep models for semantic compositionality over a sentiment treebank. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1631–1642 (2013)Google Scholar
  19. 19.
    Nallapati, R., Manning, C.D.: Legal docket-entry classification: Where machine learning stumbles. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 438–446. Association for Computational Linguistics (2008)Google Scholar
  20. 20.
    Sulea, O.-M., Zampieri, M., Malmasi, S., Vela, M., Dinu, L.P., van Genabith, J.: Exploring the use of text classification in the legal domain. arXiv preprint arXiv:1710.09306 (2017)
  21. 21.
    Undavia, S., Meyers, A., Ortega, J.E.: A Comparative study of classifying legal documents with neural networks. In: 2018 Federated Conference on Computer Science and Information Systems (FedCSIS), pp. 515–522. IEEE (2018)Google Scholar
  22. 22.
    Da Silva, N.C., et al.: Document type classification for Brazil’s supreme court using a convolutional neural network. In: The Tenth International Conference on Forensic Computer Science and Cyber Law-ICoFCS, pp. 7–11 (2018)Google Scholar
  23. 23.
    Wei, F., Qin, H., Ye, S., Zhao, H.: Empirical study of deep learning for text classification in legal document review. In: 2018 IEEE International Conference on Big Data (Big Data), pp. 3317–3320. IEEE (2018)Google Scholar
  24. 24.
    Kim, Y.: Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882 (2014)
  25. 25.
    Wang, X., Liu, Y., Chengjie, S.U.N., Wang, B., Wang, X.: Predicting polarities of tweets by composing word embeddings with long short-term memory. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), vol. 1, pp. 1343–1353 (2015)Google Scholar
  26. 26.
    Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014). JMLR.orgMathSciNetzbMATHGoogle Scholar
  27. 27.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Eya Hammami
    • 1
    Email author
  • Imen Akermi
    • 2
  • Rim Faiz
    • 1
  • Mohand Boughanem
    • 2
  1. 1.LARODEC LaboratoryUniversity of ManoubaManoubaTunisia
  2. 2.IRIT LaboratoryUniversity of Toulouse 3ToulouseFrance

Personalised recommendations

Cite paper

Buy options