Advertisement

Artificial Intelligence and Law

, Volume 27, Issue 2, pp 171–198 | Cite as

Deep learning in law: early adaptation and legal word embeddings trained on large corpora

  • Ilias ChalkidisEmail author
  • Dimitrios Kampas
Article
  • 420 Downloads

Abstract

Deep Learning has been widely used for tackling challenging natural language processing tasks over the recent years. Similarly, the application of Deep Neural Networks in legal analytics has increased significantly. In this survey, we study the early adaptation of Deep Learning in legal analytics focusing on three main fields; text classification, information extraction, and information retrieval. We focus on the semantic feature representations, a key instrument for the successful application of deep learning in natural language processing. Additionally, we share pre-trained legal word embeddings using the word2vec model over large corpora, comprised legislations from UK, EU, Canada, Australia, USA, and Japan among others.

Keywords

Natural language processing Deep learning Legal word vectors 

References

  1. Bojanowski P, Grave E, Joulin A, Mikolov T (2016) Enriching word vectors with subword information. arXiv preprint arXiv:160704606
  2. Branting LK, Yeh A, Weiss B, Merkhofer E, Brown B (2017) Inducing predictive models for decision support in administrative adjudication. In: Proceedings of the MIREL Workshop on the The 16th International Conference on Artificial Intelligence and Law, London, UKGoogle Scholar
  3. Chalkidis I, Androutsopoulos I (2017) A deep learning approach to contract element extraction. In: Proceedings of the 30th International Conference on Legal Knowledge and Information Systems, Luxembourg, pp 155–164Google Scholar
  4. Chalkidis I, Androutsopoulos I, Michos A (2017) Extracting contract elements. In: Proceedings of The 16th International Conference on Artificial Intelligence and Law, London, UK, pp 19–28Google Scholar
  5. Chalkidis I, Androutsopoulos I, Michos A (2018) Obligation and prohibition extraction using hierarchical rnns. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL 2018), Melbourne, AustraliaGoogle Scholar
  6. Do PK, Nguyen HT, Tran CX, Nguyen MT, Nguyen ML (2017) Legal Question Answering using Ranking SVM and Deep Convolutional Neural Network. CoRR abs/1703.0. arXiv:1703.05320
  7. Firth JR (1935) The technique of semantics. Trans Philos Soc 34(1):36–73CrossRefGoogle Scholar
  8. Goldberg Y (2017) Neural network methods in natural language processing. Morgan and Claypool Publishers, San RafaelCrossRefGoogle Scholar
  9. Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press, CambridgezbMATHGoogle Scholar
  10. Howard J, Ruder S (2018) Universal language model fine-tuning for text classification. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Melbourne, Australia, pp 328–339Google Scholar
  11. Kim My, Xu Y, Goebel R (2015) A convolutional neural network in legal question answering. In: Ninth International Workshop on Juris-informatics (JURISIN)Google Scholar
  12. Kudo T, Yamamoto K, Matsumoto Y (2004) Applying conditional random fields to Japanese morphological analysis. In: Proceedings of the Empirical Methods for Natural Language Processing, Barcelona, Spain, pp 230–237Google Scholar
  13. Mikolov T, Sutskever I, Chen K, Corrado G, Dean J (2013) Distributed representations of words and phrases and their compositionality. In: Proceedings of the 26th International Conference on Neural Information Processing Systems, Stateline, NVGoogle Scholar
  14. Morimoto A, Kubo D, Sato M, Shindo H,  Matsumoto Y (2017) Legal question answering system using neural attention. In: Proceedings of COLIEE - International Conference on Artificial Intelligence and Law, London, UKGoogle Scholar
  15. Nanda R, John AK, Caro LD, Boella G, Robaldo L (2017) Legal information retrieval using topic clustering and neural networks. In: 4th competition on legal information extraction and entailment, 16th international conference on artificial intelligence and law vol. 47, pp 68–78Google Scholar
  16. Nejadgholi I, Bougueng R, Witherspoon S (2017) A semi-supervised training method for semantic search of legal facts in Canadian immigration cases. Front Artif Intell Appl 302:125–134Google Scholar
  17. Nguyen T, Nguyen L, Tojo S, Satoh K, Shimazu A (2017) Single and multiple layer BI-LSTM-CRF for recognizing requisite and effectuation parts in legal texts. In: 2nd ASAIL Workshop on 16th 16th international conference on artificial intelligence and law, London, UKGoogle Scholar
  18. Nguyen T, Nguyen L, Tojo S, Satoh K, Shimazu A (2018) Recurrent neural network-based models for recognizing requisite and effectuation parts in legal texts. Artif Intell Law 26(2):169–199CrossRefGoogle Scholar
  19. O’Neill J, Buitelaar P, Robin C, Brien LO (2017) Classifying Sentential Modality in Legal Language: A Use Case in Financial Regulations, Acts and Directives. In: Proceedings of the 16th international conference on artificial intelligence and law, London, UK, pp 159–168Google Scholar
  20. Pennington J, Socher R, Manning CD (2014) GloVe: Global vectors for word representation. In: Proceedings of the conference on empirical methods in natural language processing, Doha, Qatar, pp 1532–1543Google Scholar
  21. Peters ME, Neumann M, Iyyer M, Gardner M, Clark C, Lee K, Zettlemoyer L (2018) Deep contextualized word representations. In: Conference of NA chapter of the association for computational linguistics, New Orleans, Louisiana, USAGoogle Scholar
  22. Tang G, Guo H, Guo Z, Xu S (2016) Matching law cases and reference law provision with a neural attention model. IBM China Research, BeijingGoogle Scholar
  23. Yang Z, Yang D, Dyer C, He X, Smola A, Hovy E (2016) Hierarchical attention networks for document classification. In: Proceedings of the 15th conference of the North American chapter of the association for computational linguistics: human language technologies, San Diego, CA, USA, pp 1480–1489Google Scholar

Copyright information

© Springer Nature B.V. 2018

Authors and Affiliations

  1. 1.Department of InformaticsAthens University of Economics and BusinessAthensGreece
  2. 2.IT for Innovation ServicesLuxembourg Institute of Science and TechnologyEsch-sur-AlzetteLuxembourg

Personalised recommendations