Advertisement

A Comparative Study of Summarization Algorithms Applied to Legal Case Judgments

  • Paheli BhattacharyaEmail author
  • Kaustubh Hiware
  • Subham Rajgaria
  • Nilay Pochhi
  • Kripabandhu Ghosh
  • Saptarshi Ghosh
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11437)

Abstract

Summarization of legal case judgments is an important problem because the huge length and complexity of such documents make them difficult to read as a whole. Many summarization algorithms have been proposed till date, both for general text documents and a few specifically targeted to summarizing legal documents of various countries. However, to our knowledge, there has not been any systematic comparison of the performances of different algorithms in summarizing legal case documents. In this paper, we perform the first such systematic comparison of summarization algorithms applied to legal judgments. We experiment on a large set of Indian Supreme Court judgments, and a large variety of summarization algorithms including both unsupervised and supervised ones. We assess how well domain-independent summarization approaches perform on legal case judgments, and how approaches specifically designed for legal case documents of other countries (e.g., Canada, Australia) generalize to Indian Supreme Court documents. Apart from quantitatively evaluating summaries by comparing with gold standard summaries, we also give important qualitative insights on the performance of different algorithms from the perspective of a law expert.

Keywords

Summarization Legal case judgment Supervised Unsupervised 

Notes

Acknowledgment

We sincerely acknowledge Prof. Uday Shankar and Uma Jandhyala from Rajiv Gandhi School of Intellectual Property Law, Indian Institute of Technology Kharagpur, India for their valuable feedback.

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
    Allahyari, M., et al.: Text summarization techniques: a brief survey. arXiv preprint arXiv:1707.02268 (2017)
  8. 8.
    Cao, Z., Wei, F., Li, S., Li, W., Zhou, M., Houfeng, W.: Learning summary prior representation for extractive summarization. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), vol. 2, pp. 829–833 (2015)Google Scholar
  9. 9.
    Cheng, J., Lapata, M.: Neural summarization by extracting sentences and words. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), vol. 1, pp. 484–494 (2016)Google Scholar
  10. 10.
    Chopra, S., Auli, M., Rush, A.M.: Abstractive sentence summarization with attentive recurrent neural networks. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 93–98 (2016)Google Scholar
  11. 11.
    Das, D., Martins, A.F.: A survey on automatic text summarization. Lit. Surv. Lang. Stat. II Course CMU 4, 192–195 (2007)Google Scholar
  12. 12.
    Dong, Y.: A survey on neural network-based summarization methods. CoRR abs/1804.04589 (2018). http://arxiv.org/abs/1804.04589
  13. 13.
    Erkan, G., Radev, D.R.: Lexrank: graph-based lexical centrality as salience in text summarization. J. Artif. Int. Res. 22(1), 457–479 (2004)Google Scholar
  14. 14.
    Farzindar, A., Lapalme, G.: Legal text summarization by exploration of the thematic structure and argumentative roles. Text Summarization Branches Out (2004)Google Scholar
  15. 15.
    Farzindar, A., Lapalme, G.: Letsum, an automatic legal text summarizing system. Legal knowledge and information systems, JURIX, pp. 11–18 (2004)Google Scholar
  16. 16.
    Text summarization with NLTK (2014). https://tinyurl.com/frequency-summarizer
  17. 17.
    Galgani, F., Compton, P., Hoffmann, A.: Citation based summarisation of legal texts. In: Anthony, P., Ishizuka, M., Lukose, D. (eds.) PRICAI 2012. LNCS (LNAI), vol. 7458, pp. 40–52. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-32695-0_6CrossRefGoogle Scholar
  18. 18.
    Galgani, F., Compton, P., Hoffmann, A.: Combining different summarization techniques for legal text. In: Proceedings of the Workshop on Innovative Hybrid Approaches to the Processing of Textual Data, pp. 115–123. Association for Computational Linguistics (2012)Google Scholar
  19. 19.
    Gelbart, D., Smith, J.: Beyond boolean search: flexicon, a legal tex-based intelligent system. In: Proceedings of the 3rd International Conference on Artificial Intelligence and Law, pp. 225–234. ACM (1991)Google Scholar
  20. 20.
    Gong, Y., Liu, X.: Generic text summarization using relevance measure and latent semantic analysis. In: SIGIR, pp. 19–25 (2001)Google Scholar
  21. 21.
    Grover, C., Hachey, B., Hughson, I., Korycinski, C.: Automatic summarisation of legal documents. In: Proceedings of the 9th International Conference on Artificial Intelligence and Law, pp. 243–251. ACM (2003)Google Scholar
  22. 22.
    Grover, C., Hachey, B., Korycinski, C.: Summarising legal texts: sentential tense and argumentative roles. In: Proceedings of the HLT-NAACL 03 on Text Summarization Workshop, vol. 5, pp. 33–40. Association for Computational Linguistics (2003)Google Scholar
  23. 23.
    Gupta, V., Lehal, G.S.: A survey of text summarization extractive techniques. J. Emerg. Technol. Web Intell. 2(3), 258–268 (2010)Google Scholar
  24. 24.
    Hachey, B., Grover, C.: Sentence classification experiments for legal text summarisation (2004)Google Scholar
  25. 25.
    Hachey, B., Grover, C.: Extractive summarisation of legal texts. Artif. Intell. Law 14(4), 305–345 (2006)CrossRefGoogle Scholar
  26. 26.
    He, Z., et al.: Document summarization based on data reconstruction. In: Proceedings of AAAI Conference on Artificial Intelligence, pp. 620–626 (2012)Google Scholar
  27. 27.
    Kågebäck, M., Mogren, O., Tahmasebi, N., Dubhashi, D.: Extractive summarization using continuous vector space models. In: Proceedings of the 2nd Workshop on Continuous Vector Space Models and their Compositionality (CVSC), pp. 31–39 (2014)Google Scholar
  28. 28.
    Kanapala, A., Pal, S., Pamula, R.: Text summarization from legal documents: a survey. Artif. Intell. Rev., 1–32 (2017).  https://doi.org/10.1007/s10462-017-9566-2CrossRefGoogle Scholar
  29. 29.
    Luhn, H.P.: The automatic creation of literature abstracts. IBM J. Res. Dev. 2(2), 159–165 (1958)MathSciNetCrossRefGoogle Scholar
  30. 30.
    Agrawal, M., Mehta, P., Ghosh, K.: Overview of information access in legal domain fire 2013 (2013). https://www.isical.ac.in/~fire/wn/LEAGAL/overview.pdf/
  31. 31.
    Mihalcea, R., Tarau, P.: TextRank: bringing order into texts. In: EMNLP (2004)Google Scholar
  32. 32.
    Moens, M.F., Uyttendaele, C., Dumortier, J.: Abstracting of legal cases: the salomon experience. In: Proceedings of the 6th International Conference on Artificial Intelligence and Law, pp. 114–122. ACM (1997)Google Scholar
  33. 33.
    Nallapati, R., Zhai, F., Zhou, B.: Summarunner: a recurrent neural network based sequence model for extractive summarization of documents. In: AAAI, pp. 3075–3081 (2017)Google Scholar
  34. 34.
    Nallapati, R., Zhou, B., dos Santos, C., Gulcehre, C., Xiang, B.: Abstractive text summarization using sequence-to-sequence RNNS and beyond. In: Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning, pp. 280–290 (2016)Google Scholar
  35. 35.
    Narayan, S., Cohen, S.B., Lapata, M.: Ranking sentences for extractive summarization with reinforcement learning. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), vol. 1, pp. 1747–1759 (2018)Google Scholar
  36. 36.
    Nenkova, A., McKeown, K.: A Survey of Text Summarization Techniques. In: Aggarwal, C., Zhai, C. (eds) Mining Text Data, pp. 43–76. Springer, Boston (2012).  https://doi.org/10.1007/978-1-4614-3223-4_3CrossRefGoogle Scholar
  37. 37.
    Paulus, R., Xiong, C., Socher, R.: A deep reinforced model for abstractive summarization. arXiv preprint arXiv:1705.04304 (2017)
  38. 38.
    Polsley, S., Jhunjhunwala, P., Huang, R.: Casesummarizer: a system for automated summarization of legal texts. In: Proceedings of COLING 2016, The 26th International Conference on Computational Linguistics: System Demonstrations, pp. 258–262 (2016)Google Scholar
  39. 39.
    Rush, A.M., Chopra, S., Weston, J.: A neural attention model for abstractive sentence summarization. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 379–389 (2015)Google Scholar
  40. 40.
    Saravanan, M., Ravindran, B., Raman, S.: Improving legal document summarization using graphical models. In: Proceedings of the 2006 Conference on Legal Knowledge and Information Systems: JURIX 2006: The Nineteenth Annual Conference, pp. 51–60. IOS Press (2006)Google Scholar
  41. 41.
    See, A., Liu, P.J., Manning, C.D.: Get to the point: summarization with pointer-generator networks. arXiv preprint arXiv:1704.04368 (2017)
  42. 42.
    Verma, S., Nidhi, V.: Extractive summarization using deep learning. arXiv preprint arXiv:1708.04439 (2017)
  43. 43.
    Yin, W., Pei, Y.: Optimizing sentence modeling and selection for document summarization. In: IJCAI, pp. 1383–1389 (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Paheli Bhattacharya
    • 1
    Email author
  • Kaustubh Hiware
    • 1
  • Subham Rajgaria
    • 1
  • Nilay Pochhi
    • 1
  • Kripabandhu Ghosh
    • 2
  • Saptarshi Ghosh
    • 1
  1. 1.Indian Institute of Technology KharagpurKharagpurIndia
  2. 2.Indian Institute of Technology KanpurKanpurIndia

Personalised recommendations