An Unsupervised Machine Learning Approach to Body Text and Table of Contents Extraction from Digital Scientific Articles

  • Stefan Klampfl
  • Roman Kern
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8092)


Scientific articles are predominantly stored in digital document formats, which are optimised for presentation, but lack structural information. This poses challenges to access the documents’ content, for example for information retrieval. We have developed a processing pipeline that makes use of unsupervised machine learning techniques and heuristics to detect the logical structure of a PDF document. Our system uses only information available from the current document and does not require any pre-trained model. Starting from a set of contiguous text blocks extracted from the PDF file, we first determine geometrical relations between these blocks. These relations, together with geometrical and font information, are then used categorize the blocks into different classes. Based on this logical structure we finally extract the body text and the table of contents of a scientific article. We evaluate our pipeline on a number of datasets and compare it with state-of-the-art document structure analysis approaches.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Mao, S., Rosenfeld, A., Kanungo, T.: Document structure analysis algorithms: A literature survey. Proceedings of SPIE 5010(1), 197–207 (2003)CrossRefGoogle Scholar
  2. 2.
    Kern, R., Jack, K., Hristakeva, M., Granitzer, M.: TeamBeam - Meta-Data Extraction from Scientific Literature. In: 1st International Workshop on Mining Scientific Publications (2012)Google Scholar
  3. 3.
    Peng, F., McCallum, A.: Accurate Information Extraction from Research Papers using Conditional Random Fields. In: HLTNAACL 2004, vol. 2004, pp. 329–336 (2004)Google Scholar
  4. 4.
    Councill, I.G., Giles, C.L., Kan, M.Y.: ParsCit: An open-source CRF Reference String Parsing Package. In: Proceedings of LREC, vol. 2008, pp. 661–667. Citeseer, European Language Resources Association, ELRA (2008)Google Scholar
  5. 5.
    Luong, M.T., Nguyen, T.D., Kan, M.Y.: Logical structure recovery in scholarly articles with rich document features. International Journal of Digital Library Systems 1(4), 1–23 (2011)CrossRefGoogle Scholar
  6. 6.
    Ramakrishnan, C., Patnia, A., Hovy, E., Burns, G.A.: Layout-Aware Text Extraction from Full-text PDF of Scientific Articles. Source Code for Biology and Medicine 7(1), 7 (2012)CrossRefGoogle Scholar
  7. 7.
    Gao, L., Tang, Z., Lin, X., Liu, Y., Qiu, R., Wang, Y.: Structure extraction from PDF-based book documents. In: Proceedings of the 11th Annual International ACM/IEEE Joint Conference on Digital Libraries, pp. 11–20 (2011)Google Scholar
  8. 8.
    Lin, X.: Header and Footer Extraction by Page-Association. Proceedings of SPIE 5010, 164–171 (2002)CrossRefGoogle Scholar
  9. 9.
    Granitzer, M., Hristakeva, M., Knight, R., Jack, K., Kern, R.: A Comparison of Layout based Bibliographic Metadata Extraction Techniques. In: WIMS 2012 - International Conference on Web Intelligence, Mining and Semantics, pp. 19:1–19:8. ACM, New York (2012)Google Scholar
  10. 10.
    Liu, Y., Mitra, P., Giles, C.L.: Identifying table boundaries in digital documents via sparse line detection. In: Proceeding of the 17th ACM Conference on Information and Knowledge Mining, CIKM 2008, pp. 1311–1320. ACM Press (2008)Google Scholar
  11. 11.
    Aiello, M., Monz, C., Todoran, L., Worring, M.: Document understanding for a broad class of documents. International Journal on Document Analysis and Recognition 5(1), 1–16 (2002)zbMATHCrossRefGoogle Scholar
  12. 12.
    Malerba, D., Ceci, M., Berardi, M.: Machine learning for reading order detection in document image understanding. Machine Learning in Document Analysis, 45–69 (2008)Google Scholar
  13. 13.
    Tkaczyk, D., Czeczko, A., Rusek, K.: GROTOAP: ground truth for open access publications. In: Proceedings of the 12th ACM/IEEE-CS Joint Conference on Digital Libraries, pp. 381–382 (2012)Google Scholar
  14. 14.
    Tkaczyk, D., Bolikowski, L., Czeczko, A., Rusek, K.: A Modular Metadata Extraction System for Born-Digital Articles. In: 2012 10th IAPR International Workshop on Document Analysis Systems, pp. 11–16 (March 2012)Google Scholar
  15. 15.
    Zhang, K., Shasha, D.: Simple Fast Algorithms for the Editing Distance between Trees and Related Problems. SIAM Journal on Computing 18(6), 1245–1262 (1989)MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Stefan Klampfl
    • 1
  • Roman Kern
    • 1
    • 2
  1. 1.Know-Center GmbHAustria
  2. 2.Knowledge Technologies InstituteGraz University of TechnologyGrazAustria

Personalised recommendations