Understanding Plane Geometry Problems by Integrating Relations Extracted from Text and Diagram

  • Wenbin Gan
  • Xinguo Yu
  • Chao Sun
  • Bin He
  • Mingshu Wang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10749)


Understanding problems is a critical and hard step for solving plane geometry problems. This paper presents a method for understanding plane geometry problems by integrating the information from text and diagram two modalities. Then high-confidence geometric relations are extracted for problem understanding through integrating the information separately extracted from text and diagram. And this paper also presents a function that can visually encode the extracted relations into the diagram to interactively present the visual effects of problem understanding results. The geometric relations are mined from the diagram after visual primitives are detected. The syntax-semantics (S\( ^{2} \)) model method is adopted to extract geometric relations from the text. And an integration process is used to couple the information from text and diagram to obtain the high-confidence geometric relations. The experimental results show that the proposed method can mine geometric relations in high accuracy and it can understand some problems that cannot be understood by using text only or by using diagram only.


Plane geometry problem understanding Geometric relation extraction S\(^{2}\) model Visually presentation Textual information extraction Visual information extraction 



This work is supported by the Open Projects Program of National Laboratory of Pattern Recognition (No. 201600008) and the self-determined research funds of CCNU from the colleges’ basic research and operation of MOE (No. ccnu16JYKX005).


  1. 1.
    Chen, X., Song, D., Wang, D.: Automated generation of geometric theorems from images of diagrams. Ann. Math. Artif. Intell. 74(3–4), 333–358 (2015)MathSciNetCrossRefMATHGoogle Scholar
  2. 2.
    Chou, S.C., Gao, X.S., Zhang, J.Z.: Machine Proofs in Geometry: Automated Production of Readable Proofs for Geometry Theorems. World Scientific, Singapore (1994)CrossRefMATHGoogle Scholar
  3. 3.
    Ge, Q.: Research of automated geometry reasoning and its application with constrained conditions. Ph.D. thesis, Central China Normal University (2011)Google Scholar
  4. 4.
    Guo, H.Y., Liu, Q.T., Chen, M., Huang, H., Ge, Q.: Research for facing the natural language of the geometry drawing. Comput. Sci. 39(6A), 503–506 (2012)Google Scholar
  5. 5.
    Jiang, J.: iGeo: a theorem prover of the intelligent geometry software. Ph.D. thesis, Chengdu Computer Application Research Institute of Chinese Academy of Sciences (2006)Google Scholar
  6. 6.
    Jiang, J., Zhang, J.: A review and prospect of readable machine proofs for geometry theorems. J. Syst. Sci. Complex. 25(4), 802–820 (2012)MathSciNetCrossRefMATHGoogle Scholar
  7. 7.
    Liu, L., Lu, X., Fu, S., Qu, J., Gao, L., Tang, Z.: Plane geometry figure retrieval based on bilayer geometric attributed graph matching. In: ICPR, pp. 309–314. IEEE (2014)Google Scholar
  8. 8.
    Montalvo, F.: Diagram understanding: the intersection of computer vision and graphics. Massachusetts Institute of Technology (1985)Google Scholar
  9. 9.
    Mukherjee, A., Garain, U.: Understanding of natural language text for diagram drawing. In: 13th International Conference on Artificial Intelligence and Soft Computing (2009)Google Scholar
  10. 10.
    Nakamura, Y., Furukawa, R., Nagao, M.: Diagram understanding utilizing natural language text. In: ICDAR, pp. 614–618. IEEE (1993)Google Scholar
  11. 11.
    Seo, M.J., Hajishirzi, H., Farhadi, A., Etzioni, O.: Diagram understanding in geometry questions. In: 28th AAAI Conference on Artificial Intelligence, pp. 2831–2838 (2014)Google Scholar
  12. 12.
    Seo, M.J., Hajishirzi, H., Farhadi, A., Etzioni, O., Malcolm, C.: Solving geometry problems: combining text and diagram interpretation. In: EMNLP, pp. 1466–1476 (2015)Google Scholar
  13. 13.
    Srihari, R.K.: Computational models for integrating linguistic and visual information: a survey. Artif. Intell. Rev. 8(5-6), 349–369 (1994)Google Scholar
  14. 14.
    Wong, W.K., Huang, C.W., et al.: A computer-assisted environment for understanding geometry theorem proving problems and making conjectures. Int. J. Intell. Inf. Database Syst. 3(3), 231–245 (2009)Google Scholar
  15. 15.
    Zhang, H.P., Liu, Q.: ICTCLAS. Institute of Computing Technology, Chinese Academy of Sciences (2002).
  16. 16.
    Zhang, X., Fu, H.: Recognizing and understanding of plane geometry. J. Comput. Appl. 35, 280–283, 341 (2015)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Wenbin Gan
    • 1
  • Xinguo Yu
    • 1
  • Chao Sun
    • 1
  • Bin He
    • 1
  • Mingshu Wang
    • 1
  1. 1.National Engineering Research Center for E-learningCentral China Normal UniversityWuhanChina

Personalised recommendations