Skip to main content

OrgBR-M: a method to assist in organizing bibliographic material based on formal concept analysis—a case study in educational data mining


For conducting a literature review is necessary a preliminary organization of the available bibliographic material. In this article, we present a novel method called OrgBR-M (method to organize bibliographic references), based on the formal concept analysis theory, to assist in organizing bibliographic material. Our method systematizes the organization of bibliography and proposes metrics to assist in guiding the literature review. As a case study, we apply the OrgBR-M method to perform a literature review of the educational data mining field of study.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9


  1. Entity: Anything that exists and has some understanding in a domain [16, 17].

  2. Conexp:

  3. Conexp:


  1. Ganter, B., Wille, R.: Formal Concept Analysis: Mathematical Foundations, 1st edn. Springer, New York (1997)

    MATH  Google Scholar 

  2. Ganter, B., Stumme, G., Wille, R.: Formal concept analysis: theory and applications. J. Univ. Comput. Sci. 10(8), 955 (2004).

    Article  MathSciNet  Google Scholar 

  3. Wille, R.: In Formal Concept Analysis. In: Ganter, B., Stumme, G., Wille, R. (eds.) Lecture Notes in Computer Science, vol. 3626, pp. 1–33. Springer, Berlin (2005).

  4. Ganter, B., Stumme, G., Wille, R.: Formal Concept Analysis, Foundations and Applications. Lecture Notes in Computer Science, vol. 3626. Springer, Berlin (2005)

    Book  Google Scholar 

  5. Altinel, B., Ganiz, M.C.: Semantic text classification: a survey of past and recent advances. Inf. Process. Manag. 54(6), 1129 (2018).

    Article  Google Scholar 

  6. Huynh, T., Takasu, A., Masada, T., Hoang, K.: Collaborator recommendation for isolated researchers, pp. 639–644 (2014).

  7. Rodrigues M.W., Brandão, W.C., Zárate L.E.: Recommending scientific collaboration from researchgate. In: Proceedings of the 7th Brazilian Conference on Intelligent systems (Sociedade Brasileira de Computação (SBC)), BRACIS’18, pp. 1–6 (2018).

  8. Gipp, B., Beel, J., Hentschel, C.: Scienstein: A research paper recommender. In: Proceedings of the International Conference on Emerging Trends in Computing (ICETiC’09), ICETiC’09, pp. 309–315 (2009)

  9. Galgani, F., Compton, P., Hoffmann, A.: Summarization based on bi-directional citation analysis. Inf. Process. Manag. 51(1), 1 (2015).

    Article  Google Scholar 

  10. Villegas, N.M., Sánchez, C., Díaz-Cely, J., Tamura, G.: Characterizing context-aware recommender systems: a systematic literature review. Knowl.-Based Syst. 140, 173 (2018).

    Article  Google Scholar 

  11. Raamkumar, A.S., Foo, S., Pang, N.L.S.: Rec4lrw-scientific paper recommender system for literature review and writing. Front. Artif. Intell. Appl. 275, 106 (2015).

    Article  Google Scholar 

  12. Raamkumar, A.S., Foo, S., Pang, N.: A framework for scientific paper retrieval and recommender systems. CoRR abs/1609.01415 (2016)

  13. Nart, D.D., Tasso, C.: A personalized concept-driven recommender system for scientific libraries. Procedia Comput. Sci. 38, 84 (2014).

    Article  Google Scholar 

  14. Tho, Q.T., Hui, S., Fong, A.: A citation-based document retrieval system for finding research expertise. Inf. Process. Manag. 43(1), 248 (2007).

    Article  Google Scholar 

  15. Poelmans, J., Ignatov, D.I., Viaene, S., Dedene, G., Kuznetsov, S.O.: Text Mining Scientific Papers: A Survey on FCA-Based Information Retrieval Research, pp. 273–287. Springer, Berlin (2012).

    Book  Google Scholar 

  16. Alencar, R.O.d., Zarate, L., Song, M.: Sphere-m: An ontology capture method, pp. 353–358 (2012).

  17. Sowa, J.F.: Knowledge Representation: Logical, Philosophical, and Computational Foundations, vol. 27. Brooks Cole Publishing Co., Pacific Grove (1999).

    Book  Google Scholar 

  18. Yevtushenko, S.A.: System of data analysis concept explorer. In: Proceedings of the 7th National Conference on Artificial Intelligence KII-2000, pp. 127–134 (2000)

  19. Neto, S.M., Zárate, L.E., Song, M.A.: Handling high dimensionality contexts in formal concept analysis via binary decision diagrams. Inf. Sci. 429, 361 (2018).

    Article  MathSciNet  MATH  Google Scholar 

  20. Rodrigues, M.W., Isotani, S., Zárate, L.E.: Educational data mining: a review of evaluation process in the e-learning. Telem. Inform. 35(6), 1701 (2018).

    Article  Google Scholar 

  21. Kitchenham, B., Pearl Brereton, O., Budgen, D., Turner, M., Bailey, J., Linkman, S.: Systematic literature reviews in software engineering - a systematic literature review. Inf. Software Technol. 51(1), 7 (2009).

    Article  Google Scholar 

  22. Chen, C.M., Chen, Y.Y., Liu, C.Y.: Learning performance assessment approach using web-based learning portfolios for e-learning systems. IEEE Trans. Syst. Man Cybern. C: Appl. Rev. 37(6), 1349 (2007).

    Article  Google Scholar 

  23. Erosa, V., Arroyo, P.: Technology and knowledge: enhancing the education frontiers, pp. 1501–1507 (2007).

  24. Liu, F.J., Shih, B.J.: Learning activity-based e-learning material recommendation system, pp. 343–348 (2007).

  25. Romero, C., Ventura, S.: Educational data mining: a survey from 1995 to 2005. Expert Syst. Appl. 33(1), 135 (2007).

    Article  Google Scholar 

  26. Richardson, B., Davis, K., Beach, M.: Introducing data mining techniques and software engineering to high school science students, pp. 1–6 (2008).

  27. Hongxia, J., Yao, H.: Classroom teaching quality evaluation based on neuro-fuzzy id3 algorithm 1, 166 (2008).

  28. Caballe, S., Xhafa, F., Abraham, A.: Towards an automatic real-time assessment of online discussions in computer-supported collaborative learning practices, pp. 470–475 (2008).

  29. Fernández-Luna, J.M., Huete, J.F., MacFarlane, A., Efthimiadis, E.N.: Teaching and learning in information retrieval. Inf. Retr. 12(2), 201 (2009).

    Article  Google Scholar 

  30. Hsieh, J.C., Chen, C.M., Lin, H.F.: Social interaction mining based on wireless sensor networks for promoting cooperative learning performance in classroom learning environment, pp. 219–221 (2010).

  31. Kechaou, Z., Ben Ammar, M., Alimi, A.: Improving e-learning with sentiment analysis of users’ opinions, pp. 1032–1038 (2011).

  32. Sachin, R., Vijay, M.: A survey and future vision of data mining in educational field, pp. 96–100 (2012).

  33. Ivancevic, V., Celikovic, M., Lukovic, I.: The individual stability of student spatial deployment and its implications, pp. 1–4 (2012)

  34. Li, Y., Mei, L., Wang, J.: A personalized recommendation system in e-learning environment based on semantic analysis, pp. 802–807 (2012)

  35. Baradwaj, B.K., Pal, S.: Mining educational data to analyze students’ performance. CoRR abs/1201.3417 (2012).

  36. Liu, Q., Peng, Y.: A method of unstructured information process in computer teaching evaluation system based on data mining technology, pp. 688–692 (2013).

  37. Leelathakul, N., Chaipah, K.: Quantitative effects of using facebook as a learning tool on students’ performance, pp. 87–92 (2013).

  38. Harfield, A., Jormanainen, I., Rungrattanaubol, J., Pattaranit, R.: An open monitoring environment for primary school children engaged in tablet-based learning, pp. 195–199 (2013).

  39. França, R.S.d., Amaral, H.J.C.d.: Mineraç ao de dados na identificaç ao de grupos de estudantes com dificuldades de aprendizagem no ensino de programaç ao, RENOTE: Revista Novas Tecnologias na Educaç ao 11(1) (2013).

  40. Huebner, R.A.: A survey of educational data-mining research. Res. High. Educ. J. 19(4) (2013).

  41. Jindal, R., Borah, M.D.: A survey on educational data mining and research trends. Int. J. Database Manag. Syst. 5(3), 53 (2013).

  42. Mohamad, S.K., Tasir, Z.: Educational data mining: a review. In: The 9th International Conference on Cognitive Science, vol. 97, p. 320 (2013). Procedia - Social and Behavioral Sciences

  43. Gray, G., McGuinness, C., Owende, P.: Investigating the efficacy of algorithmic student modelling in predicting students at risk of failing in tertiary education, pp. 378–380 (2013)

  44. Liu, Y.E., Mandel, T., Butler, E., Andersen, E., O’Rourke, E., Brunskill, E., Popović, Z.: Predicting player moves in an educational game a hybrid approach, pp. 106–113 (2013)

  45. Pardos, Z.A., Bergner, Y., Seaton, D.T., Pritchard, D.E.: Adapting bayesian knowledge tracing to a massive open online course in edx. In: D’Mello, S.K., Calvo, R.A., Olney A. (eds.) Proceedings of the 6th International Conference on Educational Data Mining, International Educational Data Mining Society, pp. 137–144 (2013)

  46. Maldonado, R.M., Yacef, K., Kay, J.: Data mining in the classroom discovering groups strategies at a multi-tabletop environment. In: Proceedings of the 6th International Conference on Educational Data Mining (Interntional Educational Data Mining Society), pp. 121–128 (2013)

  47. Almeda, M.V., Scupelli, P., Baker, R.S., Weber, M., Fisher, A.: Clustering of design decisions in classroom visual displays, pp. 44–48 (2014).

  48. Guleria, P., Sood, M.: Data mining in education: a review on the knowledge discovery perspective. Int. J. Data Min. Knowl. Manag. Process 4(5), 47 (2014).

    Article  Google Scholar 

  49. Paiva, R., Borges, D., Santos, J., Bittencourt, I.I., da Silva, A.P.: Lessons learned from an online open course: a Brazilian case study, pp. 229–234 (2014).

  50. Pena-Ayala, A.: Educational data mining: a survey and a data mining-based analysis of recent works. Expert Syst. Appl. 41(4), 1432 (2014).

    Article  Google Scholar 

  51. Tashakkori, R.M., Parry, R.M., Benoit, A., Cooper, R.A., Jenkins, J.L., Westveer, N.T.: Research experience for teachers: Data analysis & mining, visualization, and image processing, pp. 193–198 (2014).

  52. Guarín, C.E.L., Guzmán, E.L., González, F.A.: A model to predict low academic performance at a specific enrollment using data mining. IEEE Revista Iberoamericana de Tecnologias del Aprendizaje 10(3), 119 (2015).

    Article  Google Scholar 

  53. Heredia, D., Amaya, Y., Barrientos, E.: Student dropout predictive model using data mining techniques. IEEE Latin Am. Trans. 13(9), 3127 (2015).

    Article  Google Scholar 

  54. Ihantola, P., Vihavainen, A., Ahadi, A., Butler, M., Börstler, J., Edwards, S.H., Isohanni, E., Korhonen, A., Petersen, A., Rivers, K., Rubio, M.A., Sheard, J., Skupas, B., Spacco, J., Szabo, C., Toll, D.: Educational data mining and learning analytics in programming: Literature review and case studies, pp. 41–63 (2015).

  55. London, A., Pelyhe, A., Holló, C., Németh, T.: Applying graph-based data mining concepts to the educational sphere, pp. 358–365 (2015).

  56. In: O.C. Santos, J.G. Boticario, C. Romero, M. Pechenizkiy, A. Merceron, P. Mitros, J.M. Luna, C. Mihaescu, P. Moreno, A. Hershkovitz, S. Ventura, M. Desmarais (eds.) Proceedings of the 8th International conference on educational data mining (EDM’15), International Conference on Educational Data Mining (EDM) 2015 (International Educational Data Mining Society (IEDMS)), pp. 282–288

  57. Gramoli, V., Charleston, M., Jeffries, B., Koprinska, I., McGrane, M., Radu, A., Viglas, A., Yacef, K.: Mining autograding data in computer science education (2016).

  58. Romero, C., Ventura, S.: Educational data mining: a review of the state of the art. IEEE Trans. Syst. Man Cybern. C 40(6), 601 (2010).

    Article  Google Scholar 

  59. Ricarte, I.L.M., Junior, G.R.F.: Uma metodologia para a mineraç ao de dados oriundos de ambientes de aprendizagem apoiados por computadores. Revista Informática na Educaç ao: Teoria e Prática 14(2), 83 (2011)

    Google Scholar 

  60. Vasconcelos, F.H.L., da Silva, T.E.V., Mota, J.C.M.: Multilinear educational data analysis for evaluation of engineering education. IEEE Latin Am. Trans. 13(8), 2785 (2015).

    Article  Google Scholar 

  61. Gobert, J.D., Kim, Y.J., Pedro, M.A.S., Kennedy, M., Betts, C.G.: Using educational data mining to assess students’ skills at designing and conducting experiments within a complex systems microworld. Think. Skills Creat. 18, 81 (2015).

    Article  Google Scholar 

  62. Allen, L., McNamara, D.: You are your words: modeling students’ vocabulary knowledge with natural language processing techniques, pp. 258–265 (2015)

  63. Pedro, M.O.S., Snow, E., Baker, R., McNamara, D., Heffernan, N.: Exploring dynamical assessments of affect, behavior, and cognition and math state test achievement, pp. 85–92 (2015)

  64. Shahiri, A.M., Husain, W., Rashid, N.A.: A review on predicting student’s performance using data mining techniques. Procedia Comput. Sci. 72, 414 (2015).

    Article  Google Scholar 

  65. Campagni, R., Merlini, D., Sprugnoli, R., Verri, M.C.: Data mining models for student careers. Expert Syst. Appl. 42(13), 5508 (2015).

    Article  Google Scholar 

  66. Hung, Y.H., Chang, R.I., Lin, C.F.: Hybrid learning style identification and developing adaptive problem-solving learning activities. Comput. Hum. Behav. A 552, 55 (2016).

    Article  Google Scholar 

  67. Sabourin, J., Kosturko, L., Fitzgerald, C., Mcquiggan, S.: Student privacy and educational data mining: perspectives from industry. In: Santos, O.C., Boticario, J.G., Romero, C., Pechenizkiy, M., Merceron, A., Mitros, P., Luna, J.M., Mihaescu, C., Moreno, P., Hershkovitz, A., Ventura, S., Desmarais, M. (eds.) Proceedings of the 8th International Conference on Educational Data Mining (EDM’15), (International Educational Data Mining Society (IEDMS), pp. 164–170 (2015)

  68. Pedro, M.O.Z.S., Baker, R.S., Bowers, A.J., Heffernan, N.T.: Predicting college enrollment from student interaction with an intelligent tutoring system in middle school. In: D’Mello, S.K., Calvo, R.A. Olney, A. (eds.) Proceedings of the 6th international conference on educational data Mining (EDM’15), International Educational Data Mining Society, pp. 137–144 (2013)

  69. Olsen, J., Aleven, V., Rummel, N.: Predicting student performance in a collaborative learning environment. In: Santos, O.C., Boticario, J.G., Romero, C., Pechenizkiy, M., Merceron, A., Mitros, P., Luna, J.M., Mihaescu, C., Moreno, P., Hershkovitz, A., Ventura, S., Desmarais, M. (eds.) Proceedings of the 8th international conference on educational data mining (EDM’15), International Conference on Educational Data Mining (EDM) 2015, (International Educational Data Mining Society (IEDMS), pp. 211–217 (2015)

  70. Deerwester, S., Dumais, S.T., Furnas, G.W., Landauer, T.K., Harshman, R.: Indexing by latent semantic analysis. J. Am. Soc. Inf. Sci. 41(6), 391 (1990).<391::AID-ASI1>3.0.CO;2-9

  71. Bellegarda, J.R.: Latent semantic mapping. IEEE Signal Process. Mag. 22(5), 70 (2005).

    Article  Google Scholar 

  72. Landauer, T.K.: A solution to plato’ s problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychol Rev 104, 211 (1997). (1997)

  73. Recchia, G., Jones, M.N.: More data trumps smarter algorithms: Comparing pointwise mutual information with latent semantic analysis. Behavior Research Methods 41(3), 647 (2009).

    Article  Google Scholar 

  74. Budiu, R., Royer, C., Pirolli, P.L.: Modeling information scent: a 1446 comparison of lsa, pmi and glsa similarity measures on common 1447 tests and corpora. In: 8th Annual Conference of the Recherche d’Information Assistee Par Ordinateur (RIAO’), Pittsburgh, PA, USA (2005)

Download references


The authors acknowledge the financial support received from the CNPq (Brazilian National Council for Scientific and Technological Development), CAPES (Coordination for the Improvement of Higher Education Personnel), FAPEMIG (Foundation for Research Support of the State of Minas Gerais), and Pontifical Catholic University of Minas Gerais, Brazil.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Marcos Wander Rodrigues.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rodrigues, M.W., Zárate, L.E. OrgBR-M: a method to assist in organizing bibliographic material based on formal concept analysis—a case study in educational data mining. Int J Digit Libr 21, 423–448 (2020).

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: