Skip to main content

Advertisement

Log in

An effective deep learning pipeline for improved question classification into bloom’s taxonomy’s domains

  • Published:
Education and Information Technologies Aims and scope Submit manuscript

Abstract

Examination assessments undertaken by educational institutions are pivotal since it is one of the fundamental steps to determining students’ understanding and achievements for a distinct subject or course. Questions must be framed on the topics to meet the learning objectives and assess the student’s capability in a particular subject. The generation of examination questions from extensive text material is challenging and complicated. For example, massive volumes of textbooks make it time-consuming for faculties to annotate good-quality questions, keeping them manually well balanced. Thus, teachers rely on the Bloom’s taxonomy’s cognitive domain, a popular framework to assess students’ intellectual abilities. This study’s motivation is to propose a pipeline that could provide new questions from a given text corpus that could be retrieved from a particular input. These generated questions could be incorporated into a question recommender while being automatically classified under the specific cognitive domain under the Bloom’s taxonomy. Literature reviews showed that the work done over the Bloom’s taxonomy domain had obtained results by implementing classical machine learning methods and few with deep neural networks. The proposed network architectures have shown remarkable results and state-of-the-art architectures compared to the literature. This research study concluded that the pipeline is effective and significant in generating questions, like manually drafting questions, categorizing them into the Bloom’s taxonomy’s domains, and providing explicit content-based question recommendations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Data Availability

All the data are used in the article.

References

  • Aithal, S. G., Rao, A. B., & Singh, S. (2021). Automatic question-answer pairs generation and question similarity mechanism in Question answering system. Applied Intelligence. https://doi.org/10.1007/s10489-021-02348-9

    Article  Google Scholar 

  • Alami, N., Mallahi, M. E., Amakdouf, H., & Qjidaa, H. (2021). Hybrid method for text summarization based on statistical and semantic treatment. Multimedia Tools and Applications, 80(13), 19567–19600. https://doi.org/10.1007/s11042-021-10613-9

    Article  Google Scholar 

  • AlArfaj, A. A., & Mahmoud, H. A. H. (2022). An intelligent tree extractive text summarization deep learning. Computers Materials and Continua, 73(2), 4231–4244. https://doi.org/10.32604/cmc.2022.030090

    Article  Google Scholar 

  • Alstete, J. W., & Beutell, N. J. (2019). Business simulation and assurance of learning: Gender, academic major and business core course performance. Quality Assurance in Education, 27(4), 412–426. https://doi.org/10.1108/QAE-04-2018-0043

    Article  Google Scholar 

  • Balaha, H. M., & Saafan, M. M. (2021). Automatic exam correction framework (AECF) for the MCQS, essays, and equations matching. Ieee Access : Practical Innovations, Open Solutions, 9, 32368–32389. https://doi.org/10.1109/ACCESS.2021.3060940

    Article  Google Scholar 

  • Barbhuiya, A. A., Karsh, R. K., & Jain, R. (2021). CNN based feature extraction and classification for sign language. Multimedia Tools and Applications, 80(2), 3051–3069. https://doi.org/10.1007/s11042-020-09829-y

    Article  Google Scholar 

  • Bloom, B. S. (1956). Taxonomy of educational objectives. Vol. 1: Cognitive domain. New York: McKay, 20(24), p.1

  • Blšták, M., & Rozinajová, V. (2022). Automatic question generation based on sentence structure analysis using machine learning approach. Natural Language Engineering, 28(4), 487–517. https://doi.org/10.1017/S1351324921000139

    Article  Google Scholar 

  • Bogdanova, D., & Snoeck, M. (2019). CaMeLOT: An educational framework for conceptual data modelling. Information and Software Technology, 110, 92–107. https://doi.org/10.1016/j.infsof.2019.02.006

    Article  Google Scholar 

  • Boussakssou, M., Hssina, B., & Erittali, M. (2020). Towards an Adaptive E-learning System Based on Q-Learning Algorithm. Procedia Computer Science, 170, 1198–1203. https://doi.org/10.1016/j.procs.2020.03.028

    Article  Google Scholar 

  • Caprara, L., & Caprara, C. (2022). Effects of virtual learning environments: A scoping review of literature. Education and Information Technologies, 27(3), 3683–3722. https://doi.org/10.1007/s10639-021-10768-w

    Article  Google Scholar 

  • Chali, Y., Joty, S. R., & Hasan, S. A. (2009). Complex question answering: Unsupervised learning approaches and experiments. Journal of Artificial Intelligence Research, 35, 1–47. https://doi.org/10.1613/jair.2784

    Article  MathSciNet  MATH  Google Scholar 

  • Chang, W. C., & Chung, M. S. (2009). Automatic applying Bloom’s taxonomy to classify and analysis the cognition level of English question items, Pervasive Computing (JCPC), Joint Conferences. https://doi.org/10.1109/JCPC.2009.5420087

  • Chatzikonstantinou, C., Konstantinidis, D., Dimitropoulos, K., & Daras, P. (2021). Recurrent neural network pruning using dynamical systems and iterative fine-tuning. Neural Networks, 143, 475–488. https://doi.org/10.1016/j.neunet.2021.07.001

    Article  Google Scholar 

  • Chen, Y., & Li, H. (2020). DAM: Transformer-based relation detection for Question Answering over Knowledge Base. Knowledge-Based Systems,s 201–202, 106077. https://doi.org/10.1016/j.knosys.2020.106077

  • Chilukuri, K. C. (2020). A Novel Framework for Active Learning in Engineering Education Mapped to Course Outcomes. Procedia Computer Science, 172, 28–33. https://doi.org/10.1016/j.procs.2020.05.004

    Article  Google Scholar 

  • Cormack, S. H., Eagle, L. A., & Davies, M. S. (2020). A large-scale test of the relationship between procrastination and performance using learning analytics. Assessment and Evaluation in Higher Education, 45(7), 1046–1059. https://doi.org/10.1080/02602938.2019.1705244

    Article  Google Scholar 

  • Das, B., Majumder, M., Phadikar, S., & Sekh, A. A. (2019). Automatic generation of fill-in-the-blank question with corpus-based distractors for e-assessment to enhance learning. Computer Applications in Engineering Education, 27(6), 1485–1495. https://doi.org/10.1002/cae.22163

    Article  Google Scholar 

  • Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1, 4171–4186. https://doi.org/10.18653/v1/N19-1423

  • El Asame, M., Wakrim, M., & battou, A. (2022). Designing e-assessment activities appropriate to learner’s competency levels: Hybrid pedagogical framework and authoring tool. Education and Information Technologies, 27(2), 2543–2567. https://doi.org/10.1007/s10639-021-10607-y

    Article  Google Scholar 

  • Geetha, M. P., & Renuka, D. K. (2021). Improving the performance of aspect based sentiment analysis using fine-tuned Bert Base Uncased model. International Journal of Intelligent Networks, 2, 64–69. https://doi.org/10.1016/j.ijin.2021.06.005

    Article  Google Scholar 

  • Haris, S. S., & Omar, N. (2015). Bloom’s taxonomy question categorization using rules and N-gram approach. Journal of Theoretical and Applied Information Technology, 76(3), 401–407

    Google Scholar 

  • Harmon, O. R., Lambrinos, J., & Buffolino, J. (2010). Assessment Design and Cheating Risk in Online Instruction, Paper presented at the Online Journal of Distance Learning Administration, 13(3)

  • Heilman, M., & Smith, N. A. (2010). Good Question! statistical ranking for question generation, Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Association for Computational Linguistics, Los Angeles, California, 609–617

  • Hubalovsky, S., Hubalovska, M., & Musilek, M. (2019). Assessment of the influence of adaptive E-learning on learning effectiveness of primary school pupils. Computers in Human Behavior, 92, 691–705. https://doi.org/10.1016/j.chb.2018.05.033

    Article  Google Scholar 

  • Hudson, G. T., & Moubayed, N. A. (2021). Ask me in your own words: Paraphrasing for multitask question answering. PeerJ Computer Science, 7, 1–16. https://doi.org/10.7717/PEERJ-CS.759

    Article  Google Scholar 

  • Indurthi, S., Raghu, D., Khapra, M. M., & Joshi, S. (2017). Generating Natural Language Question-Answer Pairs from a Knowledge Graph Using a RNN Based Question Generation Model. Association for Computational Linguistics, Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, Association for Computational Linguistics, 376–385

  • Jayakodi, K., Bandara, M., Perera, I., & Meedeniya, D. (2016). WordNet and cosine similarity based classifier of exam questions using bloom’s taxonomy. International Journal of Emerging Technologies in Learning, 11(4), 142–149. https://doi.org/10.3991/ijet.v11i04.5654

    Article  Google Scholar 

  • Jonas, M., & Aditya, T. (2016). Siamese recurrent architectures for learning sentence similarity, Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, 2786–2792. https://doi.org/10.1609/aaai.v30i1.10350

  • Khodeir, N. A., Elazhary, H., & Wanas, N. (2018). Generating story problems via controlled parameters in a web-based intelligent tutoring system. International Journal of Information and Learning Technology, 35(3), 199–216. https://doi.org/10.1108/IJILT-09-2017-0085

    Article  Google Scholar 

  • Khurana, A., & Bhatnagar, V. (2022). Investigating entropy for extractive document summarization. Expert Systems with Applications, 187, https://doi.org/10.1016/j.eswa.2021.115820

  • Lamsiyah, S., Mahdaouy, E., Alaoui, A. O. E., S., & Espinasse, B. (2021). Unsupervised query-focused multi-document summarization based on transfer learning from sentence embedding models, BM25 model, and maximal marginal relevance criterion. Journal of Ambient Intelligence and Humanized Computing. https://doi.org/10.1007/s12652-021-03165-1

    Article  Google Scholar 

  • Lang, Q., Liu, X., & Deng, Y. (2021). Multi-level retrieval with semantic axiomatic fuzzy set clustering for question answering. Applied Soft Computing, 111. https://doi.org/10.1016/j.asoc.2021.107858

  • Le, N. T., Kojiri, T., & Pinkwart, N. (2014). Automatic Question Generation for Educational Applications – The State of Art. Advanced Computational Methods for Knowledge Engineering, 282, 325–338. https://doi.org/10.1007/978-3-319-06569-4_24

    Article  Google Scholar 

  • Lindberg, D., Popowich, F., Nesbit, J., & Winne, P. (2013). Generating natural language questions to support learning on-line, In Proceedings of the 14th European Workshop on Natural Language Generation, Association for Computational Linguistics, Sofia, Bulgaria. 105–114

  • Lu, W., Yu, R., Wang, S., Wang, C., Jian, P., & Huang, H. (2021). Sentence semantic matching based on 3D CNN for human robot language interaction. ACM Transactions on Internet Technology, 21(4), https://doi.org/10.1145/3450520

  • Marina, A., Sina, S., Bas, D., & Amir, H. P. A. (2021). Siamese Neural Networks for Detecting Complementary Products, Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop, Association for Computational Linguistics, 65–70

  • Mohammed, M., & Omar, N. (2020). Question classification based on Bloom’s taxonomy cognitive domain using modified TF-IDF and word2vec. Plos One, 15(3), e0230442. https://doi.org/10.1371/journal.pone.0230442

    Article  Google Scholar 

  • Mohasseb, A., Bader-El-Den, M., & Cocea, M. (2018). Question categorization and classification using grammar based approach. Information Processing & Management, 54(6), 1228–1243. https://doi.org/10.1016/j.ipm.2018.05.001

    Article  Google Scholar 

  • Mueller, M., & Thyagarajan, A. (2016). Siamese recurrent architectures for learning sentence similarity, Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, 2786–2792. https://doi.org/10.1609/aaai.v30i1.10350

  • NCERT, & Textbooks Class, X. I. I. B. (August 2021). Physics, Chemistry, History. https://ncert.nic.in/. Accessed on 5th

  • Nguyen, H. T., Duong, P. H., & Cambria, E. (2019). Learning short-text semantic similarity with word embeddings and external knowledge sources. Knowledge-Based Systems, 182, 104842. https://doi.org/10.1016/j.knosys.2019.07.013

    Article  Google Scholar 

  • Olson, A. W., Calderón-Figueroa, F., Bidian, O., Silver, D., & Sanner, S. (2021). Reading the city through its neighbourhoods: Deep text embeddings of Yelp reviews as a basis for determining similarity and change. Cities, 110, 103045. https://doi.org/10.1016/j.cities.2020.103045

    Article  Google Scholar 

  • Omar, N., Haris, S. S., Hassan, R., Arshad, H., Rahmat, M., Zainal, N. F. A., & Zulkifli, R. (2012). Automated Analysis of Exam Questions According to Bloom’s Taxonomy. Procedia - Social and Behavioral Sciences, 59, 297–303. https://doi.org/10.1016/j.sbspro.2012.09.278

    Article  Google Scholar 

  • Pal, S., Chang, M., & Iriarte, M. F. (2022). Summary generation using natural language processing techniques and cosine similarity. Intelligent Systems Design and Applications (ISDA), Lecture Notes in Networks and Systems.https://doi.org/10.1007/978-3-030-96308-8_47

  • Palivela, H. (2021). Optimization of paraphrase generation and identification using language models in natural language processing. International Journal of Information Management Data Insights, 1(2), https://doi.org/10.1016/j.jjimei.2021.100025

  • Poorman, S. G., & Mastorovich, M. L. (2020). Constructing Next Generation National Council Licensure Examination (NCLEX) (NGN) Style Questions: Help for Faculty. Teaching and Learning in Nursing, 15(1), 86–91. https://doi.org/10.1016/j.teln.2019.08.008

  • Priya, T. J., Priya, K. P. S., Jenneyl, L. R., & Uma, K. V. (2022). Automatic question generation from video. Computational Intelligence in Pattern Recognition (CIPR), Lecture Notes in Networks and Systems.https://doi.org/10.1007/978-981-19-3089-8_35

  • Qiu, X., & Huang, X. (2015). Convolutional neural tensor network architecture for community-based question answering. IJCAI’15: Proceedings of the 24th International Conference on Artificial Intelligence. https://doi.org/10.5555/2832415

  • Quan, P., Shi, Y., Niu, L., Liu, Y., & Zhang, T. (2018). Automatic Chinese Multiple-Choice Question Generation for Human Resource Performance Appraisal. Procedia Computer Science, 139, 165–172. https://doi.org/10.1016/j.procs.2018.10.235

    Article  Google Scholar 

  • Radmehr, F., & Drake, M. (2018). An assessment-based model for exploring the solving of mathematical problems: Utilizing revised Bloom’s taxonomy and facets of metacognition. Studies in Educational Evaluation, 59, 41–51. https://doi.org/10.1016/j.stueduc.2018.02.004

    Article  Google Scholar 

  • Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2020). Exploring Transfer Learning with T5: the Text-To-Text Transfer Transformer. Journal of Machine Learning Research, 21, 1–67

    MATH  Google Scholar 

  • Rajpurkar, P., Jia, R., & Liang, P. (2018). Know What You Don’t Know: Unanswerable Questions for SQuAD. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2: 784–789. https://doi.org/10.18653/v1/P18-2124

  • Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016). SQuAD: 100,000 + questions for machine comprehension of text, Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 2383–2392. https://doi.org/10.18653/v1/D16-1264

  • Ramnarain-Seetohul, V., Bassoo, V., & Rosunally, Y. (2022). Similarity measures in automated essay scoring systems: A ten-year review. Education and Information Technologies, 27(4), 5573–5604. https://doi.org/10.1007/s10639-021-10838-z

    Article  Google Scholar 

  • Ramos, I. M., Ramos, D. B., Gadelha, B. F., & De Oliveira, E. H. T. (2021). An approach to group formation in collaborative learning using learning paths in learning management systems. IEEE Transactions on Learning Technologies, 14(5), 555–567. https://doi.org/10.1109/TLT.2021.3117916

    Article  Google Scholar 

  • Ray, S. K., Singh, S., & Joshi, B. P. (2010). A semantic approach for question classification using WordNet and Wikipedia. Pattern Recognition Letters, 31(13), 1935–1943. https://doi.org/10.1016/j.patrec.2010.06.012

    Article  Google Scholar 

  • Saadullah, S. M., & Elsayed, N. (2020). An audit simulation of the substantive procedures in the revenue process – A teaching case incorporating Bloom’s taxonomy. Journal of Accounting Education, 52, 100678. https://doi.org/10.1016/j.jaccedu.2020.100678

    Article  Google Scholar 

  • Saedi, C., & Dras, M. (2021). Siamese networks for large-scale author identification. Computer Speech & Language. https://doi.org/10.1016/j.csl.2021.101241. 70,101241

    Article  Google Scholar 

  • Samiappan, D., & Chakrapani, V. (2013). Classification of ultrasound carotid artery images using texture features. International Review on Computers and Software, 8(4), 933–940

    Google Scholar 

  • Singh, J., & Sharma, Y. (2018). Encoder-Decoder Architectures for Generating Questions. Procedia Computer Science, 132, 1041–1048. https://doi.org/10.1016/j.procs.2018.05.019

    Article  Google Scholar 

  • Singh, R., Timbadia, D., Kapoor, V., Reddy, R., Churi, P., & Pimple, O. (2021). Question paper generation through progressive model and difficulty calculation on the promexa mobile application. Education and Information Technologies, 26(4), 4151–4179. https://doi.org/10.1007/s10639-021-10461-y

    Article  Google Scholar 

  • Stanescu, L., Spahiu, C. S., Udristoiu, A. I., & Spahiu, A. (2008). Question Generation for Learning Evaluation, Proceedings of the International Multiconference on Computer Science and Information Technology, IMCSIT, Wisla, Poland.https://doi.org/10.1109/IMCSIT.2008.4747291

  • Takano, Y., & Kajikawa, Y. (2019). Extracting commercialization opportunities of the Internet of Things: Measuring text similarity between papers and patents. Technological Forecasting and Social Change, 138, 45–68. https://doi.org/10.1016/j.techfore.2018.08.008

    Article  Google Scholar 

  • Van Hoeij, M. J., Haarhuls, J. C. M., Wierstra, R. F., & van Beukelen, P. (2004). Developing a classification tool based on Bloom’s taxonomy to assess the cognitive level of short essay questions. Journal of veterinary medical education, 261–267. https://doi.org/10.3138/jvme.31.3.261

  • Waite, L. H., Zupec, J. F., Quinn, D. H., & Poon, C. Y. (2020). Revised Bloom’s taxonomy as a mentoring framework for successful promotion. Currents in Pharmacy Teaching and Learning, 12(11), 1379–1382. https://doi.org/10.1016/j.cptl.2020.06.009

    Article  Google Scholar 

  • Wasim, M., Asim, M. N., Khan, M. U. G., & Mahmood, W. (2019). Multi-label biomedical question classification for lexical answer type prediction. Journal of Biomedical Informatics, 93, 103143. https://doi.org/10.1016/j.jbi.2019.103143

    Article  Google Scholar 

  • Wijanarko, B. D., Heryadi, Y., Toba, H., & Budiharto, W. (2021). Question generation model based on key-phrase, context-free grammar, and Bloom’s taxonomy. Education and Information Technologies, 26(2), 2207–2223. https://doi.org/10.1007/s10639-020-10356-4

    Article  Google Scholar 

  • Yahya, A. A., Toukal, Z., & Osman, A. (2012). Bloom’s Taxonomy—Based Classification for Item Bank Questions Using Support Vector Machines Bloom’s Taxonomy—Based Classification for Item Bank. Modern Advances in Intelligent Systems and Tools, SCI 431, Springer-Verlag, Berlin Heidelberg, 135–140. https://doi.org/10.1007/978-3-642-30732-4_17

  • Yang, J., Li, Y., Gao, C., & Zhang, Y. (2021). Measuring the short text similarity based on semantic and syntactic information. Future Generation Computer Systems, 114, 169–180. https://doi.org/10.1016/j.future.2020.07.043

    Article  Google Scholar 

  • Yeoh, P. S. Q., Lai, K. W., Goh, S. L., Hasikin, K., Hum, Y. C., Tee, Y. K., & Dhanalakshmi, S. (2021). Emergence of deep learning in knee osteoarthritis diagnosis. Computational Intelligence and Neuroscience. https://doi.org/10.1155/2021/4931437

    Article  Google Scholar 

  • Zhang, J., Rong, W., Chen, D., & Xiong, Z. (2022). Question type and answer related keywords aware question generation. Journal of Intelligent and Fuzzy Systems, 42(5), 4611–4622. https://doi.org/10.3233/JIFS-219249

    Article  Google Scholar 

  • Zhou, Q., Yang, N., Wei, F., Tan, C., Bao, H., & Zhou, M. (2017). Neural Question Generation from Text: A Preliminary Study. Huang X., Jiang J., Zhao D., Feng Y., Hong Y. (eds) Natural Language Processing and Chinese Computing, NLPCC.https://doi.org/10.1007/978-3-319-73618-1

Download references

Acknowledgements

The authors are grateful to the SRM Institute of Science and Technology, Kattankulathur Campus, Chennai, for supplying the required research facility.

Funding

This work received no specific grant from any funding agency in public, commercial or not-for-profit sectors.

Author information

Authors and Affiliations

Authors

Contributions

Harsh Sharma: Conceptualization, Formal analysis, Methodology, Investigation, Original Draft-writing, Rohan Mathur: Conceptualization, Formal analysis, Methodology, Investigation, Original Draft-writing, Tejas Chintala: Conceptualization, Formal analysis, Methodology, Investigation, Original Draft-writing, Samiappan Dhanalakshmi: Investigation, Formal analysis, Methodology, Data visualization, Data validation, Writing - review & editing, Supervision, Ramalingam Senthil: Investigation, Methodology, Data curation, Data validation, Writing - review & editing.

Corresponding author

Correspondence to Samiappan Dhanalakshmi.

Ethics declarations

Compliance with ethical standards

Ethical clearance is not applicable.

Conflict of interest

The authors declare no conflicts of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

+ Harsh Sharma, Rohan Mathur and Tejas Chintala, contributions are equal.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sharma, H., Mathur, R., Chintala, T. et al. An effective deep learning pipeline for improved question classification into bloom’s taxonomy’s domains. Educ Inf Technol 28, 5105–5145 (2023). https://doi.org/10.1007/s10639-022-11356-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10639-022-11356-2

Keywords

Navigation