Abstract
Examination assessments undertaken by educational institutions are pivotal since it is one of the fundamental steps to determining students’ understanding and achievements for a distinct subject or course. Questions must be framed on the topics to meet the learning objectives and assess the student’s capability in a particular subject. The generation of examination questions from extensive text material is challenging and complicated. For example, massive volumes of textbooks make it time-consuming for faculties to annotate good-quality questions, keeping them manually well balanced. Thus, teachers rely on the Bloom’s taxonomy’s cognitive domain, a popular framework to assess students’ intellectual abilities. This study’s motivation is to propose a pipeline that could provide new questions from a given text corpus that could be retrieved from a particular input. These generated questions could be incorporated into a question recommender while being automatically classified under the specific cognitive domain under the Bloom’s taxonomy. Literature reviews showed that the work done over the Bloom’s taxonomy domain had obtained results by implementing classical machine learning methods and few with deep neural networks. The proposed network architectures have shown remarkable results and state-of-the-art architectures compared to the literature. This research study concluded that the pipeline is effective and significant in generating questions, like manually drafting questions, categorizing them into the Bloom’s taxonomy’s domains, and providing explicit content-based question recommendations.
Similar content being viewed by others
Data Availability
All the data are used in the article.
References
Aithal, S. G., Rao, A. B., & Singh, S. (2021). Automatic question-answer pairs generation and question similarity mechanism in Question answering system. Applied Intelligence. https://doi.org/10.1007/s10489-021-02348-9
Alami, N., Mallahi, M. E., Amakdouf, H., & Qjidaa, H. (2021). Hybrid method for text summarization based on statistical and semantic treatment. Multimedia Tools and Applications, 80(13), 19567–19600. https://doi.org/10.1007/s11042-021-10613-9
AlArfaj, A. A., & Mahmoud, H. A. H. (2022). An intelligent tree extractive text summarization deep learning. Computers Materials and Continua, 73(2), 4231–4244. https://doi.org/10.32604/cmc.2022.030090
Alstete, J. W., & Beutell, N. J. (2019). Business simulation and assurance of learning: Gender, academic major and business core course performance. Quality Assurance in Education, 27(4), 412–426. https://doi.org/10.1108/QAE-04-2018-0043
Balaha, H. M., & Saafan, M. M. (2021). Automatic exam correction framework (AECF) for the MCQS, essays, and equations matching. Ieee Access : Practical Innovations, Open Solutions, 9, 32368–32389. https://doi.org/10.1109/ACCESS.2021.3060940
Barbhuiya, A. A., Karsh, R. K., & Jain, R. (2021). CNN based feature extraction and classification for sign language. Multimedia Tools and Applications, 80(2), 3051–3069. https://doi.org/10.1007/s11042-020-09829-y
Bloom, B. S. (1956). Taxonomy of educational objectives. Vol. 1: Cognitive domain. New York: McKay, 20(24), p.1
Blšták, M., & Rozinajová, V. (2022). Automatic question generation based on sentence structure analysis using machine learning approach. Natural Language Engineering, 28(4), 487–517. https://doi.org/10.1017/S1351324921000139
Bogdanova, D., & Snoeck, M. (2019). CaMeLOT: An educational framework for conceptual data modelling. Information and Software Technology, 110, 92–107. https://doi.org/10.1016/j.infsof.2019.02.006
Boussakssou, M., Hssina, B., & Erittali, M. (2020). Towards an Adaptive E-learning System Based on Q-Learning Algorithm. Procedia Computer Science, 170, 1198–1203. https://doi.org/10.1016/j.procs.2020.03.028
Caprara, L., & Caprara, C. (2022). Effects of virtual learning environments: A scoping review of literature. Education and Information Technologies, 27(3), 3683–3722. https://doi.org/10.1007/s10639-021-10768-w
Chali, Y., Joty, S. R., & Hasan, S. A. (2009). Complex question answering: Unsupervised learning approaches and experiments. Journal of Artificial Intelligence Research, 35, 1–47. https://doi.org/10.1613/jair.2784
Chang, W. C., & Chung, M. S. (2009). Automatic applying Bloom’s taxonomy to classify and analysis the cognition level of English question items, Pervasive Computing (JCPC), Joint Conferences. https://doi.org/10.1109/JCPC.2009.5420087
Chatzikonstantinou, C., Konstantinidis, D., Dimitropoulos, K., & Daras, P. (2021). Recurrent neural network pruning using dynamical systems and iterative fine-tuning. Neural Networks, 143, 475–488. https://doi.org/10.1016/j.neunet.2021.07.001
Chen, Y., & Li, H. (2020). DAM: Transformer-based relation detection for Question Answering over Knowledge Base. Knowledge-Based Systems,s 201–202, 106077. https://doi.org/10.1016/j.knosys.2020.106077
Chilukuri, K. C. (2020). A Novel Framework for Active Learning in Engineering Education Mapped to Course Outcomes. Procedia Computer Science, 172, 28–33. https://doi.org/10.1016/j.procs.2020.05.004
Cormack, S. H., Eagle, L. A., & Davies, M. S. (2020). A large-scale test of the relationship between procrastination and performance using learning analytics. Assessment and Evaluation in Higher Education, 45(7), 1046–1059. https://doi.org/10.1080/02602938.2019.1705244
Das, B., Majumder, M., Phadikar, S., & Sekh, A. A. (2019). Automatic generation of fill-in-the-blank question with corpus-based distractors for e-assessment to enhance learning. Computer Applications in Engineering Education, 27(6), 1485–1495. https://doi.org/10.1002/cae.22163
Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1, 4171–4186. https://doi.org/10.18653/v1/N19-1423
El Asame, M., Wakrim, M., & battou, A. (2022). Designing e-assessment activities appropriate to learner’s competency levels: Hybrid pedagogical framework and authoring tool. Education and Information Technologies, 27(2), 2543–2567. https://doi.org/10.1007/s10639-021-10607-y
Geetha, M. P., & Renuka, D. K. (2021). Improving the performance of aspect based sentiment analysis using fine-tuned Bert Base Uncased model. International Journal of Intelligent Networks, 2, 64–69. https://doi.org/10.1016/j.ijin.2021.06.005
Haris, S. S., & Omar, N. (2015). Bloom’s taxonomy question categorization using rules and N-gram approach. Journal of Theoretical and Applied Information Technology, 76(3), 401–407
Harmon, O. R., Lambrinos, J., & Buffolino, J. (2010). Assessment Design and Cheating Risk in Online Instruction, Paper presented at the Online Journal of Distance Learning Administration, 13(3)
Heilman, M., & Smith, N. A. (2010). Good Question! statistical ranking for question generation, Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Association for Computational Linguistics, Los Angeles, California, 609–617
Hubalovsky, S., Hubalovska, M., & Musilek, M. (2019). Assessment of the influence of adaptive E-learning on learning effectiveness of primary school pupils. Computers in Human Behavior, 92, 691–705. https://doi.org/10.1016/j.chb.2018.05.033
Hudson, G. T., & Moubayed, N. A. (2021). Ask me in your own words: Paraphrasing for multitask question answering. PeerJ Computer Science, 7, 1–16. https://doi.org/10.7717/PEERJ-CS.759
Indurthi, S., Raghu, D., Khapra, M. M., & Joshi, S. (2017). Generating Natural Language Question-Answer Pairs from a Knowledge Graph Using a RNN Based Question Generation Model. Association for Computational Linguistics, Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, Association for Computational Linguistics, 376–385
Jayakodi, K., Bandara, M., Perera, I., & Meedeniya, D. (2016). WordNet and cosine similarity based classifier of exam questions using bloom’s taxonomy. International Journal of Emerging Technologies in Learning, 11(4), 142–149. https://doi.org/10.3991/ijet.v11i04.5654
Jonas, M., & Aditya, T. (2016). Siamese recurrent architectures for learning sentence similarity, Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, 2786–2792. https://doi.org/10.1609/aaai.v30i1.10350
Khodeir, N. A., Elazhary, H., & Wanas, N. (2018). Generating story problems via controlled parameters in a web-based intelligent tutoring system. International Journal of Information and Learning Technology, 35(3), 199–216. https://doi.org/10.1108/IJILT-09-2017-0085
Khurana, A., & Bhatnagar, V. (2022). Investigating entropy for extractive document summarization. Expert Systems with Applications, 187, https://doi.org/10.1016/j.eswa.2021.115820
Lamsiyah, S., Mahdaouy, E., Alaoui, A. O. E., S., & Espinasse, B. (2021). Unsupervised query-focused multi-document summarization based on transfer learning from sentence embedding models, BM25 model, and maximal marginal relevance criterion. Journal of Ambient Intelligence and Humanized Computing. https://doi.org/10.1007/s12652-021-03165-1
Lang, Q., Liu, X., & Deng, Y. (2021). Multi-level retrieval with semantic axiomatic fuzzy set clustering for question answering. Applied Soft Computing, 111. https://doi.org/10.1016/j.asoc.2021.107858
Le, N. T., Kojiri, T., & Pinkwart, N. (2014). Automatic Question Generation for Educational Applications – The State of Art. Advanced Computational Methods for Knowledge Engineering, 282, 325–338. https://doi.org/10.1007/978-3-319-06569-4_24
Lindberg, D., Popowich, F., Nesbit, J., & Winne, P. (2013). Generating natural language questions to support learning on-line, In Proceedings of the 14th European Workshop on Natural Language Generation, Association for Computational Linguistics, Sofia, Bulgaria. 105–114
Lu, W., Yu, R., Wang, S., Wang, C., Jian, P., & Huang, H. (2021). Sentence semantic matching based on 3D CNN for human robot language interaction. ACM Transactions on Internet Technology, 21(4), https://doi.org/10.1145/3450520
Marina, A., Sina, S., Bas, D., & Amir, H. P. A. (2021). Siamese Neural Networks for Detecting Complementary Products, Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop, Association for Computational Linguistics, 65–70
Mohammed, M., & Omar, N. (2020). Question classification based on Bloom’s taxonomy cognitive domain using modified TF-IDF and word2vec. Plos One, 15(3), e0230442. https://doi.org/10.1371/journal.pone.0230442
Mohasseb, A., Bader-El-Den, M., & Cocea, M. (2018). Question categorization and classification using grammar based approach. Information Processing & Management, 54(6), 1228–1243. https://doi.org/10.1016/j.ipm.2018.05.001
Mueller, M., & Thyagarajan, A. (2016). Siamese recurrent architectures for learning sentence similarity, Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, 2786–2792. https://doi.org/10.1609/aaai.v30i1.10350
NCERT, & Textbooks Class, X. I. I. B. (August 2021). Physics, Chemistry, History. https://ncert.nic.in/. Accessed on 5th
Nguyen, H. T., Duong, P. H., & Cambria, E. (2019). Learning short-text semantic similarity with word embeddings and external knowledge sources. Knowledge-Based Systems, 182, 104842. https://doi.org/10.1016/j.knosys.2019.07.013
Olson, A. W., Calderón-Figueroa, F., Bidian, O., Silver, D., & Sanner, S. (2021). Reading the city through its neighbourhoods: Deep text embeddings of Yelp reviews as a basis for determining similarity and change. Cities, 110, 103045. https://doi.org/10.1016/j.cities.2020.103045
Omar, N., Haris, S. S., Hassan, R., Arshad, H., Rahmat, M., Zainal, N. F. A., & Zulkifli, R. (2012). Automated Analysis of Exam Questions According to Bloom’s Taxonomy. Procedia - Social and Behavioral Sciences, 59, 297–303. https://doi.org/10.1016/j.sbspro.2012.09.278
Pal, S., Chang, M., & Iriarte, M. F. (2022). Summary generation using natural language processing techniques and cosine similarity. Intelligent Systems Design and Applications (ISDA), Lecture Notes in Networks and Systems.https://doi.org/10.1007/978-3-030-96308-8_47
Palivela, H. (2021). Optimization of paraphrase generation and identification using language models in natural language processing. International Journal of Information Management Data Insights, 1(2), https://doi.org/10.1016/j.jjimei.2021.100025
Poorman, S. G., & Mastorovich, M. L. (2020). Constructing Next Generation National Council Licensure Examination (NCLEX) (NGN) Style Questions: Help for Faculty. Teaching and Learning in Nursing, 15(1), 86–91. https://doi.org/10.1016/j.teln.2019.08.008
Priya, T. J., Priya, K. P. S., Jenneyl, L. R., & Uma, K. V. (2022). Automatic question generation from video. Computational Intelligence in Pattern Recognition (CIPR), Lecture Notes in Networks and Systems.https://doi.org/10.1007/978-981-19-3089-8_35
Qiu, X., & Huang, X. (2015). Convolutional neural tensor network architecture for community-based question answering. IJCAI’15: Proceedings of the 24th International Conference on Artificial Intelligence. https://doi.org/10.5555/2832415
Quan, P., Shi, Y., Niu, L., Liu, Y., & Zhang, T. (2018). Automatic Chinese Multiple-Choice Question Generation for Human Resource Performance Appraisal. Procedia Computer Science, 139, 165–172. https://doi.org/10.1016/j.procs.2018.10.235
Radmehr, F., & Drake, M. (2018). An assessment-based model for exploring the solving of mathematical problems: Utilizing revised Bloom’s taxonomy and facets of metacognition. Studies in Educational Evaluation, 59, 41–51. https://doi.org/10.1016/j.stueduc.2018.02.004
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2020). Exploring Transfer Learning with T5: the Text-To-Text Transfer Transformer. Journal of Machine Learning Research, 21, 1–67
Rajpurkar, P., Jia, R., & Liang, P. (2018). Know What You Don’t Know: Unanswerable Questions for SQuAD. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2: 784–789. https://doi.org/10.18653/v1/P18-2124
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016). SQuAD: 100,000 + questions for machine comprehension of text, Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 2383–2392. https://doi.org/10.18653/v1/D16-1264
Ramnarain-Seetohul, V., Bassoo, V., & Rosunally, Y. (2022). Similarity measures in automated essay scoring systems: A ten-year review. Education and Information Technologies, 27(4), 5573–5604. https://doi.org/10.1007/s10639-021-10838-z
Ramos, I. M., Ramos, D. B., Gadelha, B. F., & De Oliveira, E. H. T. (2021). An approach to group formation in collaborative learning using learning paths in learning management systems. IEEE Transactions on Learning Technologies, 14(5), 555–567. https://doi.org/10.1109/TLT.2021.3117916
Ray, S. K., Singh, S., & Joshi, B. P. (2010). A semantic approach for question classification using WordNet and Wikipedia. Pattern Recognition Letters, 31(13), 1935–1943. https://doi.org/10.1016/j.patrec.2010.06.012
Saadullah, S. M., & Elsayed, N. (2020). An audit simulation of the substantive procedures in the revenue process – A teaching case incorporating Bloom’s taxonomy. Journal of Accounting Education, 52, 100678. https://doi.org/10.1016/j.jaccedu.2020.100678
Saedi, C., & Dras, M. (2021). Siamese networks for large-scale author identification. Computer Speech & Language. https://doi.org/10.1016/j.csl.2021.101241. 70,101241
Samiappan, D., & Chakrapani, V. (2013). Classification of ultrasound carotid artery images using texture features. International Review on Computers and Software, 8(4), 933–940
Singh, J., & Sharma, Y. (2018). Encoder-Decoder Architectures for Generating Questions. Procedia Computer Science, 132, 1041–1048. https://doi.org/10.1016/j.procs.2018.05.019
Singh, R., Timbadia, D., Kapoor, V., Reddy, R., Churi, P., & Pimple, O. (2021). Question paper generation through progressive model and difficulty calculation on the promexa mobile application. Education and Information Technologies, 26(4), 4151–4179. https://doi.org/10.1007/s10639-021-10461-y
Stanescu, L., Spahiu, C. S., Udristoiu, A. I., & Spahiu, A. (2008). Question Generation for Learning Evaluation, Proceedings of the International Multiconference on Computer Science and Information Technology, IMCSIT, Wisla, Poland.https://doi.org/10.1109/IMCSIT.2008.4747291
Takano, Y., & Kajikawa, Y. (2019). Extracting commercialization opportunities of the Internet of Things: Measuring text similarity between papers and patents. Technological Forecasting and Social Change, 138, 45–68. https://doi.org/10.1016/j.techfore.2018.08.008
Van Hoeij, M. J., Haarhuls, J. C. M., Wierstra, R. F., & van Beukelen, P. (2004). Developing a classification tool based on Bloom’s taxonomy to assess the cognitive level of short essay questions. Journal of veterinary medical education, 261–267. https://doi.org/10.3138/jvme.31.3.261
Waite, L. H., Zupec, J. F., Quinn, D. H., & Poon, C. Y. (2020). Revised Bloom’s taxonomy as a mentoring framework for successful promotion. Currents in Pharmacy Teaching and Learning, 12(11), 1379–1382. https://doi.org/10.1016/j.cptl.2020.06.009
Wasim, M., Asim, M. N., Khan, M. U. G., & Mahmood, W. (2019). Multi-label biomedical question classification for lexical answer type prediction. Journal of Biomedical Informatics, 93, 103143. https://doi.org/10.1016/j.jbi.2019.103143
Wijanarko, B. D., Heryadi, Y., Toba, H., & Budiharto, W. (2021). Question generation model based on key-phrase, context-free grammar, and Bloom’s taxonomy. Education and Information Technologies, 26(2), 2207–2223. https://doi.org/10.1007/s10639-020-10356-4
Yahya, A. A., Toukal, Z., & Osman, A. (2012). Bloom’s Taxonomy—Based Classification for Item Bank Questions Using Support Vector Machines Bloom’s Taxonomy—Based Classification for Item Bank. Modern Advances in Intelligent Systems and Tools, SCI 431, Springer-Verlag, Berlin Heidelberg, 135–140. https://doi.org/10.1007/978-3-642-30732-4_17
Yang, J., Li, Y., Gao, C., & Zhang, Y. (2021). Measuring the short text similarity based on semantic and syntactic information. Future Generation Computer Systems, 114, 169–180. https://doi.org/10.1016/j.future.2020.07.043
Yeoh, P. S. Q., Lai, K. W., Goh, S. L., Hasikin, K., Hum, Y. C., Tee, Y. K., & Dhanalakshmi, S. (2021). Emergence of deep learning in knee osteoarthritis diagnosis. Computational Intelligence and Neuroscience. https://doi.org/10.1155/2021/4931437
Zhang, J., Rong, W., Chen, D., & Xiong, Z. (2022). Question type and answer related keywords aware question generation. Journal of Intelligent and Fuzzy Systems, 42(5), 4611–4622. https://doi.org/10.3233/JIFS-219249
Zhou, Q., Yang, N., Wei, F., Tan, C., Bao, H., & Zhou, M. (2017). Neural Question Generation from Text: A Preliminary Study. Huang X., Jiang J., Zhao D., Feng Y., Hong Y. (eds) Natural Language Processing and Chinese Computing, NLPCC.https://doi.org/10.1007/978-3-319-73618-1
Acknowledgements
The authors are grateful to the SRM Institute of Science and Technology, Kattankulathur Campus, Chennai, for supplying the required research facility.
Funding
This work received no specific grant from any funding agency in public, commercial or not-for-profit sectors.
Author information
Authors and Affiliations
Contributions
Harsh Sharma: Conceptualization, Formal analysis, Methodology, Investigation, Original Draft-writing, Rohan Mathur: Conceptualization, Formal analysis, Methodology, Investigation, Original Draft-writing, Tejas Chintala: Conceptualization, Formal analysis, Methodology, Investigation, Original Draft-writing, Samiappan Dhanalakshmi: Investigation, Formal analysis, Methodology, Data visualization, Data validation, Writing - review & editing, Supervision, Ramalingam Senthil: Investigation, Methodology, Data curation, Data validation, Writing - review & editing.
Corresponding author
Ethics declarations
Compliance with ethical standards
Ethical clearance is not applicable.
Conflict of interest
The authors declare no conflicts of interest.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
+ Harsh Sharma, Rohan Mathur and Tejas Chintala, contributions are equal.
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Sharma, H., Mathur, R., Chintala, T. et al. An effective deep learning pipeline for improved question classification into bloom’s taxonomy’s domains. Educ Inf Technol 28, 5105–5145 (2023). https://doi.org/10.1007/s10639-022-11356-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10639-022-11356-2