Skip to main content
SpringerLink
Log in
Book cover

5th International Symposium on Data Mining Applications pp 46–60Cite as

A Benchmark Collection for Mapping Program Educational Objectives to ABET Student Outcomes: Accreditation

Part of the Advances in Intelligent Systems and Computing book series (AISC,volume 753)

Abstract

This research aims to present a collection of dataset, which represents the mapping of program education objectives to the ABET student outcomes. The dataset has been collected by the authors from 32 self-study reports from Engineering programs accredited by ABET, which are available online. The paper presents the constraints under which, the dataset was produced, because its understanding plays a vital role in the usage of this collection in future researches. To illustrate the properties and usefulness of the collection, the dataset has been cleansed, preprocessed, some features have been selected, then it has been benchmarked using nine of the widely used supervised multiclass classification techniques (Binary Relevance, Label Powerset, Classifier Chains, Pruned Sets, Random k-label sets, Ensemble of Classifier Chains, Ensemble of Pruned Sets, Multi-Label k Nearest Neighbors and Back-Propagation Multi-Label Learning). The techniques have been compared to each other using five well-known measurements (Accuracy, Hamming Loss, Micro-F, Macro-F, and Macro-F). The Ensemble of Classifier Chains and Ensemble of Pruned Sets have achieved encouraging performance compared to the other experimented multi-label classification methods. The Classifier Chains method has shown the worst performance. In general, promising results have been achieved. New research directions and baseline experimental results for future studies in educational data mining in general and in accreditation in specific have been provided.

Keywords

  • Benchmark collection
  • Program educational objectives
  • Student outcomes
  • ABET
  • Accreditation
  • Machine learning
  • Supervised multiclass classification
  • Text mining

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Fabrizio, S.: Machine learning in automated text categorization. ACM Comput. Surv. 34(1), 1–47 (2002)

    CrossRef  Google Scholar 

  2. Shweta, C.D., Maya, I., Parag, K.: Empirical studies on machine learning based text classification algorithms. Adv. Comput. Int. J. (ACIJ) 2(6), 161–169 (2011)

    CrossRef  Google Scholar 

  3. Fabricio, A.B., Daniel, C., Guimarães P.: Combined unsupervised and semi-supervised learning for data classification. In: IEEE 26th International Workshop on Machine Learning for Signal Processing (MLSP), Salerno, Italy, pp. 13–16 (2016)

    Google Scholar 

  4. Lunke, F., Yong, X., Xiaozhao, F., Jian, Y.: Low rank representation with adaptive distance penalty for semi-supervised subspace classification. Pattern Recogn. 67, 252–262 (2017). http://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=7605057

    CrossRef  Google Scholar 

  5. Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, Secaucus (2006)

    MATH  Google Scholar 

  6. Murphy, K.P.: Machine Learning: A Probabilistic Perspective, 1st edn. The MIT Press, Cambridge (2012)

    MATH  Google Scholar 

  7. Duda, R.O., Hart, P., Stork, D.: Pattern Classification, 2nd edn. Wiley-Interscience, New York (2000)

    MATH  Google Scholar 

  8. David, D.L., Robert, E.S., James, P.C., Ron, P.: Training algorithms for linear text classifiers. In: Proceedings of the 19th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 1996), pp. 298–306. ACM, New York (1996)

    Google Scholar 

  9. David, D.L.: Reuters-21578 text Categorization test collection. Distribution 1.0. Readme file (version 1.2). Manuscript (1997)

    Google Scholar 

  10. Yiming, Y.: An evaluation of statistical approaches to text categorization. Inf. Retrieval 1(1–2), 67–88 (1999)

    Google Scholar 

  11. David, D.L., Yiming, Y., Tony, G.R., Fan, L.: RCV1: a new benchmark collection for text categorization research. J. Mach. Learn. Res. 5, 361–397 (2004)

    Google Scholar 

  12. Pratiksha, Y., Gawande, S.H.: A comparative study on different types of approaches to text categorization. Int. J. Mach. Learn. Comput. 2(4), 423–426 (2012)

    Google Scholar 

  13. ABET, ABET Strategic Plan, Accreditation Board for Engineering and Technology, Inc., ABET, 1 November 1997

    Google Scholar 

  14. Engineering Accreditation Commission (ABET), Criteria for Accrediting Engineering Programs Effective for Review During the 2015–2016 Accreditation Cycle, 415 N. Charles Street Baltimore, MD 21201, United States of Ameriaca, ABET (2014)

    Google Scholar 

  15. ABET, Criteria for Accrediting Engineering Programs Effective for Reviews During the 2016–2017 Accrediting Cycle

    Google Scholar 

  16. de Baker, R.S.J.: Data mining for education. In: McGaw, B., Peterson, P., Baker, E. (eds.) International Encyclopedia of Education, 3rd edn. Elsevier, Oxford (2010)

    Google Scholar 

  17. Romero, C., Ventura, S.: Educational data mining: a survey from 1995 to 2005. Expert Syst. Appl. 33(1), 135–146 (2007)

    CrossRef  Google Scholar 

  18. de Baker, R.S.J., Yacef, K.: The state of educational data mining in 2009: a review and future vision. J. Educ. Data Min. 1(1), 1–15 (2009)

    Google Scholar 

  19. Peña-Ayala, A., Domínguez, R., Medel, J.: Educational data mining: a sample of review and study case. World J. Educ. Technol. 2, 118–139 (2009)

    Google Scholar 

  20. Romero, C., Ventura, S.: Educational data mining: a review of the state of the art. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 40(6), 601–618 (2010)

    CrossRef  Google Scholar 

  21. Ihantola, P., Vihavainen, A., Ahadi, A., Butler, M., Börstler, J., Edwards, S.H., Isohanni, E., Korhonen, A., Petersen, A., Rivers, K., Rubio, M.Á., Sheard, J., Skupas, B., Spacco, J., Szabo, C., Toll, D.: Educational data mining and learning analytics in programming: Literature review and case studies. In: Proceedings of the 2015 ITiCSE on Working Group Reports, Annual Conference on Innovation and Technology in Computer Science Education, pp. 41–63. ACM (2015). https://tutcris.tut.fi/portal/en/publications/educational-data-mining-and-learning-analytics-in-programming-literature-review-and-case-studies(6cd8ff1c-133a-4cf9-8a6e-ef61ba37ae7a).html

  22. Fatima, D., Fatima, S., Prasad, A.V.K.: A survey on research work in educational data mining. J. Comput. Eng. 17(2), 43–49 (2015)

    Google Scholar 

  23. Papamitsiou, Z., Economides, A.: Learning analytics and educational data mining in practice: a systematic literature review of empirical evidence. Educ. Technol. Soc. 17(4), 49–64 (2014)

    Google Scholar 

  24. Isha, S., Dinesh, K., Mudit, K.: A review of applications of data mining techniques for prediction of students’ performance in higher education. J. Stat. Manage. Syst. 20(4), 713–722 (2017). https://www.tandfonline.com/doi/abs/10.1080/09720510.2017.1395191

    CrossRef  Google Scholar 

  25. Raheela, A., Agathe, M., Syed Abbas, A., Najmi, G.H.: Analyzing undergraduate students’ performance using educational data mining. Comput. Educ. 113, 177–194 (2017)

    CrossRef  Google Scholar 

  26. Anwar, A.Y., Addin, O.: Automatic classification of questions into Bloom’s cognitive levels using support vector machines. In: The International Arab Conference on Information Technology. Naif Arab University for Security Science (NAUSS), Riyadh, Saudi Arabia (2013)

    Google Scholar 

  27. Anwar, A.Y., Addin, O., Mohammad, S.E.: Rocchio algorithm-based particle initialization mechanism for effective PSO classification of high dimensional data. Swarm Evol. Comput. 34, 18–32 (2017). https://www.sciencedirect.com/journal/swarm-and-evolutionary-computation

    CrossRef  Google Scholar 

  28. Addin, O., Anwar, A., Y.: Classifications of exam questions using linguistically-motivated features: a case study based on Bloom’s taxonomy. In: The Third International Arab Conference on Quality Assurance in Higher Education (IACQA 2016), pp. 889–896. Khartoum Sudan (2016)

    Google Scholar 

  29. Hamalainen, W., Vinni, M.: Comparison of machine learning methods for intelligent tutoring systems. In: ITS 2006 Proceedings of the 8th International Conference on Intelligent Tutoring Systems, Jhongli, Taiwan, pp. 525–534 (2006)

    Google Scholar 

  30. Mohamad, S.K., Tasir, Z.: Educational data mining: a review. In: The 9th International Conference on Cognitive Science, pp. 320–324. Procedia - Social and Behavioral Sciences, Kuching, Sarawak, Malaysia (2013)

    Google Scholar 

  31. Ronald, D.: The Importance of Having Data-sets. In: Proceedings of the IATUL Conferences, Paper 16 (2006)

    Google Scholar 

  32. Anwar, A.Y., Zakaria, T., Addin, O.: Bloom’s Taxonomy–based classification for item bank questions using support vector machines. In: Modern Advances in Intelligent Systems and Tools, vol. 431, pp. 135–140 (2012). https://link.springer.com/book/10.1007/978-3-642-30732-4

  33. Anwar, A.Y., Addin, O.: Automatic classification of questions into Bloom’s cognitive levels using support vector machines. In: The International Arab Conference on Information Technology, pp. 335–342. Naif Arab University for Security Science (NAUSS), Riyadh, Saudi Arabia (2011). https://scholar.google.com/scholar?oi=bibs&cluster=11863385617269352176&btnI=1&hl=en

  34. Anwar, A.Y., Addin, O., Ahmed A.A.: Educational data mining: a case study of teacher’s classroom questions. In: 13th International Conference on Intelligent Systems Design and Applications (ISDA), pp. 34–41. UPM, Selangor (2013). http://ieeexplore.ieee.org/abstract/document/6920714/

  35. Koller, D., Sahami, M.: Hierarchically classifying documents using very few words. In: International Conference on Machine Learning (ICML 1997), Nashville, Tennessee, pp. 170–178 (1997)

    Google Scholar 

  36. Weigend, A.S., Wiener, E.D., Pedersen, J.O.: Exploiting hierarchy in text categorization. Inf. Retrieval 1(3), 193–216 (1999)

    CrossRef  Google Scholar 

  37. Steven, B., Ewan, K., Edward, L.: Natural Language Processing with Python, 1st edn. O’Reilly Media, USA (2009)

    MATH  Google Scholar 

  38. Tsoumakas, G., Katakis, I., Vlahavas, I.: Mining multi-label data. In: Data Mining and Knowledge Discovery Handbook, pp. 667–685. Springer, Heidelberg (2010)

    Google Scholar 

  39. Jesse, R., Peter, R., Bernhard, P., Geoff, H.: MEKA: a multi-label/multi-target extension to Weka. J. Mach. Learn. Res. 17(21), 1–5 (2016)

    MathSciNet  MATH  Google Scholar 

  40. Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques, 2nd edn. Morgan Kaufmann, Elsevier, Amsterdam (2005)

    MATH  Google Scholar 

  41. Read, J., Pfahringer, B., Holmes, G.: Multi-label classification using ensembles of pruned sets. In: 8th IEEE International Conference on Data Mining, Pisa, Italy, pp. 995–1000. IEEE Computer Society (2008)

    Google Scholar 

  42. Sajnani, H., Javanmardi, S., McDonald, D.W., Lopes, C.V.: Multi-label classification of short text: a study on wikipedia barnstars. In: Analyzing Microtext: the Proceeding of the 2011 AAAI Workshop (2011)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Computer Science and Information Systems, Najran University, Najran, Saudi Arabia

    Addin Osman, Anwar Ali Yahya & Mohammed Basit Kamal

  2. Faculty of Computer Science and Information Systems, Thamar University, Thamar, Yemen

    Anwar Ali Yahya

Authors
  1. Addin Osman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anwar Ali Yahya
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohammed Basit Kamal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Addin Osman .

Editor information

Editors and Affiliations

  1. Prince Sultan University, Riyadh, Saudi Arabia

    Mamdouh Alenezi

  2. Prince Sultan University, Riyadh, Saudi Arabia

    Basit Qureshi

Rights and permissions

Reprints and Permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Osman, A., Yahya, A.A., Kamal, M.B. (2018). A Benchmark Collection for Mapping Program Educational Objectives to ABET Student Outcomes: Accreditation. In: Alenezi, M., Qureshi, B. (eds) 5th International Symposium on Data Mining Applications. Advances in Intelligent Systems and Computing, vol 753. Springer, Cham. https://doi.org/10.1007/978-3-319-78753-4_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-78753-4_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-78752-7

  • Online ISBN: 978-3-319-78753-4

  • eBook Packages: EngineeringEngineering (R0)

search

Navigation