Skip to main content
Book cover

Annual Conference of the Southern African Computer Lecturers' Association

SACLA 2022: ICT Education pp 58–72Cite as

The Use of Binary Scales in Rubrics to Evaluate Computer Science Assessments

  • 73 Accesses

Part of the Communications in Computer and Information Science book series (CCIS,volume 1664)

Abstract

Creating a rubric for project based assessment in ICT modules is a complex process and can be quite subjective if the wrong scales are used for rubric elements. The aim of this paper is to present a framework for developing rubrics that use binary scales for the rubric elements which will both lessen the risk of subjectivity (with focus on objective criteria) entering into the marking process and also create an opportunity for automation to be used for the marking of parts of the assessment. The framework and process used to develop a rubric are explained and an example is presented of what the final result would look like if created in this way. The framework was developed using a Design Science Research approach which is also the suggested approach to be used in the creation of a rubric. It is believed that this approach, once automated, could lead to a more dynamic approach to assessment where students can submit their work partially and get feedback immediately and lecturers can track in real time the progress of the class.

Keywords

  • Rubric development
  • Binary scale
  • Project-based assessment

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-031-21076-1_4
  • Chapter length: 15 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   59.99
Price excludes VAT (USA)
  • ISBN: 978-3-031-21076-1
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   74.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
Fig. 8.
Fig. 9.
Fig. 10.

References

  • da Alves, N.C., von Wangenheim, C.G., Hauck, J.C.R., Borgatto, A.F.: A large-scale evaluation of a rubric for the automatic assessment of algorithms and programming concepts. In: Proceedings of the 51st ACM Technical Symposium on Computer Science Education, pp. 556–562. Association for Computing Machinery, New York (2020)

    Google Scholar 

  • Dawson, P.: Assessment rubrics: towards clearer and more replicable design, research and practice. Assess. Eval. High. Educ. 42(3), 347–360 (2017)

    CrossRef  Google Scholar 

  • Dimitrov, D.M.: An approach to scoring and equating tests with binary items: piloting with large-scale assessments. Educ. Psychol. Meas. 76(6), 954–975 (2016)

    CrossRef  Google Scholar 

  • Department of Basic Education: Curriculum Assessment Policy Statements (CAPS) (2021). https://www.education.gov.za/Curriculum/CurriculumAssessmentPolicyStatements(CAPS)/tabid/420/Default.aspx. Accessed 09 Feb 2022

  • Haghdoost, A.A.: Complexity of the socioeconomic status and its disparity as a determinant of health. Int. J. Prev. Med. 3(2), 75–76 (2021)

    Google Scholar 

  • Hevner, A.R.: A three cycle view of design science research. Scand. J. Inf. Syst. 19(2), 1–6 (2007)

    Google Scholar 

  • Hevner, A.R., March, S.T., Park, J.: Design science in information systems research. Manag. Inf. Syst. Q. 28(1), 75–105 (2004)

    CrossRef  Google Scholar 

  • Imbault, C., Shore, D., Kuperman, V.: Reliability of the sliding scale for collecting affective responses to words. Behav. Res. Methods 50(6), 2399–2407 (2018). https://doi.org/10.3758/s13428-018-1016-9

    CrossRef  Google Scholar 

  • Jones, D., Tadros, A.: Successful use of rubrics to assess student performance in capstone projects. In: 2010 Annual Conference & Exposition, pp. 15.1144.1–15.1144.8. ASEE, Louisville, Kentucky (2010)

    Google Scholar 

  • Janse van Rensburg, J.T., Goede, R.: Reflecting on the use of project-based learning for 21st century competencies in an IT extended programme. In: Proceedings of the 10th Annual UNISA ISTE Conference on Mathematics, Science and Technology Education, pp. 76–84. UNISA (2019)

    Google Scholar 

  • Lukka, K.: The constructive research approach. In: Case Study Research in Logistics. Publications of the Turku School of Economics and Business Administration, Series B, vol. 1, pp. 83–101 (2003)

    Google Scholar 

  • Morolong, P.: Teachers’ views on the challenges they experience when implementing the curriculum and assessment policy statement in the foundation phase. (Dissertation), Central University of Technology, Free State (2009)

    Google Scholar 

  • Mustapha, A., Samsudin, N.A., Arbaiy, N., Mohamed, R., Hamid, I.R.: Generic assessment rubrics for computer programming courses. Turk. Online J. Educ. Technol. 15(1), 53–68 (2016)

    Google Scholar 

  • Novita, L.: The design of assessment instruments in English research. J. Engl. Educ. 3(1), 27–30 (2017)

    Google Scholar 

  • Park, H., Yan, X.: An investigation into rater performance with a holistic scale and a binary, analytic scale on an ESL writing placement test. Pap. Lang. Test. Assess. 8(2), 34–64 (2019)

    Google Scholar 

  • Simon, H.A.: The Sciences of the Artificial, 3rd edn. MIT Press, Cambridge (1996)

    Google Scholar 

  • Plomp, T., Nieveen, N.M.: Educational Design Research, 1st edn. SLO, Enschede (2013)

    Google Scholar 

  • Zano, K.: Exploring teachers’ enactment of the Curriculum and Assessment Policy Statement (CAPS) in selected Free State Province schools. (Dissertation), UNISA, Pretoria (2015)

    Google Scholar 

Download references

Acknowledgement

The full scope of the hackathon, all data used and the template rubric for use in other projects can be obtained at https://github.com/JacquiM/The-Use-of-Binary-Scales-in-Rubrics-to-Evaluate-Computer-Science-Assessments.

Author information

Authors and Affiliations

  1. School of Computer Science and Information Systems, Unit for Data Science and Computing, North-West University, Vanderbijlpark, South Africa

    Jacqui Muller & Japie J. Greeff

Authors
  1. Jacqui Muller
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Japie J. Greeff
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jacqui Muller .

Editor information

Editors and Affiliations

  1. Stellenbosch University, Stellenbosch, South Africa

    Richard J. Barnett

  2. Stellenbosch University, Stellenbosch, South Africa

    Daniel B. le Roux

  3. Stellenbosch University, Stellenbosch, South Africa

    Douglas A. Parry

  4. Stellenbosch University, Stellenbosch, South Africa

    Bruce W. Watson

Rights and permissions

Reprints and Permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Muller, J., Greeff, J.J. (2022). The Use of Binary Scales in Rubrics to Evaluate Computer Science Assessments. In: Barnett, R.J., le Roux, D.B., Parry, D.A., Watson, B.W. (eds) ICT Education. SACLA 2022. Communications in Computer and Information Science, vol 1664. Springer, Cham. https://doi.org/10.1007/978-3-031-21076-1_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-21076-1_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-21075-4

  • Online ISBN: 978-3-031-21076-1

  • eBook Packages: Computer ScienceComputer Science (R0)