Skip to main content

The Use of Binary Scales in Rubrics to Evaluate Computer Science Assessments

  • 73 Accesses

Part of the Communications in Computer and Information Science book series (CCIS,volume 1664)


Creating a rubric for project based assessment in ICT modules is a complex process and can be quite subjective if the wrong scales are used for rubric elements. The aim of this paper is to present a framework for developing rubrics that use binary scales for the rubric elements which will both lessen the risk of subjectivity (with focus on objective criteria) entering into the marking process and also create an opportunity for automation to be used for the marking of parts of the assessment. The framework and process used to develop a rubric are explained and an example is presented of what the final result would look like if created in this way. The framework was developed using a Design Science Research approach which is also the suggested approach to be used in the creation of a rubric. It is believed that this approach, once automated, could lead to a more dynamic approach to assessment where students can submit their work partially and get feedback immediately and lecturers can track in real time the progress of the class.


  • Rubric development
  • Binary scale
  • Project-based assessment

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-031-21076-1_4
  • Chapter length: 15 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
USD   59.99
Price excludes VAT (USA)
  • ISBN: 978-3-031-21076-1
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   74.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
Fig. 8.
Fig. 9.
Fig. 10.


  • da Alves, N.C., von Wangenheim, C.G., Hauck, J.C.R., Borgatto, A.F.: A large-scale evaluation of a rubric for the automatic assessment of algorithms and programming concepts. In: Proceedings of the 51st ACM Technical Symposium on Computer Science Education, pp. 556–562. Association for Computing Machinery, New York (2020)

    Google Scholar 

  • Dawson, P.: Assessment rubrics: towards clearer and more replicable design, research and practice. Assess. Eval. High. Educ. 42(3), 347–360 (2017)

    CrossRef  Google Scholar 

  • Dimitrov, D.M.: An approach to scoring and equating tests with binary items: piloting with large-scale assessments. Educ. Psychol. Meas. 76(6), 954–975 (2016)

    CrossRef  Google Scholar 

  • Department of Basic Education: Curriculum Assessment Policy Statements (CAPS) (2021). Accessed 09 Feb 2022

  • Haghdoost, A.A.: Complexity of the socioeconomic status and its disparity as a determinant of health. Int. J. Prev. Med. 3(2), 75–76 (2021)

    Google Scholar 

  • Hevner, A.R.: A three cycle view of design science research. Scand. J. Inf. Syst. 19(2), 1–6 (2007)

    Google Scholar 

  • Hevner, A.R., March, S.T., Park, J.: Design science in information systems research. Manag. Inf. Syst. Q. 28(1), 75–105 (2004)

    CrossRef  Google Scholar 

  • Imbault, C., Shore, D., Kuperman, V.: Reliability of the sliding scale for collecting affective responses to words. Behav. Res. Methods 50(6), 2399–2407 (2018).

    CrossRef  Google Scholar 

  • Jones, D., Tadros, A.: Successful use of rubrics to assess student performance in capstone projects. In: 2010 Annual Conference & Exposition, pp. 15.1144.1–15.1144.8. ASEE, Louisville, Kentucky (2010)

    Google Scholar 

  • Janse van Rensburg, J.T., Goede, R.: Reflecting on the use of project-based learning for 21st century competencies in an IT extended programme. In: Proceedings of the 10th Annual UNISA ISTE Conference on Mathematics, Science and Technology Education, pp. 76–84. UNISA (2019)

    Google Scholar 

  • Lukka, K.: The constructive research approach. In: Case Study Research in Logistics. Publications of the Turku School of Economics and Business Administration, Series B, vol. 1, pp. 83–101 (2003)

    Google Scholar 

  • Morolong, P.: Teachers’ views on the challenges they experience when implementing the curriculum and assessment policy statement in the foundation phase. (Dissertation), Central University of Technology, Free State (2009)

    Google Scholar 

  • Mustapha, A., Samsudin, N.A., Arbaiy, N., Mohamed, R., Hamid, I.R.: Generic assessment rubrics for computer programming courses. Turk. Online J. Educ. Technol. 15(1), 53–68 (2016)

    Google Scholar 

  • Novita, L.: The design of assessment instruments in English research. J. Engl. Educ. 3(1), 27–30 (2017)

    Google Scholar 

  • Park, H., Yan, X.: An investigation into rater performance with a holistic scale and a binary, analytic scale on an ESL writing placement test. Pap. Lang. Test. Assess. 8(2), 34–64 (2019)

    Google Scholar 

  • Simon, H.A.: The Sciences of the Artificial, 3rd edn. MIT Press, Cambridge (1996)

    Google Scholar 

  • Plomp, T., Nieveen, N.M.: Educational Design Research, 1st edn. SLO, Enschede (2013)

    Google Scholar 

  • Zano, K.: Exploring teachers’ enactment of the Curriculum and Assessment Policy Statement (CAPS) in selected Free State Province schools. (Dissertation), UNISA, Pretoria (2015)

    Google Scholar 

Download references


The full scope of the hackathon, all data used and the template rubric for use in other projects can be obtained at

Author information

Authors and Affiliations


Corresponding author

Correspondence to Jacqui Muller .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Muller, J., Greeff, J.J. (2022). The Use of Binary Scales in Rubrics to Evaluate Computer Science Assessments. In: Barnett, R.J., le Roux, D.B., Parry, D.A., Watson, B.W. (eds) ICT Education. SACLA 2022. Communications in Computer and Information Science, vol 1664. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-21075-4

  • Online ISBN: 978-3-031-21076-1

  • eBook Packages: Computer ScienceComputer Science (R0)