Advertisement

Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Faculty acceptance of the peer assessment collaboration evaluation tool: a quantitative study

  • 42 Accesses

Abstract

The problem this study sought to address was faculty reluctance to use new online peer-assessment tools. The purpose of this study was to examine the motivational factors that influence the acceptance of the Peer Assessment Collaboration Evaluation (PACE) Tool among faculty employed at a mid-sized university in the Southeastern United States. This study used Davis’s (1986) technology acceptance model (TAM) and motivational constructs “attitude toward using, perceived usefulness and perceived ease of use” (p. 44). The researcher used simple linear regression and standard multiple regression to determine if there was a significant relationship, if any, between the motivational constructs. The results of the linear regressions denoted positive, significant relationships between perceived ease of use of the PACE Tool and attitude toward using the PACE Tool, perceived usefulness of the PACE Tool and attitude toward using the PACE Tool; and perceived ease of use of the PACE Tool and perceived usefulness of the PACE Tool. The results of the multiple regression indicated that both perceived ease of use and perceived usefulness of the PACE Tool were positively, significantly related to attitude toward using the PACE Tool. Through faculty members’ speculations, the researcher was able to measure their motivation to use the PACE Tool. The results of this study demonstrated faculty members are motivated to use the PACE Tool, which indicates high acceptability and potential usage in the future. By understanding how faculty members perceive the PACE Tool, designers may be able to develop online peer-assessment tools that are more acceptable.

This is a preview of subscription content, log in to check access.

Fig. 1

Copyright 2018 by Massachusetts Institute of Technology. Adapted with permission (Appendix 1)

Fig. 2

Copyright 2018 by Massachusetts Institute of Technology. Adapted with permission (Appendix 1)

References

  1. Adachi, C., Tai, J. H. M., & Dawson, P. (2017). Academics’ perceptions of the benefits and challenges of self and peer assessment in higher education. Assessment & Evaluation in Higher Education,43(2), 294–306. https://doi.org/10.1080/02602938.2017.1339775.

  2. Allen, I. E., & Seaman, J. (2017, May). Digital learning compass: Distance education enrollment report 2017. Retrieved from https://onlinelearningsurvey.com/reports/digtiallearningcompassenrollment2017.pdf.

  3. Baruah, B., Ward, T., & Jackson, N. (2017, July). Is reflective writing an effective peer assessment tool for students in higher education? In Proceedings of the 2017 16th International Conference on Information Technology Based Higher Education and Training (ITHET), Ohrid, Macedonia, 1–6. https://doi.org/10.1109/ITHET.2017.8067802.

  4. Board of Governors. (2017). Online education state university system of Florida annual report 2016. Retrieved from https://www.flbog.edu/board/office/online/_doc/online_annual/Online_Annual_2016.pdf.

  5. Boud, D., Lawson, R., & Thompson, D. G. (2015). The calibration of student judgement through self-assessment: Disruptive effects of assessment patterns. Higher Education Research & Development,34(1), 45–59. https://doi.org/10.1080/07294360.2014.934328.

  6. Brooks, C. M., & Ammons, J. L. (2003). Free riding in group projects and the effects of timing, frequency, and specificity of criteria in peer assessments. Journal of Education for Business,78(5), 268–272. https://doi.org/10.1080/08832320309598613.

  7. Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research. Boston, MA: Houghton Mifflin.

  8. Cederblom, D., & Lounsbury, J. W. (1980). An investigation of user acceptance of peer evaluations. Personnel Psychology,33(3), 567–579. https://doi.org/10.1111/j.1744-6570.1980.tb00484.x.

  9. Chatterjee, S., & Hadi, A. S. (2012). Regression analysis by example (5th ed.). Hoboken, NJ: John Wiley & Sons.

  10. Cheung, R., & Vogel, D. (2013). Predicting user acceptance of collaborative technologies: An extension of the technology acceptance model for e-learning. Computers & Education,63, 160–175. https://doi.org/10.1016/j.compedu.2012.12.003.

  11. Chintalapati, N., & Daruri, V. S. K. (2017). Examining the use of YouTube as a learning resource in higher education: Scale development and validation of TAM model. Telematics and Informatics,34(6), 853–860. https://doi.org/10.1016/j.tele.2016.08.008.

  12. Cohen, J. (1992). A power primer. Psychological Bulletin,112(1), 155–159. https://doi.org/10.1037/0033-2909.112.1.155.

  13. Colton, D., & Covert, R. W. (2007). Designing and constructing instruments for social research and evaluation. San Francisco, CA: Jossey-Bass.

  14. Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods approaches (4th ed.). Thousand Oaks, CA: Sage Publications.

  15. Davis, F. D. (1986). A technology acceptance model for empirically testing new end-user information systems: Theory and results (Doctoral dissertation, Massachusetts Institute of Technology). Retrieved from https://dspace.mit.edu/handle/1721.1/15192.

  16. Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly,13(3), 319–340. https://doi.org/10.2307/249008.

  17. Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User acceptance of computer technology: A comparison of two theoretical models. Management Science,35(8), 982–1003. https://doi.org/10.1287/mnsc.35.8.982.

  18. Dick, W., Carey, L., & Carey, J. O. (2005). The systematic design of instruction (6th ed.). Boston, MA: Pearson.

  19. Dommeyer, C. J. (2012). A new strategy for dealing with social loafers on the group project: The segment manager method. Journal of Marketing Education,34(2), 113–127. https://doi.org/10.1177/0273475312450384.

  20. Ducey, A. J., & Coovert, M. D. (2016). Predicting tablet computer use: An extended technology acceptance model for physicians. Health Policy and Technology,5(3), 268–284. https://doi.org/10.1016/j.hlpt.2016.03.010.

  21. Elliot, A. C., & Woodward, W. A. (2015). IBM SPSS by example: A practical guide to statistical data analysis. Thousand Oaks, CA: Sage Publications.

  22. Faul, F., Erdfelder, E., Buchner, A., & Lang, A.-G. (2009). Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods41, 1149–1160. Retrieved from https://www.gpower.hhu.de/.

  23. Fishbein, M., & Ajzen, I. (1975). Belief, attitude, intention and behavior: An introduction to theory and research. Reading, MA: Addison-Wesley.

  24. Freeman, M., & McKenzie, J. (2002). SPARK, a confidential web–based template for self and peer assessment of student teamwork: Benefits of evaluating across different subjects. British Journal of Educational Technology,33(5), 551–569. https://doi.org/10.1111/1467-8535.00291.

  25. Friedman, B. A., Cox, P. L., & Maher, L. (2010). Best practices for the implementation of goal setting and peer assessment: Curriculum and research design. The Journal of Applied Business and Economics, 10(4), 34–46. Retrieved from https://na-businesspress.homestead.com.

  26. Gueldenzoph, L. E., & May, G. L. (2002). Collaborative peer evaluation: Best practices for group member assessments. Business Communication Quarterly,65(1), 9–20. https://doi.org/10.1177/108056990206500102.

  27. Hahs-Vaughn, D. L., & Lomax, R. G. (2013). An introduction to statistical concepts (3rd ed.). New York, NY: Routledge.

  28. Hamer, J., Ma, K. T., & Kwong, H. H. (2005, January). A method of automatic grade calibration in peer assessment. In Proceedings of the 7th Australasian Conference on Computing Education-Volume 42, Australia, 67–72. Retrieved from https://pdfs.semanticscholar.org/51ab/43f290a85c233e000b9b2cb2cd0282433817.pdf.

  29. Kelley, D. (2015). Peer evaluation within a group design project. Journal of Engineering Technology, 32(1), 44–50. Retrieved from https://www.engtech.org/jet/.

  30. Knapp, H. (2018). Intermediate statistics using SPSS. Thousand Oaks, CA: Sage Publications.

  31. Kulturel-Konak, S., Konak, A., Kremer, G. E. O., Esparragoza, I., & Yoder, G. (2014, January). Peer Evaluation and Assessment Resource (PEAR) to assess students' professional skills. In Proceedings from the IIE Annual Conference, Canada, 746–753. Retrieved from https://www.researchgate.net/profile/Guel_Kremer/publication/273967464_Peer_Evaluation_and_Assessment_Resource_PEAR_to_Assess_Students%27_Professional_Skills/links/55117c3b0cf24e9311ce655a/Peer-Evaluation-and-Assessment-Resource-PEAR-to-Assess-Students-Professional-Skills.pdf.

  32. Lemay, D. J., Morin, M. M., Bazelais, P., & Doleck, T. (2018). Modeling students' perceptions of simulation-based learning using the technology acceptance model. Clinical Simulation in Nursing,20, 28–37. https://doi.org/10.1016/j.ecns.2018.04.004.

  33. Li, X. (2015). Construct peer assessment instrument using existing e-learning tools. In Proceedings of the 2015 IEEE Frontiers in Education Conference (FIE), El Paso, TX, 1–4. https://doi.org/10.1109/FIE.2015.7344396.

  34. Liu, N. F., & Carless, D. (2006). Peer feedback: The learning element of peer assessment. Teaching in Higher Education,11(3), 279–290. https://doi.org/10.1080/13562510600680582.

  35. Mahmood, A., Choudhary, M. A., & Qurashi, A. H. (2016, September). Redesigning the way teams work smarter using comprehensive assessment of team member effectiveness (CATME). In Proceedings of the 2016 Portland International Conference on Management of Engineering and Technology (PICMET), Honolulu, HI, 1713–1718. https://doi.org/10.1109/PICMET.2016.7806768.

  36. Manis, K. T., & Choi, D. (2019). The virtual reality hardware acceptance model (VR-HAM): Extending and individuating the technology acceptance model (TAM) for virtual reality hardware. Journal of Business Research. https://doi.org/10.1016/j.jbusres.2018.10.021.

  37. Marangunić, N., & Granić, A. (2015). Technology acceptance model: A literature review from 1986 to 2013. Universal Access in the Information Society,14(1), 81–95. https://doi.org/10.1007/s10209-014-0348-1.

  38. Mayende, G., Isabwe, G. M. N., Muyinda, P. B., & Prinz, A. (2015, September). Peer assessment based assignment to enhance interactions in online learning groups. In Proceedings of the 2015 International Conference on Interactive Collaborative Learning (ICL), Italy, 668–672. https://doi.org/10.1109/ICL.2015.7318106.

  39. Misra, R. K., & Khurana, K. (2017). Employability skills among information technology professionals: A literature review. Procedia Computer Science,122, 63–70. https://doi.org/10.1016/j.procs.2017.11.342.

  40. Murdoch, J. (2015). Using self-and peer assessment at honours level: Bridging the gap between law school and the workplace. The Law Teacher,49(1), 73–91. https://doi.org/10.1080/03069400.2014.988491.

  41. Phillips, F. (2016). The power of giving feedback: Outcomes from implementing an online peer assessment system. Issues in Accounting Education,31(1), 1–15. https://doi.org/10.2308/iace-50754.

  42. Porr, D. (2016). Changes in individual student contribution to group projects before and after a peer review. Academy of Business Research Journal, 3, 32–46. Retrieved from https://www.aobronline.com/abrj

  43. Roberts, C. M. (2010). The dissertation journey: A practical and comprehensive guide to planning, writing, and defending your dissertation (2nd ed.). Thousand Oaks, CA: Corwin Press.

  44. Russell, A., Chapman, O., & Wegner, P. (1998). Molecular science: Network-deliverable curricula. Journal of Chemical Education,75(5), 578–579. https://doi.org/10.1021/ed075p578.

  45. Scott, F. J., Connell, P., Thomson, L. A., & Willison, D. (2017). Empowering students by enhancing their employability skills. Journal of Further and Higher Education,. https://doi.org/10.1080/0309877X.2017.1394989.

  46. Sharp, J. H. (2007). Development, extension, and application: A review of the technology acceptance model. Information Systems Education Journal, 5(9), 1–11. Retrieved from https://isedj.org.

  47. Simon, M. K., & Goes, J. (2013). Dissertation and scholarly research: Recipes for success. Dissertation Success, LLC.

  48. Sisodia, S., & Agarwal, N. (2017). Employability skills essential for healthcare industry. Procedia Computer Science,122, 431–438. https://doi.org/10.1016/j.procs.2017.11.390.

  49. Stevens, J. P. (1984). Outliers and influential data points in regression analysis. Psychological Bulletin,95(2), 334–344. https://doi.org/10.1037/0033-2909.95.2.334.

  50. Suleman, F. (2016). Employability skills of higher education graduates: Little consensus on a much-discussed subject. Procedia-Social and Behavioral Sciences,228, 169–174. https://doi.org/10.1016/j.sbspro.2016.07.025.

  51. Sumner, F. C. (1932). Marks as estimated by students. Education, 52(7), 429. Retrieved from https://www.projectinnovation.com/education.html.

  52. Taherdoost, H. (2018). A review of technology acceptance and adoption models and theories. Procedia Manufacturing,22, 960–967. https://doi.org/10.1016/j.promfg.2018.03.137.

  53. Tseng, S. C., & Tsai, C. C. (2007). On-line peer assessment and the role of the peer feedback: A study of high school computer course. Computers & Education,49(4), 1161–1174. https://doi.org/10.1016/j.compedu.2006.01.007.

  54. Tsitskari, E., Goudas, M., Tsalouchou, E., & Michalopoulou, M. (2017). Employers’ expectations of the employability skills needed in the sport and recreation environment. Journal of Hospitality, Leisure, Sport & Tourism Education,20, 1–9. https://doi.org/10.1016/j.jhlste.2016.11.002.

  55. Van Gennip, N. A., Segers, M. S., & Tillema, H. H. (2010). Peer assessment as a collaborative learning activity: The role of interpersonal variables and conceptions. Learning and Instruction,20(4), 280–290. https://doi.org/10.1016/j.learninstruc.2009.08.010.

  56. Vickerman, P. (2009). Student perspectives on formative peer assessment: An attempt to deepen learning? Assessment & Evaluation in Higher Education,34(2), 221–230. https://doi.org/10.1080/02602930801955986.

  57. Wen, M. L., & Tsai, C. C. (2008). Online peer assessment in an inservice science and mathematics teacher education course. Teaching in Higher Education,13(1), 55–67. https://doi.org/10.1080/13562510701794050.

  58. Wu, P. F. (2009, May). User acceptance of emergency alert technology: A case study. In Proceedings of the 6th International ISCRAM Conference, Sweden, 1–9. Retrieved from https://pdfs.semanticscholar.org/c4c8/0a300c73694450a83eb96c09916b4c276bf9.pdf.

  59. Wuttisela, K., Wuttiprom, S., Phonchaiya, S., & Saengsuwan, S. (2016). Implementation of online peer assessment in a design for learning and portfolio (D4L+P) program to help students complete science projects. TOJET: The Turkish Online Journal of Educational Technology, 15(4), 69–76. Retrieved from https://tojet.net.

  60. Yoon, H. Y. (2016). User acceptance of mobile library applications in academic libraries: An application of the technology acceptance model. The Journal of Academic Librarianship,42(6), 687–693. https://doi.org/10.1016/j.acalib.2016.08.003.

  61. Yorke, M. (2006). Employability in higher education: What it is-what it is not (Vol. 1). York, UK: Higher Education Academy.

Download references

Author information

Correspondence to Megan Podsiad.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

Informed verbal consent was obtained from all individual participants included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix 1: Authorization to use copyrighted materials

figurea

Appendix 2: Survey

figureba
figurebb
figurebc
figurebd

Appendix 3

figurec

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Podsiad, M., Havard, B. Faculty acceptance of the peer assessment collaboration evaluation tool: a quantitative study. Education Tech Research Dev (2020). https://doi.org/10.1007/s11423-020-09742-z

Download citation

Keywords

  • Peer-assessment tool
  • Faculty acceptance
  • Motivation
  • Collaborative learning
  • Peer evaluation