Crowdsourcing Real-World Feedback for Human–Computer Interaction Education
In this chapter we investigate using real-world feedback to compliment academic feedback during a course on mobile development using HCI methods. Students used crowdsourcing and macro tasking to recruit suitable end-users in order to generate feedback. During the study, we uncovered benefits and disbenefits for both staff and student stakeholders, motivations and blockers that readers and others planning to apply this method should be aware of. We report on practical matters of scalability, legal and governance issues that arise. Overall, the process proposed in the chapter produces a greatly enhanced experience for students and improves the richness of the feedback as well as the authenticity of the end-user testing experience. Challenges that are faced include incorporating this process within an academic environment with matters such as the liability of the university towards externalising student work. Overall, we were also surprised to see that harsh criticism was not taken negatively by students but was a source of motivation to improve.
KeywordsHuman computer interaction Crowdsourcing Feedback Assessment Education Mobile applications
- Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent Dirichlet allocation. The Journal of Machine Learning Research, 3, 993–1022.Google Scholar
- Cheng, J., Teevan, J., Iqbal, S. T., & Bernstein, M. S. (2015). Break it down: A comparison of macro- and microtasks. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI’15) (pp. 4061–4064). ACM, New York, NY. http://dx.doi.org/10.1145/2702123.2702146.
- Manning, C., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S., & McClosky, D. (2014). The Stanford CoreNLP natural language processing toolkit. In Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations (pp. 55–60).Google Scholar
- Noy, N. F., & McGuinness, D. L. (2001). Ontology development 101: A guide to creating your first ontology. Google Scholar
- Price, M., et al. (2012). Assessment literacy: The foundation for improving student learning (p. 2012). Oxford: OCSLD.Google Scholar
- Soilemetzidis, I., Bennett, P., Buckley, A., Hillman, N., Stoakes, G. (2014). The HEPI – HEA student academic experience survey 2014. In HEPI – HEA Spring Conference, 21 May 2014 (pp. 1–40).Google Scholar
- Wiggins G. (1990) The case for authentic assessment. Practical Assessment Research and Evaluation, 2(2). https://www.pareonline.net/getvn.asp?v=2&n=2.