Scaling Academic Writing Instruction: Evaluation of a Scaffolding Tool (Thesis Writer)

  • Christian RappEmail author
  • Peter Kauf


No thesis - no graduation. Academic writing poses manifold challenges to students, instructors and institutions alike. High labor costs, increasing student numbers, and the Bologna Process (which has reduced the period after which undergraduates in Europe submit their first thesis and thus the time available to focus on writing skills) all pose a threat to students’ academic writing abilities. This situation gave rise to the practical goal of this study: to determine if, and to what extent, academic writing and its instruction can be scaled (i.e., designed more efficiently) using a technological solution, in this case Thesis Writer (TW), a domain-specific, online learning environment for the scaffolding of student academic writing, combined with an online editor optimized for producing academic text. Compared to existing automated essay scoring and writing evaluation tools, TW is not focusing on feedback but on instruction, planning, and genre mastery. While most US-based tools, particularly those also used in secondary education, are targeting on the essay genre, TW is tailored to the needs of theses and research article writing (IMRD scheme). This mixed-methods paper reports data of a test run with a first-year course of 102 business administration students. A technology adoption model served as a frame of reference for the research design. From a student’s perspective, problems posed by the task of writing a research proposal as well as the use, usability, and usefulness of TW were studied through an online survey and focus groups (explanatory sequential design). Results seen were positive to highly positive – TW is being used, and has been deemed supportive by students. In particular, it supports the scaling of writing instruction in group assignment settings.


Computer-supported writing instruction Writing instruction Intelligent tutoring system Academic writing Evaluation 



We gratefully acknowledge the feedback of Ann Devitt (University of Dublin, Ireland), Otto Kruse (Zurich University of Applied Sciences, Winterthur, Switzerland), and Antje Proske (Technical University Dresden, Germany) on drafts of this article.


  1. Ajzen, I. (1985). From intentions to actions: A theory of planned behavior. Springer series in social psychology (Chapter: Action Control, pp. 11–39). Berlin: Springer.Google Scholar
  2. Allen, L. K., Jacovina, M. E., & McNamara, D. S. (2015). Computer-based writing instruction. In C. A. MacArthur, S. Graham, & J. Fitzgerald (Eds.), Handbook of writing research (pp. 316–329). New York: The Guildford Press.Google Scholar
  3. Barbour, R. (2007). Doing focus groups (Vol. [part 4], The Sage qualitative research kit). London: SAGE Publications.Google Scholar
  4. Bean, J. (2011). Engaging ideas. In The professor’s guide to integrating writing, critical thinking, and active learning in the classroom. San Francisco: Jossey Bass.Google Scholar
  5. Bereiter, C. (1980). Development in writing. In L. W. Gregg & E. R. Steinberg (Eds.), Cognitive processes in writing (pp. 73–96). Hillsdale: Lawrence Erlbaum.Google Scholar
  6. Bereiter, C., & Scardamalia, M. (1987). The psychology of written composition. Hillsdale: Lawrence Erlbaum.Google Scholar
  7. Bhatia, V. K. (2016). Critical reflections on genre analysis. In A. Artemeva & A. Freedman (Eds.), Genre Studies around the globe: Beyond the three traditions (Revised ed.) (pp. 17–30). Lexington: Trafford Publishing.Google Scholar
  8. Chitez, M., Erlemann, J., Ott, J., & Rapp, C. (2015a). Thesis Writer: Supporting writing processes in real time using a web-based digital learning platform: ICT Demonstration. Paper presented at the European Association for Research in Learning and Instruction (EARLI), 16th Biennial Conference. Limassol Cyprus.Google Scholar
  9. Chitez, M., Rapp, C., & Kruse, O. (2015b). Corpus-supported academic writing: How can technology help? In F. Helm, M. Bradley, M. Guarda, & S. Thouësny (Eds.), Critical CALL – Proceedings of the 2015 EUROCALL Conference, Padua, Italy (pp. 1–8). Dublin: Scholar
  10. Coiro, J., Knobel, M., Lankshear, C., & Leu, D. J. (2008). Central issues in new literacies and new literacies research. In J. Coiro, M. Knobel, C. Lankshear, & D. J. Leu (Eds.), Handbook of research on new literacies (pp. 1–21). Mahwah: Lawrence Erlbaum.Google Scholar
  11. Creswell, J. W. (2013). Qualitative inquiry and research design: Choosing among five approaches (3rd ed.). Thousand Oaks: Sage.Google Scholar
  12. Creswell, J. W., & Plano Clark, V. L. (2011). Designing and conducting mixed methods research (2nd ed.). Thousand Oaks: SAGE Publications.Google Scholar
  13. Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319–340.Google Scholar
  14. Deane, M., & O’Neill, P. (Eds.). (2011). Writing in the disciplines. London: Palgrave Macmillan.Google Scholar
  15. Doleschal, U., Mertlitsch, C., Rheindorf, M., & Wetschanow, K. (Eds.). (2013). Writing across the curriculum at work. Theorie, praxis und analyse. Wien: Lit Verlag.Google Scholar
  16. Emig, J. (1971). The composing process of twelfth graders. Urbana: National Council of Teachers of English.Google Scholar
  17. Fathema, N., Shannon, D., & Ross, M. (2015). Expanding the Technology Acceptance Model (TAM) to Examine Faculty Use of Learning Management Systems (LMSs) in higher education institutions. Journal of Online Learning & Teaching, 11(2), 210–232.Google Scholar
  18. Field, A., Miles, J., & Field, Z. (2012). Discovering statistics using R. London: Sage Publications Ltd..Google Scholar
  19. Flick, U. (2014). The SAGE handbook of qualitative data analysis. Los Angeles: SAGE.Google Scholar
  20. Flower, L., & Hayes, J. R. (1980). The dynamics of composing: Making plans and juggling constraints. In L. W. Gregg & E. R. Steinberg (Eds.), Cognitive processes in writing (pp. 31–50). Hills-dale: Lawrence Erlbaum.Google Scholar
  21. Flower, L., & Hayes, J. R. (1981). A cognitive process theory of writing. College Composition and Communication, 32(4), 365–387.Google Scholar
  22. Flower, L., Schriver, K. A., Carey, L., Haas, C., & Hayes, J. R. (1989). Planning in writing: The cognition of a constructive process. Pittsburgh: Carnegie Mellon University Retrieved from Scholar
  23. Freadman, A. (2016). The traps and trappings of genre theory. In A. Artemeva & A. Freedman (Eds.), Genre studies around the globe: Beyond the three traditions (Revised ed.) (pp. 425–452). Lexington: Trafford Publishing.Google Scholar
  24. Goodhue, D. L., & Thompson, R. L. (1995). Task-technology fit and individual performance. MIS Quarterly, 19(2), 213–236.Google Scholar
  25. Graham, S. (2018). A writer(s) within community model of writing. In C. Bazerman, V. Berninger, D. Brandt, S. Graham, J. Langer, S. Murphy, … M. Schleppegrell (Eds.), The lifespan development of writing. Urbana: National Council of English.Google Scholar
  26. Graham, S., & Perin, D. (2007). A meta-analysis of writing instruction for adolescent students. Journal of Educational Psychology, 99(3), 445–476.Google Scholar
  27. Hayes, J. R. (2012). Modeling and remodeling writing. Written Communication, 29(3), 369–388.Google Scholar
  28. Hayes, J. R., & Flower, L. S. (1980). Identifying the organization of writing processes. In L. W. Gregg & E. R. Steinberg (Eds.), Cognitive processes in writing (pp. 3–30). Hillsdale: Lawrence Erlbaum.Google Scholar
  29. Jakobs, E.-M., & Perrin, D. (Eds.). (2014). Handbook of writing and text production. Berlin: De Gruyter Mouton.Google Scholar
  30. Kellogg, R. T., & Raulerson, B. A. (2007). Improving the writing skills of college students. Psychonomic Bulletin & Review, 14(2), 237–242.Google Scholar
  31. Kent, T. (1999). Post-process theory. Beyond the writing-process paradigm. Carbondale: Southern Illinois University Press.Google Scholar
  32. Krueger, R. A., & Casey, M. A. (2015). Focus groups: A practical guide for applied research (5th ed.). Thousand Oaks: Sage Publication.Google Scholar
  33. Kruse, O. (2010). Kritisches Denken im Zeichen Bolognas: Rhetorik und realität. In U. Eberhardt (Ed.), Neue impulse in der hochschuldidaktik: Sprach- und literaturwissenschaften (pp. 45–82). Wiesbaden: Verlag für Sozialwissenschaften.Google Scholar
  34. Kruse, O. (2011). Kritisches denken als leitziel der lehre: Auswege aus der verschulungsmisere. In G. Krücken (Ed.), Innovation und kreativität an hochschulen (pp. 77–86). Wittenberg: Institut für Hochschulforschung.Google Scholar
  35. Kruse, O. (2013). Perspectives on academic writing in european higher education: genres, practices, and competences. Revista de Docencia Universitaria. REDU. Academic Writing, 11(1), 37–58.Google Scholar
  36. Kruse, O. (2016). Wissenschaftliches schreiben forschungsorientiert unterrichten, wissenschaftliches schreiben in natur- und technikwissenschaften. In A. Hirsch-Weber & S. Scherer (Eds.), Neue herausforderungen der schreibforschung (pp. 29–54). Wiesbaden: Springer Spektrum.Google Scholar
  37. Kruse, O., Chitez, M., & Peyer, E. (2016). Switzerland. In O. Kruse, M. Chitez, B. Rodrigues, & M. Castelló (Eds.), Exploring European writing cultures: Country reports on genres, writing practices and languages used in European higher education (pp. 230–254). Switzerland: Working Papers in Applied Linguistics 10, Zurich University of Applied Sciences.Google Scholar
  38. Langer, J. A., & Applebee, A. N. (2007). How writing shapes thinking: A study of teaching and learning. Colorado: WAC Clearinghouse Landmark Publications in Writing Studies Retrieved from Scholar
  39. Lillis, T., Harrington, K., Lea, M. R., & Mitchell, S. (2015). Working with academic literacies: Case studies towards transformative practice. perspectives on writing. Fort Collins: The WAC Clearinghouse and Parlor Press.Google Scholar
  40. Liu, M., Calvo, R. A., & Rus, V. (2012). G-Asks: an intelligent automatic question generation system for academic writing support. Dialogue and Discourse: Special Issue on Question Generation, 3(2), 101–124.Google Scholar
  41. Liu, M., Calvo, R. A., & Rus, V. (2014). Automatic generation and ranking of questions for critical review. Educational Technology & Society, 17(2), 333–346.Google Scholar
  42. MacArthur, C. A., Graham, S., & Fitzgerald, L. (Eds.). (2006). Handbook of writing research. New York: The Guilford Press.Google Scholar
  43. McGill, T. J., & Klobas, J. E. (2009). A task–technology fit view of learning management system impact. Computers & Education, 52(2), 496–508.Google Scholar
  44. Miles, M. B., Huberman, A. M., & Saldaña, J. (2014). Qualitative data analysis: A methods sourcebook (3rd ed.). Thousand Oaks: Sage Publications.Google Scholar
  45. New London Group. (1996). A pedagogy of multiliteracies: designing social futures. Harvard Educational Review, 66(1), 60–93.Google Scholar
  46. Oliveira, T., & Martins, M. F. (2011). Literature review of information technology adoption models at firm level. The Electronic Journal of Information Systems Evaluation, 14(1), 110–121.Google Scholar
  47. Onwuegbuzie, A. J., & Frels, R. (2016). Seven steps to a comprehensive literature review. London: Sage Publications.Google Scholar
  48. Park, S. Y. (2009). An analysis of the technology acceptance model in understanding university students’ behavioral intention to use e-Learning. Educational Technology & Society, 12(3), 150–162.Google Scholar
  49. Plano Clark, V. L., & Ivankova, N. V. (2016). Mixed methods research: A guide to the field (Vol. 3, SAGE mixed methods research series). Los Angeles: SAGE.Google Scholar
  50. Poe, M., Lerner, N., & Craig, J. (2010). Learning to communicate in science and engineering: Case studies from MIT. Cambridge: MIT Press.Google Scholar
  51. Rapp, C., Kruse, O., Erlemann, J., & Ott, J. (2015). Thesis writer – a system for supporting academic writing. In Proceedings of the 18th ACM Conference Companion on Computer Supported Cooperative Work & Social Computing (CSCW2015 Companion) (pp. 57–60). New York: ACM. Scholar
  52. Rogers, E. M. (1995). Diffusion of innovations (4th ed.). New York: Free press.Google Scholar
  53. Roscoe, R. D., Allen, L. K., Weston, J. L., Crossley, S. A., & McNamara, D. S. (2014). The Writing Pal intelligent tutoring system: usability testing and development. Computers and Composition, 34, 39–59.Google Scholar
  54. Ruhmann, G., & Kruse, O. (2014). Prozessorientierte schreibdidaktik: Grundlagen und arbeitsformen. In S. Dreyfürst & N. Sennewald (Eds.), Schreiben. Grundlagentexte zur Theorie, Didaktik und Beratung (pp. 15–34). Opladen: Barbara Budrich.Google Scholar
  55. Russell, D. R. (2002). Writing in the academic disciplines: A curricular history (2nd ed.). Carbondale: Southern Illinois Press.Google Scholar
  56. Saldaña, J. (2013). The coding manual for qualitative researchers (2nd ed.). London: SAGE Publications.Google Scholar
  57. Shermis, M. D., & Burstein, J. (Eds.). (2013). Handbook of automated essay evaluation: Current applications and new directions. London: Routledge.Google Scholar
  58. Steenbergen-Hu, S., & Cooper, H. (2014). A meta-analysis of the effectiveness of intelligent tutoring systems on college students’ academic learning. Journal of Educational Psychology, 106(2), 331–347.Google Scholar
  59. Stevenson, M., & Phakiti, A. (2014). The effects of computer-generated feedback on the quality of writing. Assessing Writing, 19, 51–65.Google Scholar
  60. Swales, J. M. (1990). Genre analysis: English in academic and research settings (Cambridge applied linguistics series). Cambridge: Cambridge University Press.Google Scholar
  61. Swales, J. M. (2004). Research genres: Explorations and applications (Cambridge applied linguistics series). Cambridge: Cambridge University Press.Google Scholar
  62. Tang, J. C., Liu, S. B., Muller, M., Lin, J., & Drews, C. (2006). Unobtrusive but invasive: using screen recording to collect field data on computer-mediated interaction. In proceedings of the 20th anniversary ACM conference on Computer supported cooperative work (pp. 479–482). Banff: ACM.Google Scholar
  63. Tarhini, A., Hone, K., & Liu, X. (2013). Factors affecting students’ acceptance of e-learning environments in developing countries: a structural equation modeling approach. International Journal of Information and Education Technology, 3(1), 54–59.Google Scholar
  64. Thaiss, C., & Zawacki, T. M. (2006). Engaged writers and dynamic disciplines. Research on the academic writing life. Portsmouth: Boynton/Cook.Google Scholar
  65. Tornatzky, L., & Fleischer, M. (1990). The process of technology innovation. Lexington: Lexington Books.Google Scholar
  66. Torrance, M., Alamargot, D., Castello, M., Garnier, F., Kruse, O., Mangen, L., et al. (Eds.). (2012). Learning to write effectively: Current trends in european research. Bradford: Emerald.Google Scholar
  67. Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: toward a unified view. MIS Quarterly, 27(3), 425–478.Google Scholar
  68. Walvoord, B. E. F., & McCarthy, L. P. (Eds.). (1990). Thinking and writing in college: A naturalistic study of students in four disciplines. Urbana: National Council of Teachers of English.Google Scholar
  69. Wenger, E. (1999). Communities of practice: Learning, meaning, and identity. UK: Cambridge University Press.Google Scholar
  70. Williams, M. D., Rana, N. P., & Dwivedi, Y. K. (2015). The unified theory of acceptance and use of technology (UTAUT): a literature review. Journal of Enterprise Information Management, 28(3), 443–488.Google Scholar
  71. Yi, M. Y., & Hwang, Y. (2003). Predicting the use of web-based information systems: self-efficacy, enjoyment, learning goal orientation, and the technology acceptance model. International Journal of Human-Computer Studies, 59(4), 431–449.Google Scholar
  72. Yu, T. K., & Yu, T. Y. (2010). Modelling the factors that affect individuals’ utilisation of online learning systems: an empirical study combining the task technology fit model with the theory of planned behavior. British Journal of Educational Technology, 41(6), 1003–1017.Google Scholar

Copyright information

© International Artificial Intelligence in Education Society 2018

Authors and Affiliations

  1. 1.Zurich University of Applied SciencesWinterthurSwitzerland
  2. 2.PrognosiX AGZurichSwitzerland

Personalised recommendations