Abstract
This paper reports on a qualitative study that explored the veridicality (i.e., the completeness and accuracy) of think-aloud protocols (TAPs) in English-as-a-foreign-language (EFL) writing and illustrates how retrospective verbal reports (RVRs) compensated for TAPs in understanding online revision. Forty-three Chinese sophomores, upon writing while thinking aloud, were asked to provide RVRs regarding revisions, and then to reflect on the veridicality of their TAPs. Their reflections were analyzed inductively. Various omissions of think-alouds were revealed, but they were perceived as not serious, and the accuracy of TAPs was stood by. Further evidence concerning the (in)veridicality was found in the RVRs when 516 episodes of RVRs and corresponding TAPs were compared, and the RVRs were found to offer additional information that concerned intermediate processes leading to revisions. Implications for using TAPs and RVRs in and for EFL writing research and classrooms are given.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Armengol, L., & Cots, J. (2009). Attention processes observed in concurrent verbal reports: Two multilingual informants writing in two languages. Language Awareness, 18, 259–276.
Barkaoui, K. (2011). Think-aloud protocols in research on essay rating: An empirical study of their veridicality and reactivity. Language Testing, 28(1), 51–75.
Bowles, M. (2010). The think-aloud controversy in second language research. New York: Routledge.
Breetvelt, I., Bergh, H., & Rijlaarsdam, G. (1994). Relations between writing processes and text quality: When and how? Cognition and Instruction, 12, 103–123.
Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data (revised ed.). Cambridge, MA: MIT Press.
Faigley, L., & Witte, S. (1981). Analyzing revision. College Composition and Communication, 34(4), 400–414.
Flower, L., & Hayes, J. (1981). A cognitive process theory of writing. College Composition and Communication, 32, 365–387.
Hu, J., & Gao, X. (2017). Using think-aloud protocol in self-regulated reading research. Educational Research Review, 22, 181–193.
Hyland, K. (2016). Methods and methodologies in second language writing research. System, 59, 116–125.
Kellogg, R. (1996). A model of working memory in writing. In C. Levy & S. Ransdell (Eds.), The science of writing (pp. 57–71). Mahwah, NJ: Erlbaum.
Perl, S. (1980). Understanding composing. College Composition and Communication, 31, 363–369.
Robinson, K. M. (2001). The validity of verbal reports in children’s subtraction. Journal of Educational Psychology, 93, 211–222.
Thomas, D. R. (2006). A general inductive approach for analyzing qualitative evaluation data. American Journal of Evaluation, 2, 237–246.
Wang, W. Y., & Wen, Q. F. (2002). L1 use in the L2 composing process: An exploratory study of 16 Chinese EFL writers. Journal of Second Language Writing, 11, 225–246.
Wilson, T. D. (1994). The proper protocol: Validity and completeness of verbal reports. Psychological Science, 5, 249–252.
Yang, C. S., Hu, G. W., & Zhang, L. J. (2014). Reactivity of concurrent verbal reporting in second language writing. Journal of Second Language Writing, 24, 51–70.
Acknowledgements
This research has received grants from The Ministry of Education, China, as part of its Planned Humanities and Social Sciences Project 19YJA740070. I sincerely thank Dr. Hui-Tzu Min, the Associate Editor, and the two reviewers, especially Reviewer 1, for their insightful comments and earnest help. I hold heartfelt thanks for The University of Auckland, New Zealand, for offering me The Doctoral Scholarship, and Professor Lawrence Jun Zhang and Professor Judy Parr for their supervision. It is my New Zealand years that have laid the foundation for this work.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendixes
Appendixes
Appendix 1: The Instructions for Retrospective Verbal Reports in Chinese
Appendix 2: The Reflection Questions in Chinese and English
-
1
-
2
Concerning your think-alouds,
-
1
Do you think you reported all your thoughts? That is, was there anything that you did not report or could not report? Please elaborate.
-
2
Do you think you reported all your thoughts accurately? That is, were there any inconsistencies between what you reported and what you thought? For example, were there any cases when, for some reasons, you were not willing to report your real thoughts but reported something else instead? Please elaborate.
Rights and permissions
About this article
Cite this article
Yang, C. The Veridicality of Think-Aloud Protocols and the Complementary Roles of Retrospective Verbal Reports: A Study from EFL Writing. Asia-Pacific Edu Res 28, 531–541 (2019). https://doi.org/10.1007/s40299-019-00453-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40299-019-00453-5