Skip to main content
SpringerLink
Log in

CAF Profiles of Iranian Writers: What We Learn from Them and Their Limitations

  • Chapter
  • First Online:
The Assessment of L2 Written English across the MENA Region

  • 275 Accesses

Abstract

This study is an attempt to report the varying profiles of Complexity, Accuracy, and Fluency (CAF) in students’ writing. In practice, the written performance of 22 Iranian freshman students at university was analysed. Having taught the writing skills during a semester, the researcher collected and analysed the final written performance of test takers through CAF rubrics to report their final examination writing profiles. Results of the study revealed that the relative contribution of each feature could to some extent assist to inform understandings of the problematic areas in written performance. In sum, the findings of this study may corroborate the very idea of using analytic measures to determine objective writing assessment. The wider discussion of this study refers to using Cognitive Diagnostic Assessment (CDA) in the written performance of students as a complement to the CAF profile analysis as a way of determining the processes, strategies, and knowledge students possess and activate in producing their exam-based writing scripts.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Ai, H., & Lu, X. (2013). A corpus-based comparison of syntactic complexity in NNS and NS university students’ writing. In A. DĂ­az-Negrillo, N. Ballier, & P. Thompson (Eds.), Automatic treatment and analysis of learner corpus data (pp. 249–264). Amsterdam: John Benjamins.

    Chapter  Google Scholar 

  • Alderson, J. C. (2005). Diagnosing foreign language proficiency: The interface between learning and assessment. London: Continuum.

    Google Scholar 

  • Alderson, J. C. (2007). The CEFR and the need for more research. The Modern Language Journal, 91, 659–663.

    Article  Google Scholar 

  • Anderson, J. R. (1995). Learning and memory: An integrated approach. New York: Wiley.

    Google Scholar 

  • Arjona, E. (1983). Language planning in the judicial system: A look at the implementation of the U.S. Court Interpreters Act. Language Planning Newsletter, 9(1), 1–6.

    Google Scholar 

  • Biber, D., Gray, B., & Staples, S. (2016). Predicting patterns of grammatical complexity across language exam task types and proficiency levels. Applied Linguistics, 37(5), 639–668.

    Article  Google Scholar 

  • Bulte, B., & Housen, A. (2012). Defining and operationalizing L2 complexity. In A. Housen, F. Kuiken, & I. Vedder (Eds.), Dimensions of L2 performance and proficiency—Investigating complexity, accuracy and fluency in SLA (pp. 21–46). Amsterdam: John Benjamins.

    Chapter  Google Scholar 

  • Bulte, B., & Housen, A. (2014). Conceptualizing and measuring short-term changes in L2 writing complexity. Journal of Second Language Writing, 26, 42–65.

    Article  Google Scholar 

  • Carson, J. E., & Kuehn, P. A. (1992). Evidence of transfer and loss in developing second language writers. Language Learning, 42, 157–179.

    Article  Google Scholar 

  • Chambers, F. (1997). What do we mean by fluency? System, 25(4), 535–544.

    Article  Google Scholar 

  • Chen, J. S., & de la Torre, J. (2014). A procedure for diagnostically modeling extant large scale assessment data: The case of the program for international student assessment in reading. Psychology, 5, 1967–1978.

    Article  Google Scholar 

  • Chen, J. S., de la Torre, J., & Zhang, Z. (2013). Relative and absolute fit evaluation in cognitive diagnosis modeling. Journal of Educational Measurement, 50, 123–140.

    Article  Google Scholar 

  • Chen, Y. H., & Baker, P. (2016). Investigating criterial discourse features across second language development: Lexical bundles in rated learner essays, CEFR B1, B2 and C1. Applied Linguistics, 37(6), 849–880.

    Google Scholar 

  • Crookes, G. (1989). Planning and interlanguage variability. Studies in Second Language Acquisition, 11, 367–383.

    Article  Google Scholar 

  • Crossley, S. A., & McNamara, D. S. (2009). Computational assessment of lexical differences in L1 and L2 writing. Journal of Second Language Writing, 18, 119–135.

    Article  Google Scholar 

  • Crossley, S. A., Salsbury, T., & McNamara, D. S. (2012). Predicting the proficiency level of language learners using lexical indices. Language Testing, 29(2), 243–263.

    Article  Google Scholar 

  • Cui, Y., & Leighton, J. P. (2009). The hierarchy consistency index: Evaluating person fit for cognitive diagnostic assessment. Journal of Educational Measurement, 46, 429–449.

    Article  Google Scholar 

  • Dahmardeh, M., & Shahmirzadi, N. (2016). Measuring the written performance quality in terms of (CAF) complexity, accuracy and fluency constructs. Pertanika Journal of Social Science and Humanities, 24(2), 639–654.

    Google Scholar 

  • Daller, H., & Xue, H. (2007). Lexical richness and the oral proficiency of Chinese EFL students. In H. Daller, J. Milton, & J. Treffers-Daller (Eds.), Modeling and assessing vocabulary knowledge (pp. 150–164). Cambridge: Cambridge University Press.

    Chapter  Google Scholar 

  • De la Torre, J. (2009). DINA model and parameter estimation: A didactic. Journal of Educational and Behavioral Statistics, 34(1), 115–130.

    Article  Google Scholar 

  • Edelsky, C. (1982). Writing in a bilingual program: The relation of L1 and L2 texts. TESOL Quarterly, 16, 211–228.

    Article  Google Scholar 

  • Ellis, R. (1987). Interlanguage variability in narrative discourse: Style in the use of past tense. Studies in Second Language Acquisition, 18, 229–323.

    Google Scholar 

  • Ellis, R., & Barkhuizen, G. (2005). Analyzing learner language. New York: Oxford University Press.

    Google Scholar 

  • Ellis, R., & Yuan, F. (2004). The effects of planning on fluency, complexity, and accuracy in second language narrative writing. Studies in Second Language Acquisition, 26, 59–84.

    Article  Google Scholar 

  • Embretson, S. E. (1998). A cognitive design system approach to generating valid tests: Application to abstract reasoning. Psychological Methods, 3(3), 380–396.

    Article  Google Scholar 

  • Fangyuan, Y., & Ellis, R. (2003). The effect of pre-task planning and on-line planning on fluency, accuracy and complexity in second language oral production. Applied Linguistics, 2(1), 1–27.

    Google Scholar 

  • Ferris, D. R. (1994). Lexical and syntactic features of ESL writing by students at different levels of L2 proficiency. TESOL Quarterly, 28(2), 414–420.

    Article  Google Scholar 

  • Foster, P., & Skehan, P. (1996). The influence of planning and task type on second language performance. Studies in Second Language Acquisition, 18, 299–323.

    Article  Google Scholar 

  • Foster, P., Tonkyn, A., & Wigglesworth, G. (2000). Measuring spoken language: A unit for all reasons. Applied Linguistics, 21(3), 354–375.

    Article  Google Scholar 

  • Frase, L. T., Faletti, J., Ginther, A., & Grant, L. (1999). Computer analysis of the TOEFL test of written English. Princeton, NJ: Educational Testing Service.

    Google Scholar 

  • Freed, B. (2000). Is fluency, like beauty, in the eyes (and ears) of the beholder? In H. Riggenbach (Ed.), Perspectives on fluency (pp. 243–265). Ann Arbor, MI: The University of Michigan Press.

    Google Scholar 

  • George, A. C., & Robitzsch, A. (2014). Multiple group cognitive diagnosis models, with an emphasis on differential item functioning. Psychological Test and Assessment Modeling, 56(4), 405–432.

    Google Scholar 

  • Grant, L., & Ginther, A. (2000). Using computer-tagged linguistic features to describe L2 writing differences. Journal of Second Language Writing, 9(2), 123–145.

    Article  Google Scholar 

  • Guillot, M. N. (1999). Fluency and its teaching. Philadelphia: Multilingual Matters.

    Google Scholar 

  • Hilton, H. (2008). The link between vocabulary knowledge and spoken L2 fluency. Language Learning Journal, 36(2), 153–166.

    Article  Google Scholar 

  • Hou, L., la Torre, J. D., & Nandakumar, R. (2014). Differential item functioning assessment in cognitive diagnostic modeling: Application of the Wald test to investigate DIF in the DINA model. Journal of Educational Measurement, 51(1), 98–125.

    Article  Google Scholar 

  • Housen, A., & Kuiken, F. (2009). Complexity, accuracy and fluency in second language acquisition. Applied Linguistics, 30(4), 461–473.

    Article  Google Scholar 

  • Hulstijn, J. H. (2007). The shaky ground beneath the CEFR: Quantitative and qualitative dimensions of language proficiency. The Modern Language Journal, 91, 663–667.

    Article  Google Scholar 

  • Hunt, K. (1965). Grammatical structures written at three grade levels. Champaign, IL: National Council of Teachers of English.

    Google Scholar 

  • Jang, E. E. (2008). A review of cognitive diagnostic assessment for education: Theory and application. International Journal of Testing, 8(3), 290–295.

    Article  Google Scholar 

  • Jang, E. E. (2009). Cognitive diagnostic assessment of L2 reading comprehension ability: Validity arguments for Fusion Model application to LanguEdge assessment. Language Testing, 26(1), 31–73.

    Article  Google Scholar 

  • Jarvis, S. (2017). Grounding lexical diversity in human judgments. Language Testing, 34(4), 537–553.

    Article  Google Scholar 

  • Jarvis, S., & Crossley, S. A. (Eds.). (2012). Approaching language transfer through text classification: Explorations in the detection-based approach. Bristol, UK: Multilingual Matters.

    Google Scholar 

  • Jarvis, S., Grant, L., Bikowski, D., & Ferris, D. (2003). Exploring multiple profiles of highly rated learner compositions. Journal of Second Language Writing, 12(4), 377–403.

    Article  Google Scholar 

  • Johnson, W. (1944). Studies in language behavior: I. A program of research. Psychological Monographs, 56, 1–15.

    Article  Google Scholar 

  • Knoch, U., Roushad, A., & Storch, N. (2014). Does the writing of undergraduate ESL students develop after one year of study in an English-medium university? Assessing Writing, 21, 1–17.

    Article  Google Scholar 

  • Koponen, M., & Riggenbach, H. (2000). Overview: Varying perspectives on fluency. In H. Riggenbach (Ed.), Perspectives on fluency (pp. 5–24). Ann Arbor, MI: University of Michigan Press.

    Google Scholar 

  • Kunnan, A. J., & Jang, E. E. (2009). Diagnostic feedback in language assessment. In M. Long & C. Doughty (Eds.), Handbook of second and foreign language teaching (pp. 610–625). Malden, MA: Wiley-Blackwell.

    Chapter  Google Scholar 

  • Lally, C. G. (2000). First language influences in second language composition: The effect of pre-writing. Foreign Language Annals, 33, 428–432.

    Article  Google Scholar 

  • Larsen-Freeman, D. (2009). Adjusting expectations: The study of complexity, accuracy, and fluency in second language acquisition. Applied Linguistics, 30(4), 579–589.

    Article  Google Scholar 

  • Larsen-Freeman, D., & Long, M. H. (1991). An introduction to second language acquisition research. Harlow: Longman Group.

    Google Scholar 

  • Lefrançois, P. (2001). Le Point sur les transferts dans l’ecriture en langue seconde [Transfer in second-language writing]. Canadian Modern Language Review, 58, 223–245.

    Article  Google Scholar 

  • Leighton, J. P., & Gierl, M. J. (2007). Defining and evaluating models of cognition used in educational measurement to make inferences about examinees’ thinking processes. Educational Measurement: Issues and Practice, 26(2), 3–16.

    Article  Google Scholar 

  • Lennon, P. (1990). Investigating fluency in EFL: A quantitative approach. Language Learning, 40(2), 387–417.

    Article  Google Scholar 

  • Liu, J. (2008). The generic and rhetorical structures of expositions in English by Chinese ethnic minorities: A perspective from intracultural contrastive rhetoric. Language and Intercultural Communication, 8, 2–20.

    Article  Google Scholar 

  • Lu, X. (2011). A corpus-based evaluation of syntactic complexity measures as indices of college-level ESL writers’ language development. TESOL Quarterly, 45(1), 36–62.

    Article  Google Scholar 

  • Lu, X. (2012). The relationship of lexical richness to the quality of ESL learners’ oral narratives. The Modern Language Journal, 96(2), 190–208.

    Article  Google Scholar 

  • Lu, X., & Ai, H. (2015). Syntactic complexity in college-level English writing: Differences among writers with diverse L1 backgrounds. Journal of Second Language Writing, 29, 16–27.

    Article  Google Scholar 

  • Nichols, P. D. (1994). A framework for developing cognitively diagnostic assessments. Review of Educational Research, 64(4), 575–603.

    Article  Google Scholar 

  • Norris, J. M., & Ortega, L. (2009). Towards an organic approach to investigating CAF in instructed SLA: The case of complexity. Applied Linguistics, 30(4), 555–578.

    Article  Google Scholar 

  • Ortega, L. (1999). Planning and focus on form in second language oral performance. Studies in Second Language Acquisition, 21(2), 109–148.

    Article  Google Scholar 

  • Ortega, L. (2003). Syntactic complexity measures and their relationship to L2 proficiency: A research synthesis of college-level L2 writing. Applied Linguistics, 24(4), 492–518.

    Article  Google Scholar 

  • Ortega, L., & Iberri-Shea, G. (2005). Longitudinal research in second language acquisition: Recent trends and future directions. Annual Review of Applied Linguistics, 25, 26–45.

    Article  Google Scholar 

  • Paquot, M. (2013). Lexical bundles and L1 transfer effects. International Journal of Corpus Linguistics, 18, 391–417.

    Article  Google Scholar 

  • Pica, T., Holliday, L., Lewis, N., & Morgenthaler, L. (1989). Comprehensible output as an outcome of linguistic demands on the learner. Studies in Second Language Acquisition, 11, 63–90.

    Article  Google Scholar 

  • Rankin, T. (2012). The transfer of V2: Inversion and negation in German and Dutch learners of English. International Journal of Bilingualism, 16, 139–158.

    Article  Google Scholar 

  • Read, J. (2000). Assessing vocabulary. Cambridge, UK: Cambridge University Press.

    Book  Google Scholar 

  • Schachter, J., & Celce-Murcia, M. (1977). Some reservations concerning error analysis. TESOL Quarterly, 11, 441–451.

    Article  Google Scholar 

  • Sen, S., & Bradshaw, L. (2017). Comparison of relative fit indices for diagnostic model selection. Applied Psychological Measurement, 41, 422–438.

    Article  Google Scholar 

  • Skehan, P. (1998a). A cognitive approach to language learning. Oxford: Oxford University Press.

    Google Scholar 

  • Skehan, P. (1998b). Second language acquisition research and task-based instruction. In J. Willis & D. Willis (Eds.), The challenge and change in language teaching. Oxford: Heinemann.

    Google Scholar 

  • Skehan, P., & Foster, P. (1999). The influence of task structure and processing conditions on narrative retellings. Language Learning, 49, 93–120.

    Article  Google Scholar 

  • Sotillo, S. M. (2000). Discourse functions and syntactic complexity in synchronous and asynchronous communication. Language Learning and Technology, 4, 82–119.

    Google Scholar 

  • Storch, N., & Tapper, J. (2009). The impact of an EAP course on postgraduate writing. Journal of English for Academic Purposes, 8(3), 207–223.

    Article  Google Scholar 

  • Terzi, R., & Sen, S. (2019). A nondiagnostic assessment for diagnostic purposes: Q-matrix validation and item-based model fit evaluation for the TIMSS 2011 assessment. SAGE Open, 9(1), 1–11.

    Article  Google Scholar 

  • Tetreault, J., Blanchard, D., & Cahill, A. (2013). A report on the first native language identification shared task. In Proceedings of the eighth workshop on innovative use of NLP for building educational applications (pp. 48–57). Atlanta, GA: Association for Computational Linguistics.

    Google Scholar 

  • Underwood, M. (1990). Task-related variation in past tense morphology. Unpublished master’s thesis, Institute of Education, University of London.

    Google Scholar 

  • Uysal, H. H. (2008). Tracing the culture behind writing: Rhetorical patterns and bidirectional transfer in L1 and L2 essays of Turkish writers in relation to educational context. Journal of Second Language Writing, 17, 183–207.

    Article  Google Scholar 

  • Van Patten, B. (1990). Attending to content and form in the input: An experiment in consciousness. Studies in Second Language Acquisition, 12, 287–301.

    Article  Google Scholar 

  • van Vuuren, S. (2013). Information structural transfer in advanced Dutch EFL writing: A cross-linguistic longitudinal study. Linguistics in the Netherlands, 30, 173–187.

    Article  Google Scholar 

  • van Weijen, D., van den Bergh, H., Rijlaarsdam, G., & Sanders, T. (2009). L1 use during L2 writing: An empirical study of a complex phenomenon. Journal of Second Language Writing, 18, 235–250.

    Article  Google Scholar 

  • Vercellotti, M. L. (2012). Complexity, accuracy, and fluency as properties of language performance: The development of the multiple subsystems over time and in relation to each other (Doctoral Dissertation). University of Pittsburgh.

    Google Scholar 

  • Verspoor, M., Lowie, W., & Van Dijk, M. (2008). Variability in second language development from a dynamic systems perspective. Modern Language Journal, 92(2), 214–231.

    Article  Google Scholar 

  • Wang, Z., Guo, L., & Bian, Y. (2014). Comparison of DIF detecting methods in cognitive diagnostic test. Acta Psychologica Sinica, 46(12), 1923–1932.

    Article  Google Scholar 

  • Way, D. P., Joiner, E. G., & Seaman, M. A. (2000). Writing in the secondary foreign language classroom: The effects of prompts and tasks on novice learners of French. The Modern Language Journal, 84, 171–184.

    Article  Google Scholar 

  • Wolfe-Quintero, K., Inagaki, S., & Kim, H.-Y. (1998). Second language development in writing: Measures of fluency, accuracy, and complexity. Honolulu: University of Hawaii Press.

    Google Scholar 

  • Yang, W., Lu, X., & Weigle, S. A. (2015). Different topics, different discourse: Relationships among writing topic, measures of syntactic complexity, and judgments of writing quality. Journal of Second Language Writing, 28, 53–67.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Islamic Azad University, Tehran, Iran

    Niloufar Shahmirzadi

Authors
  1. Niloufar Shahmirzadi
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Social Sciences and International Studies, University of Exeter, Exeter, UK

    Lee McCallum

  2. Dubai Men's College, Higher Colleges of Technology, Dubai, United Arab Emirates

    Christine Coombe

Appendix 1: Sample CAF Coding for a Student’s Script

Appendix 1: Sample CAF Coding for a Student’s Script

1.1 Essay Title: Renewing Perspective

One human ability is to get under the influence of other characters and characteristics like happiness, sadness, creativity, independency, and so on. People are able to share all these characteristics with each other as long as they are living with each other (Repetitions). When two people are in contact with each other for example by working and living in a same place, little by little after they get to know each other (Repetitions), they may want to be like each other (Lack of lexical diversity, repetition of word type). One of the most important characteristics that could be shared in my society, is being logical. I think if any people would be logical or wise, too many problems might be gone. Like the problem (Repetitions) of economy, finance or other things.

But on the other hand (False starts), intelligence is a kind of characteristics that is being shared between people.

And the third one is happiness, they are always happy. Even people knows (Lack of grammatical accuracy) the people of my society (False start) as welcoming, happy (Lack of lexical diversity) and kind people (Repetitions) which are all same very good points.

Rights and permissions

Reprints and permissions

Copyright information

© 2020 The Author(s)

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Shahmirzadi, N. (2020). CAF Profiles of Iranian Writers: What We Learn from Them and Their Limitations. In: McCallum, L., Coombe, C. (eds) The Assessment of L2 Written English across the MENA Region. Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-030-53254-3_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-53254-3_5

  • Published:

  • Publisher Name: Palgrave Macmillan, Cham

  • Print ISBN: 978-3-030-53253-6

  • Online ISBN: 978-3-030-53254-3

  • eBook Packages: EducationEducation (R0)

Publish with us

Policies and ethics

Search

Navigation