Skip to main content

Advertisement

SpringerLink
Log in

Performance on a Triangulated Reading Test Battery: A Study of Language Learners’ Individual Differences and Retrospective Perceptions

  • Assessment
  • Published:
Psychological Studies Aims and scope Submit manuscript

Abstract

Individual differences, as a source of test bias, intertwine with systematic variance in test scores. In this study, we explored second language (L2) learners’ performance on a triangulated test battery (with varied subtests and item types) in relation to a number of individual differences. Later through a retrospective task, we obtained learners’ attitudes and preferences toward the most appropriate test method. The study followed a 4 (psycholinguistic mechanisms as predictors) × 3 (test methods as criterion variable 1) × 2 (item types as criterion variable 2) factorial design. The data were collected from a cohort of college-level students using the group embedded figures test, willingness to communicate (WTC) scale, Michigan proficiency test, and a reading comprehension test battery. We found that (a) field independency and a high WTC significantly determine performance on multiple-choice and short-answer tests, (b) proficiency and word size are significant predictors of essay-type test performance, (c) WTC and field independency are significant predictors of inferential knowledge, and (d) success in display items was not significantly correlated with any of the given variables. The retrospective task revealed that a meaningfully higher percentage of the participants favor multiple-choice questions over essay-type and short-answer tests. A plausible conclusion was offered by arguing that while success in controlled and receptive tests is more associated with such personality attributes as self-confidence, analytic style, risk taking, learning styles, and motivation, productive and open-ended tasks primarily call for linguistic skills and a good command of L2 proficiency.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Baba, K. (2009). Aspects of lexical proficiency in writing summaries in a foreign language. Journal of Second Language Writing, 18(3), 191–208.

    Article  Google Scholar 

  • Bachman, L., & Palmer, A. (1996). Language testing in practice: Designing and developing useful language tests. Oxford: Oxford University Press.

    Google Scholar 

  • Baeten, M., Dochy, F., & Struyven, K. (2008). Students’ approaches to learning and assessment preferences in a portfolio-based learning environment. Instructional Science, 36, 359–374.

    Article  Google Scholar 

  • Barneti-Foster, D., & Nagy, P. (1996). Undergraduate student response strategies to test questions of varying format. Higher Education, 32(2), 177–198.

    Article  Google Scholar 

  • Birenbaum, M. (1997). Assessment preferences and their relationship to learning strategies and orientations. Higher Education, 33(1), 71–84.

    Article  Google Scholar 

  • Birenbaum, M. (2007). Assessment and instruction preferences and their relationship with test anxiety and learning strategies. Higher Education, 53, 749–768.

    Article  Google Scholar 

  • Birenbaum, M., & Feldman, R. (1998). Relationships between learning patterns and attitudes towards two assessments formats. Educational Research, 40(1), 90–97.

    Article  Google Scholar 

  • Blanton, E. (2004). The influence of students’ cognitive style on a standardized reading test administered in three different formats (unpublished doctoral dissertation). University of Central Florida.

  • Bloom, B. (1953). The thought processes in lectures and discussions. Journal of General Education, 7, 160–169.

    Google Scholar 

  • Brown, H. (2000). Principles of language learning and teaching. New York: Addison Wesley, Longman.

    Google Scholar 

  • Brown, H. (2004). Language assessment: Principles and classroom practices. White Plains, NY: Pearson Education.

    Google Scholar 

  • Brown, H. (2007). Principles of language learning and teaching (5th ed.). White Plains, NY: Pearson Education.

    Google Scholar 

  • Cao, Y. (2006). Effects of field dependence-independence cognitive styles and cueing strategies on student’s recall and comprehension (unpublished doctoral dissertation). The faculty of Virginia Polytechnic Institute and State University.

  • Cao, Y., & Philp, J. (2006). Interactional context and willingness to communicate: A comparison of behavior in whole class, group and dyadic interaction. System, 34, 480–493.

    Article  Google Scholar 

  • Chamorro-Premuzic, T., Furnham, A., Dissou, G., & Heaven, P. (2005). Personality and preference for academic assessment. Learning and Individual Differences, 15, 247–256.

    Article  Google Scholar 

  • Chapelle, C., & Green, P. (1992). Field independence/dependence in second-language acquisition research. Language Learning, 42(1), 47–83.

    Article  Google Scholar 

  • Crookall, D., & Oxford, R. (1991). Dealing with anxiety: Some practical activities for language learners and teacher trainees. In E. Horwitz & D. Young (Eds.), Language anxiety: From theory and research to classroom implications (pp. 141–148). Englewood Cliffs, NJ: Prentice Hall.

    Google Scholar 

  • Daniels, H. (1996). Interaction of cognitive style and learner control of presentation mode in the hypermedia environment (unpublished doctoral dissertation). The faculty of Virginia Polytechnic Institute and State University.

  • Danili, E., & Reid, N. (2006). Cognitive factors that can potentially affect pupils’ test performance. Chemistry Education Research and Practice, 7, 64–83.

    Article  Google Scholar 

  • Davey, B., & Menek, D. (1989). The importance of cognitive style in children’s acquisition of reading skill. Early Child Development and Care, 51, 49–64.

    Article  Google Scholar 

  • Dörnyei, Z., & Kormos, J. (2000). The role of individual and social variables in oral task performance. Language Teaching Research, 4(3), 275–300.

    Article  Google Scholar 

  • Entwistle, N., & Entwistle, A. (1991). Contrasting forms of understanding for degree examinations: The student experience and its implications. Higher Education, 22, 205–227.

    Article  Google Scholar 

  • Fahim, M., & Tabatabaian, M. (2012). Concept maps, cloze tests, and multiple-choice tests: A think-aloud approach to the comparison of the strategies utilized in different test formats. Journal of American Science, 8(8), 131–138.

    Google Scholar 

  • Farthing, D., Jones, D., & McPhee, D. (1998). Permutational multiple-choice questions: An objective and efficient alternative to essay-type examination questions. Paper presented at the ITICSE, Dublin.

  • Fehrenbach, C. (1994). Cognitive style of gifted and average readers. Roeper Review, 16(4), 290–292.

    Article  Google Scholar 

  • Fulcher, G. (2010). Practical language testing. London: Hodder Education.

    Google Scholar 

  • Fulmer, B. (2010). An investigation of willingness to communicate, communication apprehension, and self-esteem in the workplace (unpublished master’s thesis). University of Tennessee.

  • Furnham, A., Batey, M., & Martin, N. (2011). How would you like to be evaluated? The correlates of students’ preferences for assessment methods. Personality and Individual Differences, 50, 259–263.

    Article  Google Scholar 

  • García-Ros, R., & Pérez-González, F. (2011). Assessment preferences of pre-service teachers: Analysis according to academic level and relationship with learning styles and motivational orientation. Teaching in Higher Education, 16, 719–731.

    Article  Google Scholar 

  • George, D., & Mallery, P. (2003). SPSS for Windows step by step: A simple guide and reference (4th ed.). Boston: Allyn & Bacon.

    Google Scholar 

  • Ghapanchi, Z., & Dashti, Z. (2011). The relationship between cognitive style of impulsivity and display, referential, and inferential reading comprehension questions among Iranian EFL University students. Canadian Social Science, 7(6), 227–233.

    Google Scholar 

  • Haastrup, K. (1991). Lexical inferencing procedures or talking about words. Tübingen: Gunter Narr.

    Google Scholar 

  • Hall, J. (2000). Field dependence-independence and computer-based instruction in geography (unpublished doctoral dissertation). University of Blacksburg, Virginia.

  • Hansen, J., & Stansfield, C. (1981). The relationship of field dependent-independent cognitive styles to foreign language achievement. Language Learning, 31(2), 349–367.

    Article  Google Scholar 

  • Hassani, L., & Maasum, M. (2012). A study of students’ reading performance in two test formats of summary writing and open-ended questions. Procedia-Social and Behavioral Sciences, 69, 915–923.

    Article  Google Scholar 

  • Hassmén, P., & Hunt, D. (1994). Human self-assessment in multiple-choice testing. Journal of Educational Measurement, 31, 149–160.

    Article  Google Scholar 

  • Hinkel, E. (2011). The handbook of research in second language teaching and learning. London: Routledge.

    Google Scholar 

  • Horwitz, E. (2001). Language anxiety and achievement. Annual Review of Applied Linguistics, 21, 112–126.

    Article  Google Scholar 

  • Hu, H., & Nassaji, H. (2014). Lexical inferencing strategies: The case of successful versus less successful inferencers. System, 45, 27–38.

    Article  Google Scholar 

  • Hughes, A. (2003). Testing for language teachers (2nd ed.). Cambridge: Cambridge University Press.

    Google Scholar 

  • Iannone, P., & Simpson, A. (2015). Students’ preferences in undergraduate mathematics assessment. Studies in Higher Education, 40, 1046–1067.

    Article  Google Scholar 

  • Jamieson, J. (1992). The cognitive styles of reflection/impulsivity and field independence and ESL success. Modern Language Journal, 76, 491–501.

    Article  Google Scholar 

  • Jaswal, V., & Markman, E. (2001). Learning proper and common names in inferential vs. ostensive contexts. Child Development, 72(3), 768–776.

    Article  PubMed  Google Scholar 

  • Kaivanpanah, S., & Alavi, S. (2008). The role of linguistic knowledge in word-meaning inferencing. System, 36, 172–195.

    Article  Google Scholar 

  • Kang, S. (2005). Dynamic emergence of situational willingness to communicate in a second language. System, 33, 277–291.

    Article  Google Scholar 

  • Kastner, M., & Stangla, B. (2011). Multiple-choice and constructed response tests: Do test format and scoring matter? Procedia-Social and Behavioral Sciences, 12, 263–273.

    Article  Google Scholar 

  • Keck, C. (2006). The use of paraphrase in summary writing: A comparison of L1 and L2 writers. Journal of Second Language Writing, 15, 261–278.

    Article  Google Scholar 

  • Keshavarz, M., & Salimi, H. (2007). Collocational competence and cloze test performance: A study of Iranian EFL learners. International Journal of Applied Linguistics, 17(1), 81–92.

    Article  Google Scholar 

  • Khodadady, E., Fatemi, A., & Etminan, S. (2012). Cognitive styles and performance on schema-based cloze multiple-choice item tests: A fairness issue. Journal of Language Teaching and Research, 3(4), 806–813.

    Google Scholar 

  • Khodadady, E., & Zeynaly, S. (2012). Field-dependence/independence cognitive style and performance on the IELTS listening comprehension. International Journal of Linguistics, 4(3), 622–635.

    Article  Google Scholar 

  • Landis, J., & Koch, G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174.

    Article  PubMed  Google Scholar 

  • Larsen-Freeman, D. (2001). Individual cognitive/affective learner contributions and differential success in second language acquisition. In M. Breen (Ed.), Learner contributions to language learning (pp. 12–24). Harlow: Pearson Education Limited.

    Google Scholar 

  • Laufer, B., & Nation, P. (1995). Vocabulary size and use: Lexical richness in L2 written production. Applied Linguistics, 16, 307–322.

    Article  Google Scholar 

  • Leyu, Q. (2001). A consideration of learners’ individual differences in classroom language teaching. Memoires of Fukui University of Technology, 31(2), 79–86.

    Google Scholar 

  • Liu, J. (1998). The effect of test methods on testing reading. Foreign Language Teaching and Research, 2, 48–52.

    Article  Google Scholar 

  • Liu, F. (2009). The effect of three test methods on reading comprehension: An experiment. Asian Social Science, 5(6), 147–153.

    Article  Google Scholar 

  • Ma, Q. (2013). Matching vocabulary learning process with learning outcome in L2 academic writing: An exploratory case study. Linguistics and Education, 24, 237–246.

    Article  Google Scholar 

  • MacIntyre, P. (2001). Conceptualizing willingness to communicate in a L2: A situational model of L2 confidence and affiliation. The Modern Language Journal, 82, 545–562.

    Article  Google Scholar 

  • MacIntyre, P., Babin, P., & Clément, R. (1999). Willingness to communicate: Antecedents and consequences. Communication Quarterly, 47(2), 215–233.

    Article  Google Scholar 

  • MacIntyre, P., Baker, S., Clement, R., & Conrad, S. (2001). Willingness to communicate, social support, and language-learning orientations of immersion students. Studies in Second Language Acquisition, 23, 369–388.

    Article  Google Scholar 

  • MacIntyre, P., & Charos, C. (1996). Personality, attitudes, and affect as predictors of second language communication. Journal of Language and Social Psychology, 5, 3–26.

    Article  Google Scholar 

  • MacIntyre, P., Clément, R., Dornyei, Z., & Noels, K. (1998). Conceptualizing willingness to communicate in a L2: A situational model of L2 confidence and affiliation. The Modern Language Journal, 82, 545–562.

    Article  Google Scholar 

  • MacIntyre, P., & Gardner, R. (1991). Language anxiety: Its relationship to other anxieties and to processing in native and second language. Language Learning, 41, 513–534.

    Article  Google Scholar 

  • MacIntyre, P., Noels, K., & Clement, R. (1997). Biases in self-ratings of second language proficiency: The role of language anxiety. Language Learning, 47(2), 265–287.

    Article  Google Scholar 

  • Messick, S. (1978). Personality consistencies in cognition and creativity. In S. Messick & Associates (Eds.), Individuality in learning: Implications of cognitive style and creativity for human development (pp. 4–33). San Francisco, CA: Jossey-Bass.

    Google Scholar 

  • Nassaji, H. (2003). L2 vocabulary learning from context: Strategies, knowledge sources, and their relationship with success in L2 lexical inferencing. TESOL Quarterly, 37(4), 645–670.

    Article  Google Scholar 

  • Nassaji, H. (2004). The relationship between depth of vocabulary knowledge and L2 learners’ lexical inferencing strategy use and success. Canadian Modern Language Review, 61(1), 107–134.

    Article  Google Scholar 

  • Nicolaou, A., & Xistouri, X. (2011). Field dependence/independence cognitive style and problem posing: An investigation with sixth grade students. Educational Psychology, 31(5), 611–627.

    Article  Google Scholar 

  • O’Brein, T., Butler, S., & Bernold, L. (2001). Group embedded figure test and academic achievement in engineering education. Journal of Engineering Education, 17(1), 89–92.

    Google Scholar 

  • Pishghadam, R., & Tabatabaian, M. (2011). Emotional intelligence: Can it be a predictor of performance on different test formats? International Journal of Linguistics, 3(1), 1–21.

    Article  Google Scholar 

  • Richardson, J., & Turner, T. (2000). Field dependence revisited I: Intelligence. Educational Psychology, 20, 255–270.

    Article  Google Scholar 

  • Saito, Y., & Samimy, K. (1996). Foreign language anxiety and language performance: A study of learner anxiety in beginning, intermediate, and advanced level college students of Japanese. Foreign Language Annals, 29, 239–251.

    Article  Google Scholar 

  • Salmani-Nodoushan, M. (2007). Is field dependence or independence a predicator of EFL reading performance? TESL Canada Journal, 24(2), 82–108.

    Article  Google Scholar 

  • Scouller, K., & Prosser, M. (1994). Students’ experiences in studying for multiple-choice question examinations. Studies in Higher Education, 19(3), 267–279.

    Article  Google Scholar 

  • Shohamy, E. (2014). The power of tests: A critical perspective on the uses of language tests. London: Routledge.

    Google Scholar 

  • Skehan, P. (1998). A cognitive approach to language learning. Oxford: Oxford University Press.

    Google Scholar 

  • Snow, R. (1989). Cognitive-conative aptitude interactions in learning. In R. Kanfer, P. Ackerman, & R. Cudeck (Eds.), Abilities, motivation, and methodology (pp. 435–474). Hillsdale, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Spielberger, C., & Gorsuch, R. (1983). Manual for the state-trait anxiety inventory: Self-evaluation questionnaire. Palo Alto, CA: Consulting Psychologists Press.

    Google Scholar 

  • Swain, M. (2000). The output hypothesis and beyond: Mediating acquisition through collaborative dialogue. In J. Lantolf (Ed.), Sociocultural theory and second language learning (pp. 97–114). Oxford: Oxford University Press.

    Google Scholar 

  • Swain, M., & Lapkin, S. (2002). Talking it through: Two French immersion learners’ response to reformulation. International Journal of Educational Research, 37, 285–304.

    Article  Google Scholar 

  • Teemant, A. (2010). ESL student perspectives on university classroom testing practices. Journal of the Scholarship of Teaching and Learning, 10(3), 89–105.

    Google Scholar 

  • Tinajero, C., & Paramo, M. (1998). Field dependence-independence cognitive style and academic achievement: A review of research and theory. European Journal of Psychology of Education, 13, 227–251.

    Article  Google Scholar 

  • Traub, R., & MacRury, K. (1990). Multiple-choice vs. free response in the testing of scholastic achievement. In K. Ingenkamp & R. Jager (Eds.), Tests und trends 8: Jahrbuch der Pädagogischen Diagnostik (pp. 128–159). Weinheim und Basel: Beltz.

    Google Scholar 

  • Tremblay, A. (2011). Proficiency assessment standards in second language acquisition research: ‘Clozing’ the gap. Studies in Second Language Acquisition, 33, 339–372.

    Article  Google Scholar 

  • Tsaparlis, G. (2005). Non-algorithmic quantitative problem solving in university physical chemistry: A correlation study of the role of selective cognitive factors. Research in Science and Technological Education, 23, 125–148.

    Article  Google Scholar 

  • University of Cambridge Local Examinations Syndicate. (2002). Certificate of proficiency in English test. Cambridge: UCLES.

    Google Scholar 

  • Walstad, W., & Becker, W. (1994). Achievement differences on multiple-choice and essay tests in economics. American Economic Review, 84, 193–196.

    Google Scholar 

  • Wang, Y. (2004). The relationship between second language written performance and the level of willingness to communicate in class: A quantitative analysis of a second-year Chinese class. In The Proceedings of the Biennial Conference of the Asian Studies Association of Australia. Canberra.

  • Watering, G., Gijbels, D., Dochy, F., & Rijt, J. (2008). Students’ assessment preferences, perceptions of assessment, and their relationships to study results. High Education, 56, 645–658.

    Article  Google Scholar 

  • Weaver, C. (2005). Using the Rasch model to develop a measure of second language learners’ willingness to communicate within a language classroom. Journal of Applied Measurement, 6, 396–415.

    PubMed  Google Scholar 

  • Wen, W.P., & Clément, R. (2010) A Chinese conceptualisation of willingness to communicate in ESL. Language, Culture and Curriculum, 16(1), 18–38.

    Article  Google Scholar 

  • Wesche, M., & Paribakht, T. (2010). Lexical inferencing in a first and second language: Cross-linguistic dimensions. Bristol: Multilingual Matters.

    Google Scholar 

  • Widdowson, H. (1979). Explorations in applied linguistics. Oxford: Oxford University Press.

    Google Scholar 

  • Witkin, H., Moore, C., Goodenough, D., & Cox, P. (1977). Field dependent and field independent cognitive styles and their educational implications. Review of Educational Research, 47, 1–64.

    Article  Google Scholar 

  • Witkin, H., Oltman, P., Raskin, E., & Karp, S. (1971). A manual for the embedded figures test. Palo Alto: Consulting Psychologists Press.

    Google Scholar 

  • Wyss, R. (2002). Field independent/dependent learning styles and L2 acquisition. Journal of ELT, 49, 125–128.

    Google Scholar 

  • Xie, Q. (2011). Willingness to communicate in English among secondary school students in the rural Chinese English as a foreign language (EFL) classroom (unpublished doctoral dissertation). Auckland: AUT University.

  • Yashima, T. (2002). Willingness to communicate in second language: The Japanese EFL context. Modern Language Journal, 86, 55–66.

    Article  Google Scholar 

  • Yashima, T., Zenuk-Nishide, L., & Shimizu, K. (2004) The influence of attitudes and affect on willingness to communicate and second language communication. Language Learning, 54(1), 119–152.

    Article  Google Scholar 

  • Zarei, F. (2012). Can ambiguity tolerance/intolerance be a source of bias on C-tests? US-China Foreign Language, 10(3), 1010–1019.

    Google Scholar 

  • Zhang, L. (2004). Field-dependence/independence: Cognitive style or perceptual ability? Personality and Individual Differences, 37(6), 1295–1311.

    Article  Google Scholar 

  • Zhang, L. (2008). Constructivist pedagogy in strategic reading instruction: Exploring pathways to learner development in the English as a second language classroom. Instructional Science, 36(2), 89–116.

    Article  Google Scholar 

  • Zhang, L., & Wu, A. (2009). Chinese senior high school EFL students’ metacognitive awareness and reading-strategy. Reading in a Foreign Language, 21(1), 37–59.

    Google Scholar 

  • Zheng, Y., Cheng, L., & Klinger, D. (2007). Do test formats in reading comprehension affect second-language students’ test performance differently? TESL Canada Journal, 25(1), 65–80.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of English Language Studies, Shahid Bahonar University of Kerman, Kerman, Iran

    Shima Ghahari & Sima Ahmadinejad

Authors
  1. Shima Ghahari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sima Ahmadinejad
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shima Ghahari.

Ethics declarations

Conflict of interest

Authors declare no conflicts of interest with respect to the content, authorship, and/or publication of this article.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ghahari, S., Ahmadinejad, S. Performance on a Triangulated Reading Test Battery: A Study of Language Learners’ Individual Differences and Retrospective Perceptions. Psychol Stud 61, 245–258 (2016). https://doi.org/10.1007/s12646-016-0364-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12646-016-0364-8

Keywords

Search

Navigation