Abstract
The purpose of this study was to investigate differential performance of boys and girls on open-ended (OE) and multiple-choice (MC) items on the 1988 and 1991 International Assessment of Educational Progress (IAEP) mathematics test. In the 1988 mathematics assessment, a representative sample of approximately 1,000 13-year-olds in each of the six participating countries was assessed. In the 1991 mathematics assessment, a representative sample of 9- and 13-year-olds (approximately 1,650 from each age group) in some 20 participating countries was assessed. Analyses of both assessments yielded results that indicated that boys generally performed better than girls in mathematics. In the 1988 assessment, gender effects were larger on MC items than on OE items, corresponding to results of earlier studies. However, the 1991 IAEP assessment produced contrary results: gender effects tended to be larger for OE items than for MC items. These inconsistent results challenge the assertion that girls perform relatively better on OE test items, and suggest that item format alone cannot account for gender differences in mathematics performance. Further investigation of the data revealed that the inconsistent patterns of gender effects with regard to item format were related to the difficulty level of the items, regardless of item format. Correlations between item difficulty and item gender effect size were computed for age 13 in the 1988 assessment and for ages 9 and 13 in the 1991 assessment. The correlations obtained were 0.26, 0.47, and 0.53, respectively, suggesting that the more difficult the items, the better boys perform relative to girls.
Similar content being viewed by others
REFERENCES
Bell, R. C., & Hay, J. A. (1987). Differences and biases in English language examination formats. British Journal of Educational Psychology, 57, 212–220.
Beller, M., & Gafni, N. (1996). The 1991 International Assessment of Educational Progress in Mathematics and Sciences: The gender differences perspective. Journal of Educational Psychology, 88, 365–377.
Benbow, C. P. (1988). Sex differences in mathematical reasoning ability in intellectually talented preadolescents: Their nature, effects, and possible causes. Behavioral and Brain Sciences, 11, 169–183.
Ben-Shakhar, G., & Sinai, Y. (1991). Gender differences in multiple-choice tests: The role of differential guessing tendencies. Journal of Educational Measurement, 28, 23–35.
Bennet, R. E. (1993). On the meanings of constructed-response. In R. E. Bennet & W. C. Ward (Eds.), Construction versus choice in cognitive measurement: Issues in constructedresponse, performance testing and portfolio assessment (pp. 1–27). Hillsdale, NJ.
Bolger, N. (1984). Gender differences in academic achievement according to method of measurement. Paper presented at the annual meeting of the American Psychological Association, Toronto.
Bolger, N., & Kellaghan, T. (1990). Method of measurement and gender differences in scholastic achievement. Journal of Educational Measurement, 27, 165–174.
Breland, H. M., Danos, D. O., Kahn, H. D., Kubota, M. Y., & Bonner, M. W. (1994). Performance versus objective testing and gender: An exploratory study of an Advanced Placement History Examination. Journal of Educational Measurement, 31, 275–293.
Bridgeman, B., & Morgan, R. (1996). Success in college for students with discrepancies between performance on multiple-choice and essay tests. Journal of Educational Psychology, 88, 333–340.
Duffy, J., & Gunther, G. (1997). Gender and mathematical problem solving. Sex Roles, 37, 477–494.
Feingold, A. (1992). Sex differences in variability: A new look at an old controversy. Review of Educational Research, 62, 61–84.
Gafni, N., & Melamed, E. N. (1994). Differential tendencies to guess as a function of gender and lingual-cultural reference group. Studies in Educational Evaluation, 20, 309–319.
Gelleman, E. S., & Berkowitz, M. (1993). Test item type: What students prefer and why? College Student Journal, 27, 17–26.
Hedges, L. V., & Nowell, A. (1995). Sex differences in mental test scores, variability, and numbers of high-scoring individuals. Science, 269, 41–45.
Hyde, J. S., Fennema, I., & Lamon, S. J. (1990). Gender differences in mathematics performance attitudes/affect: A meta-analysis. Psychological Bulletin, 107, 139–155.
Lapointe, A. E., Mead, N. A., & Askew, J., M. (1992). Learning mathematics. The international assessment of educational progress. Princeton, NJ: Educational Testing Service. Report No. 22-CAEP-01.
Lapointe, A. E., Mead, N. A., & Phillips (1989). A world of differences. An international assessment of mathematics and science. The international Assessment of Educational Progress. Princeton, NJ: Educational Testing Service.
Linn, M. C. (1991). Gender differences in educational achievement. In Sex equity in educational opportunity, achievement, and testing. (Proceedings of the 1991 ETS Invitational Conference). ETS, Princeton, NJ 08541.
Lumis, M., & Stevenson, H. W. (1990). Gender differences in beliefs and achievement: A cross-cultural study. Developmental Psychology, 26, 254–263.
Maccoby, E. E., & Jacklin, C. N. (1974). The psychology of sex differences. Stanford, CA: Stanford University Press.
Mazzeo, J., Schmitt, A., & Bleistein, C. (1993). Sex-related differences on constructed response and multiple-choice sections of Advanced Placement Examinations (CB Report No. 92–7, ETS RR 93–5). New York: College Entrance Examination Board.
Murphy, R. J. L. (1980). Sex differences in GCE Examination entry statistics and success rates. Educational Studies, 6, 169–178.
Murphy, R. J. L. (1982). Sex differences in objective test performance. British Journal of Educational Psychology, 52, 213–219.
Stanley, J. C. (1993). Boys and girls who reason well mathematically. In G. R. Bock & K. Ackrill (Eds.), Ciba Foundation Symposium 178, The origins and development of high ability (pp. 119–138). Chichester, England: Wiley.
Steinkamp, M. W., Harnisch, D. L., Walberg, H. J., & Tsai, S. N. (1985). Cross-national gender differences in mathematics attitude and achievement among 13-year-olds. The Journal of Mathematical Behavior, 4, 259–277.
The 1991 IAEP Assessment. Objectives for mathematics, science, and geography (1991). Educational Testing Service, Princeton NJ.
Traub, R. E., & MacRury, K. (1990). Antwort-Ausswahl vs. Freie-Antwort-Aufgaben Bei Lernerfolgstestes. In Test und Trends 8: Jarbuch der Paedgogischen Diagnostik, K. Ingekamp and R. S. Jager (eds.). Weinheim. Germany: Beltz-Verlag Publishing Co. (English-language version, entitled Multiple-choice vs. free-response in the testing of scholastic achievement, is available from the authors.)
Willingham, W.W., & Cole, N. S. (1997). Gender and fair assessment. Mahwah, NJ: Lawrence Erlbaum Associates.
Wolter, K. M. (1985). Introduction to variance estimation. New York: John Wiley & Sons.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Beller, M., Gafni, N. Can Item Format (Multiple Choice vs. Open-Ended) Account for Gender Differences in Mathematics Achievement?. Sex Roles 42, 1–21 (2000). https://doi.org/10.1023/A:1007051109754
Issue Date:
DOI: https://doi.org/10.1023/A:1007051109754