Abstract
The core goals for education systems in the twenty-first century have shifted from teaching clearly defined knowledge and skills to promoting lifelong learners who are eager, as well as able, to face the demands and challenges of a truly global society. As a reflection of the emerging policy interest in a more comprehensive definition of adolescent and young adult success, national and international large-scale assessments (LSAs) have started broadening their focus to include noncognitive outcomes in addition to achievement results. In this chapter we describe the opportunities and challenges associated with noncognitive outcome assessments, with a special focus on the implementation in the Programme for International Student Assessment (PISA) 2015. We provide research-based rationales why LSAs should focus on noncognitive outcomes, which specific outcomes deserve most attention, and how measurement approaches can be enhanced for more validity and efficiency. Two main categories of noncognitive outcomes are described in more detail: first, constructs that can be tied back to the Big Five model of human personality; second, variables representing well-being or general quality of life indicators, comprising factors such as health, subjective well-being, and psychological functioning. Examining the interplay between cognitive and noncognitive factors is essential to better understand what policies can promote learning and prepare students to face twenty-first century challenges. National and international LSAs can shed light on the factors that are most strongly associated, either positively or negatively, with student development. The success of LSAs as helpful tools and monitoring systems for policy makers and educators will depend to a growing extent on how well they can serve the policy and public demand for enhanced measurement of noncognitive outcomes.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
This chapter expands on a technical paper that was presented to the PISA 2015 Questionnaire Expert Group (QEG) in May 2012 (Doc. QEG 2012 − 05 Doc 10).
References
Abraham, C. (2012). Mapping change mechanisms and behavior change techniques: A systematic approach to promoting behavior change through text. In C. Abraham & M. Kools (Eds.), Writing health communication: An evidence-based guide for professionals (pp. 99–116). London: Sage.
Ackerman, P. L., & Ellingsen, V. J. (2014). Vocabulary overclaiming: A complete approach. Ability, personality, self-concept correlates, and gender differences. Intelligence, 46, 2016–2227.
Ajzen, I. (1987). Attitudes, traits, and actions: Dispositional prediction of behavior in personality and social psychology. Advances in Experimental Social Psychology, 20, 1–63.
Allen, J., van der Velden, R., Helmschrott, S., Martin, S., Massing, N., Rammstedt, B., & von Davier, M. (2013). The development of the PIAAC background questionnaires. In OECD (Ed.), Technical report of the survey of adult skills (PIAAC) (pp. 1–48). Paris: OECD.
Almlund, M., Duckworth, A. L., Heckman, J. J., & Kautz, T. D. (2011). Personality psychology and economics (No. w16822). Bonn: IZA. http://nbn-resolving.de/urn:nbn:de:101:1-201104113733. Accessed 16 June 2016.
Ashford, S., Edmunds, J., & French, D. P. (2010). What is the best way to change self‐efficacy to promote lifestyle and recreational physical activity? A systematic review with meta‐analysis. British Journal of Health Psychology, 15(2), 265–288.
Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84(2), 191.
Bandura, A. (1994). Self‐efficacy. New York: Wiley.
Ben-Arieh, A. (2008). The child indicators movement: Past, present, and future. Child Indicators Research, 1(1), 3–16. doi:10.1007/s12187-007-9003-1.
Benet-Martinez, V., & John, O. P. (1998). Los Cinco Grandes across cultures and ethnic groups: Multitrait multimethod analyses of the Big Five in Spanish and English. Journal of Personality and Social Psychology, 75(3), 729–750. doi:10.1037/0022-3514.75.3.729.
Benet-Martínez, V., & Karakitapoglu-Aygün, Z. (2003). The interplay of cultural syndromes and personality in predicting life satisfaction comparing Asian Americans and European Americans. Journal of Cross-Cultural Psychology, 34(1), 38–60.
Berry, C. M., Ones, D. C., & Sackett, P. R. (2007). Interpersonal deviance, organizational deviance, and their common correlates: A review and meta–analysis. Journal of Applied Psychology, 92, 410–424.
Bertling, J. P. (2014, May). Improving the contextual questionnaires for the National Assessment of Educational Progress: Plans for NAEP Core contextual modules. White paper presented at the meeting of the National Assessment Governing Board, Boston, MA.
Bertling, J. P. (2015). The future of the NAEP survey questionnaires: Measuring what matters for student achievement in the U.S. Presentation at the 2015 CCSSO-NCSA Conference. San Diego, CA.
Bertling, J. P., & Kyllonen, P. C. (2013, August). Using anchoring vignettes to detect and correct for response styles in PISA questionnaires. In M. Prenzel (Chair), The attitudes-achievement-paradox: How to interpret correlational patterns in cross-cultural studies. Symposium conducted at the EARLI 2013 Conference, Munich, Germany.
Bertling, J. P., Borgonovi, F., & Almonte, D. E. (2016). Psychosocial skills in large-scale assessments: Trends, challenges, and policy implications. In A. Lipnevich, F. Preckel, & R. Roberts (Eds.), Psychosocial skills and school systems. Berlin: Springer.
Bertrams, A., & Dickhäuser, O. (2009). High–school students’ need for cognition, self–control capacity, and school achievement: Testing a mediation hypothesis. Learning and Individual Differences, 19, 135–138.
Boarini, R., Comola, M., Smith, C., Manchin, R., & de Keulenaer, F. (2012). What makes a better life? The determinants of subjective well-being in OECD countries—Evidence from the Gallup World Poll (OECD Statistics Working Papers, 2012/03). Paris, France: OECD Publishing.
Boarini, R., Kolev, A., & McGregor, A. (2014). Measuring well-being and progress in countries at different stages of development: Towards a more universal conceptual framework (OECD Development Centre Working Papers, No. 325). Paris: OECD Publishing. doi:10.1787/5jxss4hv2d8n-en.
Bong, M., & Clark, R. E. (1999). Comparison between self-concept and self-efficacy in academic motivation research. Educational Psychologist, 34, 139–153.
Borghans, L., Duckworth, A. L., Heckman, J. J., & ter Weel, B. (2008). The economics and psychology of personality traits. Journal of Human Resources, 43(4), 972–1059.
Bowles, S., Gintis, H., & Osborne, M. (2001). The determinants of earnings: A behavioral approach. Journal of Economic Literature, 39(4), 1137–1176. doi:10.1257/jel.39.4.1137.
Bradshaw, J., Keung, A., Rees, G., & Goswami, H. (2011). Children’s subjective well-being: International comparative perspectives. Children and Youth Services Review, 33(4), 548–556.
Briley, D. A., Domiteaux, M., & Tucker-Drob, E. M. (2014). Achievement-relevant personality: Relations with the Big Five and validation of an efficient instrument. Learning and Individual Differences, 32, 26–39.
Brown, A., & Maydeu-Olivares, A. (2011). Item response modeling of forced-choice questionnaires. Educational and Psychological Measurement, 71(3), 460–502.
Buckley, J. (2009). Cross-national response styles in international educational assessments: Evidence from PISA 2006. https://edsurveys.rti.org/PISA/documents/Buckley_PISAresponsestyle.pdf. Accessed 16 June 2016.
Cacioppo, J. T., & Petty, R. E. (1982). The need for cognition. Journal of Personality and Social Psychology, 42(1), 116–131.
Cacioppo, J. T., Petty, R. E., & Kao, C. F. (1984). The efficient assessment of need for cognition. Journal of Personality Assessment, 48, 306–307.
Cacioppo, J. T., Petty, R. E., Feinstein, J. A., & Jarvis, W. B. G. (1996). Dispositional differences in cognitive motivation: The life and times of individuals varying in need for cognition. Psychological Bulletin, 119(2), 197–253.
Carlson, S. M., Zelazo, P. D., & Faja, S. (2013). Executive function. In P. D. Zelazo (Ed.), The Oxford handbook of developmental psychology (Body and Mind, Vol. 1, pp. 706–743). New York: Oxford University Press.
Casas, F. (2011). Subjective social indicators and child and adolescent well-being. Child Indicators Research, 4(4), 555–575.
CASEL—Collaborative for Academic, Social, and Emotional Learning. (2013). 2013 CASEL guide: Effective social and emotional learning programs—preschool and elementary school. Chicago: Author.
Cohen, J., McCabe, L., Michelli, N. M., & Pickeral, T. (2009). School climate: Research, policy, practice, and teacher education. Teachers College Record, 111(1), 180–213.
Comber, L. C., & Keeves, J. P. (1973). Science education in nineteen countries: An empirical study. Stockholm: Almqvist & Wiksell.
Converse, J. M., & Presser, S. (1986). Survey questions: Handcrafting the standardized questionnaire (Vol. 63). Thousand Oaks: Sage.
Costa, P. T., & McCrae, R. R. (1992). Revised NEO Personality Inventory (NEO–PI–R) and NEO Five–Factor Inventory (NEO–FFI) professional manual. Odessa: Psychological Assessment Resources.
Cummins, R., & Lau, A. (2005). Personal wellbeing index—School children (3rd ed.). Melbourne: School of Psychology, Deakin University.
De Fruyt, F., & De Clercq, B. (2014). Antecedents of personality disorder in childhood and adolescence: Toward an integrative developmental model. Annual Review of Clinical Psychology, 10(1), 449–476.
De Fruyt, F., Bockstaele, M., Taris, R., & Van Hiel, A. (2006). Police interview competencies: Assessment and associated traits. European Journal of Personality, 20(7), 567–584.
De Fruyt, F., De Bolle, M., MaCrae, R. R., Terracciano, A., & Costa, P. T., Jr. (2009). Assessing the universal structure of personality in early adolescence with the NEO-PI-R and NEO-PI-3 in 24 Cultures. Assessment, 16(3), 301–311. doi:10.1177/1073191109333760.
Diener, E., Lucas, R. E., & Oishi, S. (2005). Subjective well-being. In C. R. Snyder & S. J. Lopez (Eds.), Handbook of positive psychology (pp. 63–73). Oxford: University Press.
Diener, E., Oishi, S., & Lucas, R. E. (2015). National accounts of subjective well-being. American Psychologist, 70(3), 234–242.
Dillman, D. A., Smyth, J. D., & Christian, L. M. (2014). Internet, phone, mail, and mixed-mode surveys: The tailored design method. Hoboken: Wiley.
Dolan, P., Peasgood, T., & White, M. (2008). Do we really know what makes us happy? A review of the economic literature on the factors associated with subjective well-being. Journal of Economic Psychology, 29(1), 94–122.
Duckworth, A. L., & Carlson, S. M. (2014). Self-regulation and school success. In B. W. Sokol, F. M. E. Grouzet, & U. Mueller (Eds.), Self-regulation and autonomy: Exploring the social, developmental, educational, and neurological dimensions of human conduct (pp. 208–230). New York: Cambridge.
Duckworth, A. L., & Yeager, D. S. (2015). Measurement matters: Assessing personal qualities other than cognitive ability for educational purposes. Educational Researcher, 44, 237–251.
Duckworth, A. L., Peterson, C., Matthews, M. D., & Kelly, D. R. (2007). Grit: Perseverance and passion for long-term goals. Journal of Personality and Social Psychology, 92(6), 1087–1101. doi:10.1037/0022-3514.92.6.1087.
Durlak, J. A., Weissberg, R. P., & Pachan, M. (2010). A meta-analysis of after-school programs that seek to promote personal and social skills in children and adolescents. American Journal of Community Psychology, 45, 294–309. doi:10.1007/s10464-010-9300-6.
Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The impact of enhancing students’ social and emotional learning: A meta‐analysis of school‐based universal interventions. Child Development, 82(1), 405–432. doi:10.1111/j.1467-8624.2010.01564.x.
Dweck, C. (2006). Mindset: The new psychology of success. New York: Random House.
Dweck, C. S., Walton, G. M., & Cohen, G. L. (2011). Academic tenacity: Mindsets and skills that promote long-term learning. White paper prepared for the Gates Foundation. Seattle, WA.
Evans, J., Macrory, I., & Randall, C. (2015). Measuring national well-being: Life in the UK, 2015. London: Office for National Statistics.
Farrington, C. A., Roderick, M., Allensworth, E., Nagaoka, J., Keyes, T. S., Johnson, D. W., & Beechum, N. O. (2012). Teaching adolescents to become learners. The role of noncognitive factors in shaping school performance: A critical literature review. Chicago: University of Chicago Consortium on Chicago School Research.
Federal Interagency Forum on Child and Family Statistics. (2009). America’s children: Key national indicators of well-being. Washington, DC: US Government Printing Office. http://www.childstats.gov/pdf/ac2009/ac_09.pdf. Accessed 16 June 2016.
Feeney, J. R., & Goffin, R. D. (2015). The overclaiming questionnaire: A good way to measure faking? Personality and Individual Differences, 82, 248–252.
Ferla, J., Valcke, M., & Cai, Y. (2009). Academic self-efficacy and academic self-concept: Reconsidering structural relationships. Learning and Individual Differences, 19(4), 499–505.
Flynn, F. J. (2005). Having an open mind: The impact of openness to experience on interracial attitudes and impression formation. Journal of Personality and Social Psychology, 88(5), 816–826.
Fowler, F. J. (2009). Survey research methods (4th ed.). Thousand Oaks: Sage.
Gabrieli, C. (2015, March). Non-cognitive skills and education policy: Research and practice considerations. In: M. R. West & G. J. Whitehurst (Chairs), Ready to be counted? Incorporating noncognitive skills into education policy. Symposium at the Brown Center on Education Policy, Washington, DC.
Galton, F. (1892). Hereditary genius: An inquiry into its laws and consequences. London: Macmillan.
Garrow, J. S., & Webster, J. (1985). Quetelet’s index (W/H2) as a measure of fatness. International Journal of Obesity, 9(2), 147–153.
Gehlbach, H. (2015a, April 15). Name that baby: Why ‘non-cognitive’ factors need a new name. Education Week Blog. http://blogs.edweek.org/edweek/rick_hess_straight_up/2015/04/non-cognitive_factors_need_new_name.html. Accessed 16 June 2016.
Gehlbach, H. (2015b). Seven survey sins. The Journal of Early Adolescence, 35(5–6), 883–897. doi:10.1177/0272431615578276.
Gilman, R., & Huebner, S. (2003). A review of life satisfaction research with children and adolescents. School Psychology Quarterly, 18(2), 192–205.
Ginsburg, A., & Smith, M. S. (2013, December). Key Education Indicators (KEI): Making sense of NAEP contextual variables. Presentation at the National Assessment Governing Board Meeting, Baltimore, MD.
Goldberg, L. R. (1993). The structure of phenotypic personality traits. American Psychologist, 48(1), 26–34.
Goldberg, L. R., Johnson, J. A., Eber, H. W., Hogan, R., Ashton, M. C., Cloninger, C. R., et al. (2006). The international personality item pool and the future of public-domain personality measures. Journal of Research in Personality, 40(1), 84–96.
Goodman, R. (2001). Psychometric properties of the strengths and difficulties questionnaire. Journal of the American Academy of Child & Adolescent Psychiatry, 40(11), 1337–1345.
Hardy, C. L., Bukowski, W. M., & Sippola, L. K. (2002). Stability and change in peer relationships during the transition to middle-level school. Journal of Early Adolescence, 22, 117–142. doi:10.1177/0272431602022002001.
Heckman, J. J., & Kautz, T. (2013). Fostering and measuring skills: Interventions that improve character and cognition (IZA Discussion Paper No. 7750). Bonn, Germany: Institute for the Study of Labor.
Heckman, J. J., Stixrud, J., & Urzua, S. (2006). The effects of cognitive and noncognitive abilities on labor market outcomes and social behavior. Journal of Labor Economics, 24, 411–482.
Helliwell, J., Layard, R., & Sachs, J. (2015). World happiness report. The earth institute, Columbia University. http://hdl.handle.net/2429/44498. Accessed 16 June 2016.
Hopkins, D., & King, G. (2010). Improving anchoring vignettes: Designing surveys to correct interpersonal incomparability. Public Opinion Quarterly, 74, 1–22.
Huebner, E. S., & Dew, T. (1996). The interrelationships of positive affect, negative affect, and life satisfaction in an adolescent sample. Social Indicators Research, 38(2), 129–137.
Huebner, E. S., Suldo, S. M., Smith, L. C., & McKnight, C. G. (2004). Life satisfaction in children and youth: Empirical foundations and implications for school psychologists [Special issue]. Psychology in the Schools, 41, 81–93.
James, W. (1907). The energies of men. Science, 25, 321–332.
Janosz, M., LeBlanc, M., Boulerice, B., & Tremblay, R. E. (1997). Disentangling the weight of school dropout predictors: A test on two longitudinal samples. Journal of Youth and Adolescence, 26(6), 733–762.
Jebb, A. T., Saef, R., Parrigon, S., & Woo, S. E. (2016). The need for cognition: Key concepts, assessment, and role in educational outcomes. In A. Lipnevich, F. Preckel, & R. Roberts (Eds.), Psychosocial skills and school systems. Berlin: Springer.
Jencks, C. (1979). Who gets ahead? The determinants of economic success in America. New York: Basic Books.
John, O. P. (1990). The “Big Five” factor taxonomy: Dimensions of personality in the natural language and in questionnaires. In L. Pervin (Ed.), Handbook of personality: Theory and research (pp. 66–100). New York: Guilford Press.
John, O. P., & DeFruyt, F. (2015). Framework for the longitudinal study of social and emotional skills in cities. Paris: OECD.
John, O. P., & Srivastava, S. (1999). The Big Five trait taxonomy: History, measurement, and theoretical perspectives. In L. A. Pervin & O. P. John (Eds.), Handbook of personality: Theory and research (2nd ed., pp. 102–138). New York: Guilford.
Jourdan, D., Samdal, O., Diagne, F., & Carvalho, G. S. (2008). The future of health promotion in schools goes through the strengthening of teacher training at a global level. Promotion and Education, 15(3), 36–38.
Jude, N. (2016). The assessment of learning contexts in PISA. In S. Kuger, E. Klieme, N. Jude, & D. Kaplan (Eds.), Assessing contexts of learning: An international perspective. Dordrecht: Springer.
Judge, T. A., & Bono, J. E. (2001). Relationship of core self-evaluations traits—Self-esteem, generalized self-efficacy, locus of control, and emotional stability—With job satisfaction and job performance: A meta-analysis. Journal of Applied Psychology, 86(1), 80–92.
Kahneman, D., Krueger, A. B., Schkade, D. A., Schwarz, N., & Stone, A. A. (2004). A survey method for characterizing daily life experience: The day reconstruction method. Science, 306, 1776–1780.
Kaplan, D., & Kuger, S. (2016). The methodology of PISA: Past, present, and future. In S. Kuger, E. Klieme, N. Jude, & D. Kaplan (Eds.), Assessing contexts of learning: An international perspective. Dordrecht: Springer.
Kaplan, D., & Su, D. (2016). On matrix sampling and imputation of context questionnaires with implications for the generation of plausible values in large-scale assessments. Journal of Educational and Behavioral Statistics, 41(1), 57–80.
Kaplan, D., & Wu, D. (2014, April). Imputation issues relevant to context questionnaire rotation. In J. P. Bertling (Chair), Spiraling contextual questionnaires in educational large-scale assessments. Symposium conducted at the meeting of the National Council on Measurement in Education, Philadelphia, PA.
Kautz, T., Heckman, J. J., Diris, R., Ter Weel, B., & Borghans, L. (2014). Fostering and measuring skills: Improving cognitive and non-cognitive skills to promote lifetime success (No. w20749). National Bureau of Economic Research. http://www.nber.org/papers/w20749. Accessed 16 June 2016.
Keehner, M., Agard, C., Berger, M., Shu, Z., Bertling, J., & Carney, L. (2014). Analyzing data from the NAEP TEL Wells task: Potential reporting insights from interactions, context, and scores (Research Memorandum on NAEP TEL Task Component). Princeton: Educational Testing Service.
King, G., & Wand, J. (2007). Comparing incomparable survey responses: Evaluating and selecting anchoring vignettes. Political Analysis, 15, 46–66.
Kyllonen, P. C. (2016). Designing tests to measure interpersonal attributes and noncognitive skills. In S. Lane, M. R. Raymond, & T. M. Haladyna (Eds.), Handbook of test development (2nd ed., pp. 190–211). New York: Routledge.
Kyllonen, P. C., & Bertling, J. P. (2013). Innovative questionnaire assessment methods to increase cross-country comparability. In L. Rutkowski, M. von Davier, & D. Rutkowski (Eds.), Handbook of international large-scale assessment: Background, technical issues, and methods of data analysis (pp. 277–286). Boca Raton: CRC Press.
Kyllonen, P. C., & Bertling, J. P. (2014). Anchoring vignettes reduce Bias in noncognitive rating scale responses. Report submitted to OECD.
Levin, H. M. (2012). More than just test scores. Prospects, 42(3), 269–284.
Lippman, L. H., Moore, K. A., & McIntosh, H. (2011). Positive indicators of child well-being: A conceptual framework, measures, and methodological issues. Applied Research in Quality of Life, 6(4), 425–449.
Lleras, C. (2008). Do skills and behaviors in high school matter? The contribution of noncognitive factors in explaining differences in educational attainment and earnings. Social Science Research, 37(3), 888–902.
Long, J. E., & Caudill, S. B. (1991). The impact of participation in intercollegiate athletics on income and graduation. The Review of Economics and Statistics, 73(3), 525–531.
Martin, M. O., & Mullis, I. V. (2013). TIMSS and PIRLS 2011: Relationships among reading, mathematics, and science achievement at the fourth grade–Implications for early learning. Amsterdam: International Association for the Evaluation of Educational Achievement.
McCrae, R. R., & Costa, P. T. (1989). The structure of interpersonal traits: Wiggin’s circumplex and the five-factor model. Journal of Personality and Social Psychology, 45, 586–595.
McCrae, R. R., & Costa, P. T. (1996). Toward a new generation of personality inventories: Theoretical contexts for the five-factor model. In J. S. Wiggins (Ed.), The five-factor model of personality: Theoretical perspectives (pp. 51–87). New York: Guilford Press.
McGill, R. K., Hughes, D., Alicea, S., & Way, N. (2012). Academic adjustment across middle school: The role of public regard and parenting. Developmental Psychology, 48, 1003–1018. doi:10.1037/a0026006.
Mervielde, I., & De Fruyt, F. (2002). Assessing children’s traits with the hierarchical personality inventory for children. In B. De Raad & M. Perugini (Eds.), Big Five assessment (pp. 129–146). Gottingen: Hogrefe and Huber Publishers.
Mickelson, R. A. (1990). The attitude-achievement paradox among Black adolescents. Sociology of Education, 63(1), 44–61.
Nagaoka, J., Farrington, C. A., Ehrlich, S. B., & Heath, R. (2015). Foundations for young adult success: A developmental framework. Chicago: University of Chicago Consortium on Chicago School Research.
Nansel, T. R., Overpeck, M., Pilla, R. S., Ruan, W. J., Simons-Morton, B., & Scheidt, P. (2001). Bullying behaviors among US youth: Prevalence and association with psychosocial adjustment. JAMA, 285(16), 2094–2100.
National Academy of Sciences. (2012). Education for life and work: Developing transferable knowledge and skills in the 21st century. Washington, DC: National Academies Press.
National Assessment Governing Board. (2012). Policy statement on NAEP background questions and the use of contextual data in NAEP reporting. Washington, DC: U.S. Department of Education, National Assessment Governing Board.
Nyhus, E. K., & Pons, E. (2005). The effects of personality on earnings. Journal of Economic Psychology, 26(3), 363–384.
O’Connor, M. C., & Paunonen, S. V. (2007). Big Five personality predictors of post-secondary academic performance. Personality and Individual Differences, 43(5), 971–990.
OECD. (2010). Pathways to success: How knowledge and skills at age 15 shape future lives in Canada. Paris: OECD Publishing.
OECD. (2013a). OECD guidelines on measuring subjective well-being. Paris: OECD Publishing. doi:10.1787/9789264191655-en.
OECD. (2013b). PISA 2012 assessment and analytical framework: Mathematics, reading, science, problem solving and financial literacy. Paris: OECD Publishing.
OECD. (2014a). Education at a glance 2014: OECD indicators. Paris: OECD Publishing.
OECD. (2014b). PISA 2012 technical report. Paris: OECD Publishing.
OECD. (2015a). How’s life? 2015: Measuring well-being. Paris: OECD Publishing.
OECD. (2015b). Call for tender n° 100001311: Longitudinal study of social and emotional skills in cities. Paris: OECD. http://www.oecd.org/callsfortenders/CfT%20100001311%20Longitudinal%20Study%20of%20Social%20and%20Emotional%20Skills%20in%20Cities.pdf. Accessed 10 July 2016.
OECD. (2015c). Universal basic skills: What countries stand to gain. Paris: OECD Publishing.
OECD. (2016). PISA 2015 assessment and analytical framework: Science, reading, mathematic and financial literacy. Paris: OECD Publishing.
Paulhus, D. L., & Dubois, P. J. (2014). Application of the overclaiming technique to scholastic assessment. Educational and Psychological Measurement, 74(6), 975–990.
Paulhus, D. L., Harms, P. D., Bruce, M. N., & Lysy, D. C. (2003). The over-claiming technique: Measuring self-enhancement independent of ability. Journal of Personality and Social Psychology, 84(4), 890–904.
Paunonen, S. V., & Ashton, M. C. (2001). Big five factors and facets and the prediction of behavior. Journal of Personality and Social Psychology, 81(3), 524–539.
Pellegrino, J., & Hilton, M. (Eds.). (2012). Education for life and work: Developing transferable knowledge and skills in the 21st century. Washington, DC: National Academy Press.
Pfeifer, C., & Cornelißen, T. (2010). The impact of participation in sports on educational attainment—New evidence from Germany. Economics of Education Review, 29(1), 94–103.
Pollard, E. L., & Lee, P. D. (2003). Child well-being: A systematic review of the literature. Social Indicators Research, 61(1), 59–78.
Poropat, A. E. (2009). A meta-analysis of the five-factor model of personality and academic performance. Psychological Bulletin, 135(2), 322–338.
Poropat, A. E. (2014a). A meta-analysis of adult-rated child personality and academic performance in primary education. British Journal of Educational Psychology, 84(2), 239–252. doi:10.1111/bjep.12019.
Poropat, A. E. (2014b). Other-rated personality and academic performance: Evidence and implications. Learning and Individual Differences, 34(2014), 24–32.
Pozzebon, J. A., Visser, B. A., Ashton, M. C., Lee, K., & Goldberg, L. R. (2010). Psychometric characteristics of the Oregon vocational interest scales. Journal of Personality Assessment, 92, 168–174. doi:10.1080/00223890903510431.
Preckel, F., Holling, H., & Vock, M. (2006). Academic underachievement: Relationship with cognitive motivation, achievement motivation, and conscientiousness. Psychology in the Schools, 43, 401–411.
Prenzel, M. (2012). Empirische Bildungsforschung morgen: Reichen unsere bisherigen Forschungsansätze aus? [Empirical educational research tomorrow: Do our current research approaches suffice?]. In M. Gläser-Zikuda, T. Seidel, C. Rohlfs, A. Gröschner, & S. Ziegelbauer (Eds.), Mixed Methods in der empirischen Bildungsforschung (pp. 273–285). Münster: Waxmann.
Ravens-Sieberer, U., Kokonyei, G., & Thomas, C. (2004). School and health. In C. Currie, C. Roberts, A. Morgan, R. Smith, W. Settertobulte, O. Samdal, & V. B. Rasmussen (Eds.), Young people’s health in context—health behaviour in school-aged children (HBSC) study: International report from the 2001/2002 survey. Health policy for children and adolescents No. 4 (pp. 184–195). Copenhagen: WHO.
Ravens-Sieberer, U., Herdman, M., Devine, J., Otto, C., Bullinger, M., Rose, M., & Klasen, F. (2014). The European KIDSCREEN approach to measure quality of life and well-being in children: Development, current application, and future advances. Quality of Life Research, 23(3), 791–803.
Richardson, M., Abraham, C., & Bond, R. (2012). Psychological correlates of university students’ academic performance: A systematic review and meta–analysis. Psychological Bulletin, 138(2), 353–387.
Roberts, B. W., Kuncel, N. R., Shiner, R., Caspi, A., & Goldberg, L. R. (2007). The power of personality: The comparative validity of personality traits, socioeconomic status, and cognitive ability for predicting important life outcomes. Perspectives on Psychological Science, 2, 313–345.
Robertson-Kraft, C., & Duckworth, A. L. (2014). True grit: Trait-level perseverance and passion for long-term goals predicts effectiveness and retention among novice teachers. Teachers College Record (1970), 116(3), http://www.tcrecord.org/Content.asp?ContentId=17352
Ryan, R. M., & Deci, E. L. (2000). Self–determination theory and the facilitation of intrinsic motivation, social development, and well–being. American Psychologist, 55, 68–78.
Rychen, D. S., & Salganik, L. H. (Eds.). (2003). Defining and selecting key competencies. Cambridge, MA: Hogrefe & Huber.
Sacks, D. W., Stevenson, B., & Wolfers, J. (2010). Subjective well-being, income, economic development and growth (No. w16441). National Bureau of Economic Research.
Saha, R., Huebner, E. S., Suldo, S. M., & Valois, R. F. (2010). A longitudinal study of adolescent life satisfaction and parenting. Child Indicators Research, 3(2), 149–165.
Santos, D., & Primi, R. (2014). Social and emotional development and school learning. A measurement proposal in support of public policy. São Paulo: Ayrton Senna Institute.
Saucier, G., & Goldberg, L. R. (2002). Assessing the big five: Applications of 10 psychometric criteria to the development of marker scales. In B. de Raad & M. Perugini (Eds.), Big Five assessment (pp. 29–58). Goettingen: Hogrefe & Huber.
Scalise, K., Mustafic, M., & Greiff, S. (2016). Dispositions for collaborative problem solving. In S. Kuger, E. Klieme, N. Jude, & D. Kaplan (Eds.), Assessing contexts of learning: An international perspective. Dordrecht: Springer.
Schleicher, A. (2015, March). Proposals for PISA 2018 international questionnaire options. Report presented at 39th meeting of the PISA Governing Board. Paris: OECD.
Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124(2), 262–274.
SHL. (2013). OPQ32r technical manual version 1.0.. Thames Ditton: SHL Group.
Soto, C. J., John, O. P., Gosling, S. D., & Potter, J. (2011). Age differences in personality traits from 10 to 65: Big Five domains and facets in a large cross-sectional sample. Journal of Personality and Social Psychology, 100, 330–348.
Specht, J., Egloff, B., & Schmukle, S. C. (2011). Stability and change of personality across the life course: The impact of age and major life events on mean-level and rank-order stability of the Big Five. Journal of Personality and Social Psychology, 101, 862–882.
Spector, P. E. (1982). Behavior in organizations as a function of employee locus of control. Psychological Bulletin, 91, 482–497.
Spengler, M., Lüdtke, O., Martin, R., & Brunner, M. (2013). Personality is related to educational outcomes in late adolescence: Evidence from two large-scale achievement studies. Journal of Research in Personality, 47(5), 613–625. doi:10.1016/j.jrp.2013.05.008.
Stark, S., Chernyshenko, O. S., & Drasgow, F. (2005). An IRT approach to constructing and scoring pairwise preference items involving stimuli on different dimensions: An application to the problem of faking in personality assessment. Applied Psychological Measurement, 29, 184–201.
Statham, J., & Chase, E. (2010). Childhood wellbeing: A brief overview. Loughborough: Childhood Wellbeing Research Centre.
Stiglitz, J., Sen, A., & Fitoussi, J.-P. (2009). Report of the commission on the measurement of economic performance and social progress. http://www.stiglitz-sen-fitoussi.fr/documents/rapport_anglais.pdf. Accessed 16 June 2016.
Suldo, S. M., & Huebner, E. S. (2006). Is extremely high life satisfaction during adolescence advantageous? Social Indicators Research, 78(2), 179–203.
Suldo, S. M., Riley, K. N., & Shaffer, E. J. (2006). Academic correlates of children and adolescents’ life satisfaction. School Psychology International, 27(5), 567–582.
The Children’s Society. (2015). The Good Childhood Report 2015. https://www.childrenssociety.org.uk/sites/default/files/TheGoodChildhoodReport2015.pdf. Accessed 16 June 2016.
Topp, C. W., Østergaard, S. D., Søndergaard, S., & Bech, P. (2015). The WHO-5 well-being index: A systematic review of the literature. Psychotherapy and Psychosomatics, 84(3), 167–176.
Torsheim, T., & Wold, B. (2001). School-related stress, support, and subjective health complaints among early adolescents: A multilevel approach. Journal of Adolescence, 24(6), 701–713.
Tough, P. (2012). How children succeed: Grit, curiosity, and the hidden power of character. New York: Houghton Mifflin Harcourt.
Trautwein, U., & Möller, J. (2016). Self-concept: Determinants and consequences of academic self-concept in school contexts. In A. Lipnevich, F. Preckel, & R. Roberts (Eds.), Psychosocial skills and school systems (pp. 187–214). Berlin: Springer.
Tupes, E. C., & Christal, R. E. (1992). Recurrent personality factors based on trait ratings. Journal of Personality, 60(2), 225–251 (Reprinted from Report No. ASD-TR-61–97 AD-267 778, 1961, Lackland Air Force Base, TX: Personnel Research Laboratory, Wright Air Development Division).
UNICEF. (2007). Child poverty in perspective: An overview of child well-being in rich countries (Inncocenti Report Card 7). Florence: UNICEF.
UNICEF Spain. (2012). Children’s well‐being from their own point of view. What affects the children’s wellbeing in the first year of compulsory secondary education in Spain? Madrid: UNICEF Spain.
Van de Gaer, E., Grisay, A., Schulz, W., & Gebhardt, E. (2012). The reference group effect: An explanation of the paradoxical relationship between academic achievement and self-confidence across countries. Journal of Cross-Cultural Psychology, 43(8), 1205–1228.
Van de Vijver, F. J. R., & He, J. (2016). Bias assessment and prevention in noncognitive outcome measures in context assessments. In S. Kuger, E. Klieme, N. Jude, & D. Kaplan (Eds.), Assessing contexts of learning: An international perspective. Dordrecht: Springer.
Wang, M. T., & Eccles, J. S. (2012). Adolescent behavioral, emotional, and cognitive engagement trajectories in school and their differential relations to educational success. Journal of Research on Adolescence, 22(1), 31–39.
Way, N., & Greene, M. L. (2006). Trajectories of perceived friendship quality during adolescence: The patterns and contextual predictors. Journal of Research on Adolescence, 16, 293–320.
Way, N., Reddy, R., & Rhodes, J. (2007). Students’ perceptions of school climate during the middle school years: Associations with trajectories of psychological and behavioral adjustment. American Journal of Community Psychology, 40, 194–213.
Weekley, J. A., & Ployhart, R. E. (Eds.). (2006). Situational judgment tests: Theory, measurement, and application. Mahway: Erlbaum.
West, M. R., Kraft, M. A., Finn, A. S., Martin, R. E., Duckworth, A. L., Gabrieli, C. F., & Gabrieli, J. D. (2015). Promise and paradox measuring students’ non-cognitive skills and the impact of schooling. Educational Evaluation and Policy Analysis, 38(1), 148–170.
Winkelmann, L., & Winkelmann, R. (1998). Why are the unemployed so unhappy? Evidence from the panel data. Economica, 65(257), 1–15.
Ziegler, M., Danay, E., Heene, M., Asendorpf, J., & Bühner, M. (2012). Openness, fluid intelligence, and crystallized intelligence: Toward an integrative model. Journal of Research in Personality, 46(2), 173–183.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Bertling, J.P., Marksteiner, T., Kyllonen, P.C. (2016). General Noncognitive Outcomes. In: Kuger, S., Klieme, E., Jude, N., Kaplan, D. (eds) Assessing Contexts of Learning. Methodology of Educational Measurement and Assessment. Springer, Cham. https://doi.org/10.1007/978-3-319-45357-6_10
Download citation
DOI: https://doi.org/10.1007/978-3-319-45357-6_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-45356-9
Online ISBN: 978-3-319-45357-6
eBook Packages: EducationEducation (R0)