Skip to main content
SpringerLink
Log in

Reassessing the Assessment of Change in At-Risk Youth: Conflict and Coherence in Overall Versus Contextual Assessments of Behavior

  • Published:
Journal of Psychopathology and Behavioral Assessment Aims and scope Submit manuscript

Abstract

This research examined how a contextual approach to personality assessment can reveal change processes that are obscured by measures of overall behavior frequencies. Using field observations of 336 children from three summers at a program for at-risk youth, we illustrate how children’s social experiences change over time, how their reactions to these experiences change, and how both processes contribute to changes in the overall frequencies of their prosocial, aggressive, and withdrawn behavior. Children showing opposite patterns of change in their environments and their reactions to them were nevertheless similar in their overall amount of change. The results clarify how changes in reactions and social experiences can be disentangled and reintegrated in order to deepen our understanding of personality change processes. Implications for change assessments that rely on overall behavior summaries are highlighted for program, individual, and intra-individual levels of analysis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. Although age and gender were not our main focus, and the small number of girls dictated caution, we summarize the results for difference scores (T2 - T1). For aggressive reactions, there was a main effect for age, F(1, 332) = 4.09, p < .05; younger children showed smaller increases than older ones (Ms = .08, .19). A gender x event interaction was found, F(4, 1328) = 2.59, p < .04. Girls’ aggression to peer talk increased (.24), whereas boys’ decreased (−.12); the reverse was found for reactions to adult instruct (−.05, .07). For withdrawal, there was an age effect (Ms = −.16, –.32 for younger vs. older), F(1, 332) = 6.21, p < .02, and gender (Ms = −.34, –.14, for girls vs. boys), F(1, 332) = 10.73, p < .001, and a 3-way interaction, F(4, 1328) = 4.14, p < .005. Older girls showed steeper decreases in withdrawal to adult warn/discipline than younger girls (Ms = −.52, –.15), but not to peer tease/boss (−.12, –.24). The reverse was found for boys. For prosocial, there was an interaction between gender and type of reaction, F(4, 1328) = 3.11, p < .02. Girls showed steeper increases than boys in their prosocial reactions to adult praise (.36, .13) and to adult instruct (.23, .03), whereas boys showed larger increases in their prosocial reactions to peer talk (.15, –.04).

  2. We performed stepwise regressions with changes in the 5 event rates and 5 reaction rates as predictors, entering age and gender at step 1, and using a .01 entry threshold for other predictors. Age and gender accounted for little variance (R 2 < .03); older children showed less change in withdrawn and prosocial behavior. Both event change and reaction change predicted overall change; event change entered first for aggressive and prosocial behavior. For aggression change the predictors to enter (in order) were adult warn/discipline (∆R 2 = .23), aggression to adult instruct (.09), aggression to adult warn/discipline (.05), and peer tease/boss (.02); R 2 for the full model was .40. For prosocial the predictors were adult warn/discipline (∆R 2 = .16), prosocial to adult instruct (.07), and adult instruct (.02), and full-model R 2 was .28. For withdrawal the predictors were: withdrawal to adult instruct (∆R 2 = .17), adult warn/discipline (.11), and withdrawal to adult warn/discipline (.02), and the full-model R 2 was .32.

References

  • Achenbach, T. M., & Rescorla, L. A. (2001). Manual for the ASEBA school-age forms & profiles. Burlington: University of Vermont.

    Google Scholar 

  • August, G. J., Realmuto, G. M., Hektner, J. M., & Bloomquist, M. L. (2001). An integrated components preventive intervention for aggressive elementary school children: the Early Risers Program. Journal of Consulting and Clinical Psychology, 69(4), 614–626.

    Article  PubMed  Google Scholar 

  • Barkley, R. A. (1988). Child behavior rating scales and checklists. In M. Rutter, A. H. Tuna, & I. S. Lann (Eds.), Assessment and diagnosis in child psychopathology (pp. 113–155). New York: Guilford.

    Google Scholar 

  • Barlow, D. H., & Nock, M. K. (2009). Why can’t we be more idiographic in our research? Perspectives on Psychological Science, 4, 19–21.

    Article  Google Scholar 

  • Block, J. (1989). Critique of the act frequency approach to personality. Journal of Personality and Social Psychology, 2, 234–245.

    Article  Google Scholar 

  • Bryk, A. S., & Raudenbush, S. W. (1992). Hierarchical linear models. Newbury Park: Sage.

    Google Scholar 

  • Cervone, D., Shadel, W. G., & Jencius, S. (2001). Social-cognitive theory of personality assessment. Personality and Social Psychology Review, 5, 33–50.

    Article  Google Scholar 

  • Chambers, J. M., & Hastie, T. J. (1992). Statistical models in S. Pacific Grove: Wadsworth.

    Google Scholar 

  • Connor, D. F., Miller, K. P., Cunningham, J. A., & Melloni, R. H. (2002). What does getting better mean? Child improvement and measure of outcome in residential treatment. The American Journal of Orthopsychiatry, 72, 110–117.

    Article  PubMed  Google Scholar 

  • De Los Reyes, A., & Kazdin, A. (2005). Informant discrepancies in the assessment of childhood psychopathology: a critical review, theoretical framework, and recommendations for further study. Psychological Bulletin, 131, 483–509.

    Article  PubMed  Google Scholar 

  • Dirks, M. A., Treat, T. A., & Weersing, V. R. (2007a). Integrating theoretical, measurement, and intervention models of youth social competence. Clinical Psychology Review, 27, 327–347.

    Article  PubMed  Google Scholar 

  • Dirks, M. A., Treat, T. A., & Weersing, V. R. (2007b). The situation specificity of youth responses to peer provocation. Journal of Clinical Child and Adolescent Psychology, 36, 621–628.

    Article  PubMed  Google Scholar 

  • Dodge, K. A., McClaskey, C., & Feldman, E. (1985). Situational approach to the assessment of social competence in children. Journal of Consulting and Clinical Psychology, 53, 344–353.

    Article  PubMed  Google Scholar 

  • Dodge, K. A., Dishion, T. J., & Lansford, J. E. (2006). Deviant peer influences in programs for youth. New York: Guilford.

    Google Scholar 

  • Drabick, D. A. G., Gadow, K. D., & Loney, J. (2008). Co-occurring ODD and GAD symptom groups: source-specific syndromes and cross-informant comorbidity. Journal of Clinical Child and Adolescent Psychology, 37, 314–326.

    Article  PubMed  Google Scholar 

  • Eddy, J. M., Dishion, T. J., & Stoolmiller, M. (1998). The analysis of intervention change in children and families: methodological and conceptual issues embedded in intervention studies. Journal of Abnormal Child Psychology, 26, 53–69.

    Article  PubMed  Google Scholar 

  • Farrell, A. D., Ampy, L. A., & Meyer, A. L. (1998). Identification and assessment of problematic interpersonal situations for urban adolescents. Journal of Clinical Child Psychology, 27, 293–305.

    Article  PubMed  Google Scholar 

  • Fournier, M. A., Moskowitz, D. S., & Zuroff, D. C. (2008). Integrating dispositions, signatures, and the interpersonal domain. Journal of Personality and Social Psychology, 94, 531–545.

    Article  PubMed  Google Scholar 

  • Funder, D. C. (2009). Persons, behaviors and situations: an agenda for personality psychology in the postwar era. Journal of Research in Personality, 43, 120–126.

    Article  Google Scholar 

  • Furr, R. M. (2009). Personality psychology as a truly behavioural science. European Journal of Personality, 23, 369–401.

    Article  Google Scholar 

  • Granic, I., & Patterson, G. R. (2006). Toward a comprehensive model of antisocial development: a dynamic systems approach. Psychological Review, 113, 101–131.

    Article  PubMed  Google Scholar 

  • Gresham, F. M., & Elliot, S. N. (1990). Social skills rating system manual. Circle Pines: American Guidance Service.

    Google Scholar 

  • Gresham, F. M., Cook, C. R., Collins, T., Rasethwane, K., Dart, E., Truelson, E., et al. (2010). Developing a change-sensitive brief behavior rating scale as a progress monitoring tool for social behavior: an example using the Social Skills Rating System-Teacher Form. School Psychology Review, 39, 364–379.

    Google Scholar 

  • Hartley, A. G., Zakriski, A. L., & Wright, J. C. (2011). Probing the depths of informant discrepancies: contextual influences on divergence and convergence. Journal of Clinical Child and Adolescent Psychology, 40, 54–66.

    Article  PubMed  Google Scholar 

  • Haynes, S. N., Mumma, G. H., & Pinson, C. (2009). Idiographic assessment: conceptual and psychometric foundations of individualized behavioral assessment. Clinical Psychology Review, 29, 179–191.

    Article  PubMed  Google Scholar 

  • Hershberger, S. L., Plomin, R., & Pedersen, N. L. (1995). Traits and metatraits: their reliability, stability, and shared genetic influence. Journal of Personality and Social Psychology, 69, 673–685.

    Article  PubMed  Google Scholar 

  • Hoffenaar, P. J., & Hoeksema, J. B. (2002). The structure of oppositionality: response dispositions and situational aspects. Journal of Psychology and Psychiatry and Allied Health Disciplines, 43, 375–385.

    Article  Google Scholar 

  • Kagan, J. (2001). The need for new constructs. Psychological Inquiry, 12, 84–103.

    Article  Google Scholar 

  • Kazdin, A. E. (2006). Arbitrary metrics: implications for identifying evidence-based treatments. The American Psychologist, 61, 42–49.

    Article  PubMed  Google Scholar 

  • Kempes, M., Matthys, W., de Vries, H., & van Engeland, H. (2010). Children’s aggressive responses to neutral peer behavior: a form of unprovoked reactive aggression. Psychiatry Research, 176, 219–223.

    Article  PubMed  Google Scholar 

  • Ladd, G. W., & Profilet, S. M. (1996). The Child Behavior Scale: a teacher-report measure of young children’s aggressive, withdrawn, and prosocial behaviors. Developmental Psychology, 32, 1008–1024.

    Article  Google Scholar 

  • Lewis, M. (2001). Issues in the study of personality development. Psychological Inquiry, 12, 67–83.

    Article  Google Scholar 

  • Lutz, M. N., Fantuzzo, J., & McDermott, P. (2002). Multidimensional assessment of emotional and behavioral adjustment problems of low-income preschool children: development and initial validation. Early Childhood Research Quarterly, 17, 338–355.

    Article  Google Scholar 

  • Mash, E. J., & Hunsley, J. (2005). Evidence-based assessment of child and adolescent disorders: issues and challenges. Journal of Clinical Child and Adolescent Psychology, 34, 362–379.

    Article  PubMed  Google Scholar 

  • Matthys, W., Maassen, G. H., Cuperus, J. M., & Van Engeland, H. (2001). The assessment of the situational specificity of children’s problem behaviour in peer-peer context. Journal of Child Psychology and Psychiatry, 42, 413–420.

    Article  PubMed  Google Scholar 

  • Maxwell, S. E., & Delaney, H. D. (1990). Designing experiments and analyzing data. Belmont: Wadsworth.

    Google Scholar 

  • McCrae, R. R., & Costa, P. T., Jr. (1999). A Five-Factor theory of personality. In L. A. Pervin & O. P. John (Eds.), Handbook of personality: Theory and research (2nd ed., pp. 139–153). New York: Guilford.

    Google Scholar 

  • McDermott, P. A. (1993). National standardization of uniform multisituational measures of child and adolescent behavior pathology. Psychological Assessment, 5, 413–424.

    Article  Google Scholar 

  • Measelle, J. R., John, O. P., Ablow, J. C., Cowan, P. A., & Cowan, C. P. (2005). Can children provide coherent, stable, and valid self-reports on the Big Five dimensions? A longitudinal study from ages 5 to 7. Journal of Personality and Social Psychology, 89, 90–106.

    Article  PubMed  Google Scholar 

  • Mischel, W. (2009). From personality and assessment (1968) to personality science, 2009. Journal of Research in Personality, 43, 282–290.

    Article  Google Scholar 

  • Mischel, W., & Shoda, Y. (1995). A cognitive-affective system theory of personality: reconceptualizing situations, dispositions, dynamics, and invariance in personality structure. Psychological Review, 102, 246–268.

    Article  PubMed  Google Scholar 

  • National Institute of Child Health and Human Development Early Child Care Research Network. (2003). Does amount of time spent in child care predict socioemotional adjustment during the transition to kindergarten? Child Development, 74, 976–1005.

    Article  Google Scholar 

  • Nock, M. K., & Kurtz, S. M. S. (2005). Direct behavioral observation in school settings: bringing science to practice. Cognitive and Behavioral Practice, 12, 359–370.

    Article  Google Scholar 

  • Noftle, E. E., & Fleeson, W. (2010). Age differences in big five behavior averages and variabilities across the adult life span: moving beyond retrospective, global summary accounts of personality. Psychology and Aging, 25, 95–107.

    Article  PubMed  Google Scholar 

  • Reynolds, C. R., & Kamphaus, R. W. (2002). Behavior assessment system for children (2nd ed.). Circle Pines: American Guidance Service.

    Google Scholar 

  • Roberts, B. W., & Caspi, A. (2001). Personality development and the person-situation debate: it’s déjà vu all over again. Psychological Inquiry, 12, 104–109.

    Article  Google Scholar 

  • Scotti, J. R., Morris, T. L., McNeil, C. B., & Hawkins, R. P. (1996). DSM-IV and disorders of childhood and adolescence: can structural criteria be functional? Journal of Consulting and Clinical Psychology, 64, 1177–1191.

    Article  PubMed  Google Scholar 

  • Shoda, Y., Mischel, W., & Wright, J. C. (1994). Intraindividual stability in the organization and patterning of behavior: incorporating psychological situations into the idiographic analysis of personality. Journal of Personality and Social Psychology, 67, 674–687.

    Article  PubMed  Google Scholar 

  • Smith, R. E., Shoda, Y., Cumming, S. P., & Smoll, F. L. (2009). Behavioral signatures at the ballpark: intraindividual consistency of adults’ situation-behavior patterns and their interpersonal consequences. Journal of Research in Personality, 43, 187–195.

    Article  Google Scholar 

  • Tremblay, R. E. (2000). The development of aggressive behavior during childhood: what have we learned in the past century? International Journal of Behavioral Development, 24, 129–141.

    Article  Google Scholar 

  • Van Mechelen, I. (2009). A royal road to understanding the mechanisms underlying person-in-context behavior. Journal of Research in Personality, 43, 179–186.

    Article  Google Scholar 

  • Van Mechelen, I., & Kiers, H. A. L. (1999). Individual differences in anxiety responses to stressful situations: a three-mode component analysis model. European Journal of Personality, 13, 409–428.

    Article  Google Scholar 

  • Vansteelandt, K., & Van Mechelen, I. (1998). Individual differences in situation-behavior profiles: a triple-typology model. Journal of Personality and Social Psychology, 75, 751–765.

    Article  Google Scholar 

  • Volpe, R. J., DiPerna, J. C., Hintze, J. M., & Shapiro, E. S. (2005). Observing students in classroom settings: a review of seven coding schemes. School Psychology Review, 34, 454–474.

    Google Scholar 

  • Walker, H. M., & McConnell, S. R. (1995). Walker-McConnell scale of social competence and school adjustment, adolescent version. NY: Wadsworth.

    Google Scholar 

  • Weisz, J. R., Doss, A. J., & Hawley, K. M. (2005). Youth psychotherapy outcome research: a review and critique of the evidence base. Annual Review of Psychology, 56, 337–363.

    Article  PubMed  Google Scholar 

  • Wright, J. C., & Mischel, W. (1987). A conditional approach to dispositional constructs: the local predictability of social behavior. Journal of Personality and Social Psychology, 53, 1159–1177.

    Article  PubMed  Google Scholar 

  • Wright, J. C., & Zakriski, A. L. (2001). A contextual analysis of externalizing and mixed syndrome boys: when syndromal similarity obscures functional dissimilarity. Journal of Consulting and Clinical Psychology, 69, 457–470.

    Article  PubMed  Google Scholar 

  • Wright, J. C., Zakriski, A. L., & Drinkwater, M. (1999). Developmental psychopathology and the reciprocal patterning of behavior and environment: distinctive situational and behavioral signatures of internalizing, externalizing, and mixed-syndrome children. Journal of Consulting and Clinical Psychology, 67, 95–107.

    Article  PubMed  Google Scholar 

  • Wright, J. C., Lindgren, K. P., & Zakriski, A. L. (2001). Syndromal versus contextualized personality assessment: differentiating environmental and dispositional determinants of boys’ aggression. Journal of Personality and Social Psychology, 81, 1176–1189.

    Article  PubMed  Google Scholar 

  • Zakriski, A. L., Wright, J. C., & Underwood, M. K. (2005). Gender similarities and differences in children’s social behavior: finding personality in contextualized patterns of adaptation. Journal of Personality and Social Psychology, 88, 844–855.

    Article  PubMed  Google Scholar 

  • Zakriski, A. L., Wright, J. C., & Cardoos, S. (2011). Evaluating deviancy training within residential treatment: Individual and group effects of peer-nominated deviant talk. Manuscript under review.

Download references

Author information

Authors and Affiliations

  1. Walter S. Hunter Laboratory of Psychology, Brown University, 89 Waterman Street, Providence, RI, 02912, USA

    Jack C. Wright

  2. Department of Psychology, Connecticut College, 270 Mohegan Avenue, New London, CT, 06320, USA

    Audrey L. Zakriski

  3. Walter S. Hunter Laboratory of Psychology, Brown University, 89 Waterman Street, Providence, RI, 02912, USA

    Anselma G. Hartley

  4. Wediko Children’s Services, 72-74 East Dedham St., Boston, MA, 02118, USA

    Harry W. Parad

Authors
  1. Jack C. Wright
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Audrey L. Zakriski
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anselma G. Hartley
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Harry W. Parad
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jack C. Wright.

Additional information

We are deeply grateful to the children, parents, staff, and administrators of Wediko Children’s Services, whose cooperation made it possible to collect the data reported here. We would also like to thank the research coordinators and assistants for their dedication to the project. The first and second authors made equal contributions to this work; the order of their authorship was randomly determined. This research was partially supported by award number R15MH076787 from the National Institute of Mental Health. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Mental Health or the National Institutes of Health.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wright, J.C., Zakriski, A.L., Hartley, A.G. et al. Reassessing the Assessment of Change in At-Risk Youth: Conflict and Coherence in Overall Versus Contextual Assessments of Behavior. J Psychopathol Behav Assess 33, 215–227 (2011). https://doi.org/10.1007/s10862-011-9233-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10862-011-9233-x

Keywords

Search

Navigation