Skip to main content

Advertisement

SpringerLink
Log in

Diversity and Social Justice Implications for Outcome Approaches to Evaluation

  • Original Article
  • Published:
International Journal for the Advancement of Counselling Aims and scope Submit manuscript

Abstract

It is important that counsellors provide evidence regarding the efficacy of their services. Although there has been an increased focus on generic outcome measurement of counselling programs and services, little attention has been paid to accountability issues for meeting the needs of diverse populations. This article highlights the increasing relevance of cultural diversity for evaluation practices, using examples from the Canadian context. An overview is provided of the key issues for evaluation of programs and services involving culturally diverse individuals and how evaluation must incorporate social justice issues within service provision. Evaluation strategies are elaborated along with suggestions for enhancing the intercultural competence of evaluation personnel.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Arthur, N., & Collins, S. (2005). Culture-infused counselling: Celebrating the Canadian mosaic pp. 3–40. Calgary, AB: Counselling Concepts.

    Google Scholar 

  • Arthur, N., & Merali, N. (2005). Counselling immigrants and refugees. In N. Arthur, & S. Collins (Eds.), Culture-infused counselling: Celebrating the Canadian Mosaic (pp. 331–360). Calgary, AB: Counselling Concepts.

    Google Scholar 

  • Anderson-Draper, M. H. (2006). Understanding cultural competence through the evaluation of breaking the silence: A project to generate critical knowledge about family violence within immigrant communities. The Canadian Journal of Program Evaluation, 21(2), 59–79.

    Google Scholar 

  • Arthur, N., & Pedersen, P. (2008). Case incidents in counseling for international transitions. Alexandria, VA: American Counseling Association.

    Google Scholar 

  • Arredondo, P., Toropek, R., Brown, S. P., Jones, J., Locke, D. C., Sanchez, J., et al. (1996). Operationalization of the multicultural counseling competencies. Journal of Multicultural Counseling and Development, 24(1), 42–78.

    Google Scholar 

  • Bell, L. A. (1997). Theoretical foundations for social justice education. In M. Adams, L. A. Bell, & P. Griffin (Eds.), Teaching for diversity and social justice: A sourcebook (pp. 3–15). New York: Routledge.

    Google Scholar 

  • Bozzo, S. L. (2002). Evaluation capacity building in the voluntary/nonprofit sector. The Canadian Journal of Program Evaluation, 17(3), 75–92.

    MathSciNet  Google Scholar 

  • Baker, E. A., & Teaser-Polk, C. (1998). Measuring community capacity: where do we go from here? Health Education & Behavior, 25(3), 279–283. doi:10.1177/109019819802500304.

    Article  CAS  Google Scholar 

  • Bernal, G., Bonillo, J., & Bellido, C. (1995). Ecological validity and cultural sensitivity of outcome research: Issues for the cultural adaptation and development of psychosocial treatments with Hispanics. Journal of Abnormal Child Psychology, 67(16), 329–342.

    Google Scholar 

  • Beiser, M., Wood, M., Barwick, C., Berry, J. W., deCosta, G., Milne, W., et al. (1988). After the door has been opened: Mental health issues affecting immigrants and refugees in Canada. Ottawa, ON: Ministries of Multiculturalism and Citizenship,and Health and Welfare.

    Google Scholar 

  • Cross, T. L. (1988). Cultural competence continuum. Focal Point, 1–3. Retrieved November 2, 2005 from http://www.nysccc.org/T-Rarts/CultCompCont.html.

  • Constantine, M. (2002). Predictors of satisfaction with counseling: Racial and ethnic minority clients’ attitudes towards counseling and ratings of their counselors’ general and multicultural counseling competence. Journal of Counseling Psychology, 49(2), 255–264. doi:10.1037/0022-0167.49.2.255.

    Article  Google Scholar 

  • Cahill, M., & Martland, S. (1996). Soft systems: Non-hierarchical evaluation of career counselling programs. Guidance and Counselling, 96(11), 15–19.

    Google Scholar 

  • Clyne, G., & Edwards, R. (2002). Understanding economic evaluations: A guide for health and human services. The Canadian Journal of Program Evaluation, 17(3), 1–23.

    Google Scholar 

  • Christie, A., & Barela, E. (2005). The Delphi technique as a method for increasing inclusion in the evaluation process. The Canadian Journal of Program Evaluation, 20(1), 105–122.

    Google Scholar 

  • Cannon, L. W., Higginbotham, E., & Leung, M. L. (1988). Race and class bias in qualitative research on women. Gender & Society, 2(4), 449–462. doi:10.1177/089124388002004003.

    Article  Google Scholar 

  • Carrick, R., Mitchell, A., & Lloyd, K. (2001). User involvement in research: Power and compromise. Journal of Community & Applied Social Psychology, 11, 217–225. doi:10.1002/casp.623.

    Article  Google Scholar 

  • Clancy, C., Stryer, D., & Eisenberg, J. M. (2002). From publication to public action: Agency for healthcare research and quality (AHRQ) perspectives on ethnicity and race-related outcomes research. Ethnicity & Health, 7(4), 287–290. doi:10.1080/1355785022000060745.

    Article  Google Scholar 

  • Collins, S., & Arthur, N. (in press a). Culture-infused counselling: A fresh look at a classic framework of multicultural counselling competencies. Counseling Psychology Quarterly.

  • Collins, S., & Arthur, N. (in press b). Culture-infused counseling: A model for developing cultural competence. Counseling Psychology Quarterly.

  • Collins, S., Arthur, N., & Wong-Wiley, G. (in press) Enhancing reflective practice in multicultural counseling through cultural auditing. Journal of Counseling and Development.

  • Constantine, M., Hage, S. M., Kindaichi, M., & Bryant, R. M. (2007). Social justice and multicultural issues: Implications for the practice and training of counselors and counseling psychologists. Journal of Counseling and Development, 85, 24–29.

    Google Scholar 

  • Dickson, G., & Green, K. L. (2001). Participatory action research: Lessons learned with aboriginal grandmothers. Health Care for Women International, 22, 471–482. doi:10.1080/073993301317094290.

    Article  PubMed  CAS  Google Scholar 

  • Darou, W. G., Kurtness, J., & Hum, A. (2000). The impact of conducting research with a First Nation. Canadian Journal of Counselling, 34(1), 43–54.

    Google Scholar 

  • Esses, V. M., & Gardner, R. C. (1996). Multiculturalism in Canada: context and current status. Canadian Journal of Behavioural Science, 28(3), 145. doi:10.1037/h0084934.

    Google Scholar 

  • Greene, B. (2005). Psychology, diversity and social justice: Beyond heterosexism and across the cultural divide. Counselling Psychology Quarterly, 18, 295–306. doi:10.1080/09515070500385770.

    Article  Google Scholar 

  • Grieger, K., & Ponterotto, J. G. (1995). A framework for assessment in multicultural counseling. In J. G. Ponterotto, J. M. Casas, L. A. Suzuki, & C. M. Alexander (Eds.), Handbook of multicultural counselling (pp. 357–374). Thousand Oaks, CA: Sage.

    Google Scholar 

  • Guillermo, B., & Scharron-Del-Rio, M. R. (2001). Are empirically supported treatments valid for ethnic minorities? Toward an alternative approach for treatment research. Cultural Diversity & Ethnic Minority Psychology, 7(4), 328–342. doi:10.1037/1099-9809.7.4.328.

    Article  Google Scholar 

  • Glasgow, R. E., Lichtenstein, E., & Marcus, A. C. (2003). Why don’t we see more translation of health promotion research to practice? Rethinking the efficacy-to-effectiveness transition. American Journal of Public Health, 93(8), 1261–1267.

    Article  PubMed  Google Scholar 

  • Helms, J. E. (1994). How multiculturalism obscures racial factors in the therapy process: Comment on Ridley et al. (1994), Sodowsky et al. (1994), Ottavi et al. (1994), and Thompson et al. (1994). Journal of Counseling Psychology, 41, 162–165. doi:10.1037/0022-0167.41.2.162.

    Article  Google Scholar 

  • House, E. R. (2004). The role of the evaluator in a political world. The Canadian Journal of Program Evaluation, 19(2), 1–16.

    Google Scholar 

  • Ibrahim, F. A. (1991). Contribution of cultural worldview to generic counselling and development. Special Issue: Multiculturalism as a fourth force in counselling. Journal of Counseling and Development, 70(1), 13–19.

    Google Scholar 

  • Lalande, V. (2005). One size doesn’t fit all: Evaluating outcomes of career services for diverse clients. Building tomorrow today consultation proceedings. Edmonton, AB: Career Development Association of Alberta.

    Google Scholar 

  • Lalande, V., & Huston, M. (2004). Developing an accountability framework for career development practices. NATCON Papers. Retrieved November 11, 2005 from http://www.natcon.org/natcon/nav_e.cfm?s=main&p=natcon_papers.

  • Lalande, V., & Laverty, A. (2005). Counselling girls and women. In N. Arthur, & S. Collins (Eds.), Culture-infused counselling: Celebrating the Canadian mosaic (pp. 387–414). Calgary, AB: Counselling Concepts.

    Google Scholar 

  • Larsen, K. & Istance, D.(2001). Lifelong learning for all. OECD observer, March. Centre for Educational Research and Innovation: OECD.

  • Mason, G. (1991). Longitudinal research in program evaluation. In A. J. Love (Ed.), Evaluation methods sourcebook. Ottawa, ON: Canadian Evaluation Society.

    Google Scholar 

  • McNamara, C. (1998). Basic guide to program evaluation. Retrieved March 10, 2004 from http://www.mapnp.org/library/evaluation/fnl_eval.htm#anchor1585345.

  • Meadows, L. M., Lagendyk, L. E., Thurston, W. E., & Eisener, A. C. (2003). Balancing culture, ethics, and methods in qualitative health research with aboriginal peoples. International Journal of Qualitative Methods, 2(4), 1–25.

    Google Scholar 

  • Merali, N. (1998). Resolution of value conflicts in multicultural counselling. Canadian Journal of Counselling, 33(1), 57–66.

    Google Scholar 

  • McKenzie, B. (2001). Perspectives on the changing policy/practice environment in child welfare. Report presented to the Tri-Provincial Child and Family Services Symposium. Saskatoon: Saskatchewan. November.

  • McMahon, M., & Patton, W. (2002). Using qualitative assessment in career counselling. International Journal for Educational and Vocational Guidance, 2, 51–66. doi:10.1023/A:1014283407496.

    Article  Google Scholar 

  • MacLellan-Wright, M. F., Patten, S., dela Cruz, A. M., & Flaherty, A. (2007). A participatory approach to the development of an evaluation framework; Process, pitfalls, and payoffs. The Canadian Journal of Program Evaluation, 22(1), 99–124.

    Google Scholar 

  • Nelson, T. D., & Steele, R. (2006). Beyond Efficacy and Effectiveness: A Multifaceted Approach to Treatment Evaluation. Professional Psychology, Research and Practice, 37(4), 389–397. doi:10.1037/0735-7028.37.4.389.

    Article  Google Scholar 

  • Neufeld, A., Harrison, M. J., Stewart, M. J., Hughes, K. D., & Spitzer, D. L. (2001). Immigrant women’s experience as family caregivers: Support and barriers. Edmonton, AB: Prairie Centre of Excellence for Research on Immigration and Integration.

    Google Scholar 

  • Obrecht, M. (1999). Distilling stakeholder input for program evaluation priority setting. The Canadian Journal of Program Evaluation, 14(2), 119–134.

    Google Scholar 

  • Organisation for Economic Co-operation and Development. (2002). Glossary of key terms in evaluation and results based management. Retrieved October 26, 2005 from http://www.oecd.org/dataoecd/43/54/35336188.pdf.

  • O’Neill, P. (2005). The ethics of problem definition. Canadian Psychology, 46(1), 13–20. doi:10.1037/h0085819.

    MathSciNet  Google Scholar 

  • O’Hanlon, W. H., & Weiner-Davis, M. (1989). In search of solutions: A new direction in psychotherapy. New York: Norton.

    Google Scholar 

  • Pandiani, J. A., Banks, S. M., Barmley, J., Pomeroy, S., & Simon, M. (2002). Measuring access to mental health care: A multi-indicator approach to program evaluation. Evaluation and Program Planning, 25(3), 271–285.

    Article  Google Scholar 

  • Ponterotto, J. G. (1988). Racial/ethnic minority research in the Journal of Counselling Psychology: A content analysis and methodological critique. Journal of Counseling Psychology, 35(4), 410–418. doi:10.1037/0022-0167.35.4.410.

    Article  Google Scholar 

  • Patton, M. Q. (1990). Qualitative evaluation and research methods. Newbury Park, CA: Sage.

    Google Scholar 

  • Pedersen, P. (1991). Multiculturalism as a generic approach to counseling. Journal of Counseling and Development, 70, 6–12.

    Google Scholar 

  • Pettifor, J. (1994). Ethics and social justice in program evaluation: Are evaluators value-free? Canadian Journal of School Psychology, 10(2), 138–146.

    Article  Google Scholar 

  • Patton, M. Q. (1999). Organizational development and evaluation. The Canadian Journal of Program Evaluation, 93–113.

  • Prairie Centre of Excellence for Research on Immigration and Integration. (1999). The settlement experiences of refugees in Alberta. Edmonton, AB: Author.

  • Pedersen, P. (2001). Multiculturalism and the paradigm shift in counselling: Controversies and alternative futures. Canadian Journal of Counselling, 35, 15–25.

    Google Scholar 

  • Plant, P. (2001). Quality in careers guidance. Paper prepared for the OECD Career Guidance Policy Review. Retrieved January 4, 2004 from www.oecd.org/edu/careerguidance.

  • Patton, M. Q. (2002). Qualitative research and evaluation methods. Newbury Park, CA: Sage.

    Google Scholar 

  • Park, S., & Lahman, M. K. (2003). Bridging perspectives of parents, teachers and co-researchers: methodological reflections on cross-cultural research. Reflective Practice, 4(3), 375–383. doi:10.1080/1462394032000112264.

    Article  Google Scholar 

  • Ponterotto, J. G., & Grieger, I. (2008). Guidelines and competencies for cross-cultural counseling research. In P. Pedersen, J. G. Draguns, W. J. Lonner, & J. E. Trimble (Eds.), Counseling across cultures (pp. 57–72, 6th ed.). Thousand Oaks, CA: Sage.

    Google Scholar 

  • Pandiani, J. A., Banks, S. M., Barmley, J., Pomeroy, S., & Simon, M. (2002). Measuring access to mental health care: A multi-indicator approach to program evaluation. Evaluation and Program Planning, 25(3), 271–285. doi:doi:10.1016/S0149-7189(02)00021-6.

    Article  Google Scholar 

  • Quintana, S. M., Troyano, N., & Taylor, G. (2001). Cultural validity and inherent challenges in quantitative methods for multicultural research. In J. Ponterotto, J.M. Casas, L. Suzuki, & C.M. Alexander (Eds.), Handbook of multicultural counselling (pp. 604–630). Thousand Oaks, CA: Sage.

    Google Scholar 

  • Reid, C. (2004). Advancing women’s social justice agendas: A feminist action research framework. International Journal of Qualitative Methods, 3(3). Article 1. Retreived February 16, 2005 from http://www.ualberta.ca/`iiqm/backissues/3_3/html/reid.html.

  • Statistics Canada. (1996). 1996 Census. Retrieved October 1, 2003, from http://www12.statcan.ca/english/census01/info/census96.cfm.

  • Sue, S. (1999). Science, ethnicity, and bias: Where have we gone wrong? The American Psychologist, 54(12), 1070–1077. doi:10.1037/0003-066X.54.12.1070.

    Article  PubMed  CAS  Google Scholar 

  • Statistics Canada.(2004a). Aboriginal people of Canada. Retrieved July 6, 2004, from http://www12.statcan.ca/english/census01/products/analytic/companion/abor/canada.cfm.

  • Statistics Canada.(2004b). Visible minority populations, 2001 Census Canada. Retrieved March 24, 2004, from http://www.cic.gc.ca.

  • Stewart, J. (2005). Assessment and multicultural counselling. In N. Arthur, & S. Collins (Eds.), Culture-infused counselling: Celebrating the Canadian mosaic (pp. 239–262). Calgary, AB: Counselling Concepts.

    Google Scholar 

  • Sue, D. W., & Sue, D. (1990). Counseling the culturally different: Theory and practice. New York, NY: Wiley.

    Google Scholar 

  • Spath, R., & Pine, B. A. (2004). Using the case study approach for improved programme evaluations. Child & Family Social Work, 9, 57–63. doi:10.1111/j.1365-2206.2004.00310.x.

    Article  Google Scholar 

  • Sue, D. W., Carter, R. T., Casa, J. M., Fouad, N. A., Ivey, A. E., Jensen, M., et al. (1998). Multicultural counselling competencies: Individual and organizational development. Thousand Oaks, CA: Sage.

    Google Scholar 

  • Tse, C. (2003). A baseline assessment and program evaluation of the diversity initiative services at a healthcare organization. Dissertation Abstracts International: Section B: The Sciences & Engineering. 64(5-B), 2003, 2429. US: Univ Microfilms International.

  • Vingilis, E., & Pederson, L. (2001). Using the right tools to answer the right questions: The importance of evaluative research techniques for heath services evaluation research in the 21st century. The Canadian Journal of Program Evaluation, 16(2), 1–26.

    Google Scholar 

  • Vera, E. M., & Speight, S. L. (2003). Multicultural competence, social justice, and counseling psychology: Expanding our roles. The Counseling Psychologist, 31, 253–272. doi:10.1177/0011000003031003001.

    Article  Google Scholar 

  • Westen, D., Novotny, C. M., & Thompson-Brenner, H. (2004). The empirical status of empirically supported psychotherapies: Assumptions, findings, and reporting in controlled clinical trials. Psychological Bulletin, 130(4), 631–663. doi:10.1037/0033-2909.130.4.631.

    Article  PubMed  Google Scholar 

  • Woodcock, R. (2005). Cognitive-behavioral theraphy and ethnic minority groups: A review of outcome research. Unpublished master’s project, Campus Alberta Applied Psychology, Calgary, Alberta, Canada.

  • Wong, D. F. K., Sun, S. Y. K., Tse, J., & Wong, F. (2002). Evaluating the outcomes of a cognitive-behavioural group intervention model for persons at risk of developing mental health problems in Hong Kong: A pretest-postest study. Research on Social Work Practice, 12(4), 534–545. doi:10.1177/1049731502012004005.

    Article  Google Scholar 

  • Young, I. M. (1990). Justice and the politics of difference. Princeton, NJ: Princeton University Press.

    Google Scholar 

  • Zunker, V. G. (2002). Career counseling. Applied concepts of life planning (6th ed.). Pacific Grove, CA: Brooks/Cole.

    Google Scholar 

Download references

Acknowledgments

The development of this article was partially funded by a Standard Research Grant awarded to the first authors from the Social Sciences and Humanities Council of Canada.

Author information

Authors and Affiliations

  1. Division of Applied Psychology, Faculty of Education, University of Calgary, 2500 University Dr. NW, Calgary, Alberta, Canada, T2N 1N4

    Nancy Arthur & Vivian Lalande

Authors
  1. Nancy Arthur
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vivian Lalande
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nancy Arthur.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Arthur, N., Lalande, V. Diversity and Social Justice Implications for Outcome Approaches to Evaluation. Int J Adv Counselling 31, 1–16 (2009). https://doi.org/10.1007/s10447-008-9063-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10447-008-9063-z

Keywords

Search

Navigation