Keywords

Introduction

The evaluation of the Culturally Competent Leadership Program (CCLP) had three distinct aims:

  1. 1.

    To appraise the achievements of CCLP objectives;

  2. 2.

    To assess the soundness of the CCLP approach and component strategies, and analyse their respective performance in relation to achieving targeted objectives and sustainability of results achieved; and,

  3. 3.

    To analyse constraints, lessons learned, the use of evidence-based best practices and evaluate the strategic opportunity to build upon achievements to inform further CCLPs.

The discussion in this chapter begins with a background context to the CCLP, including a synthesis of the reviewed literature on culturally competent leadership, and the rationale for cultural competence in the Australian higher education sector. The chapter then outlines the program’s goals, objectives and intended outputs, as well as an overview of its structure and format. The remainder of the chapter focuses on the evaluation of the CCLP, which covers the methodology employed and the evaluation implementation undertaken.

Program Context and Background

The University of Sydney’s Strategic Plan 2020 emphasises the development of cultural competence for all staff and builds leadership quality in this area. It states that the university is ‘committed to a series of actions to… develop (staff) capacity as agents of cultural change’ (p. 44), and that ‘staff have a particular responsibility to… demonstrate leadership in this area’ (p. 43). The Strategic Plan (p. 13) states specifically that academic staff should:

participate effectively in intercultural settings in research, in the classroom, and in the day-to-day life of the University. They should be open to a diversity of ways of being, doing and knowing, as well as looking for, and understanding, the context of those engaged in, or affected by, our research and education.

The Strategic Plan also embeds cultural competence as a graduate quality in all undergraduate degrees and commits to collaborating with the NCCC on skills development ‘through a shared commitment to a more collective, relational model for learning and teaching, [and to] embed the development of cultural competence in the curriculum’ (p. 36).

The University of Sydney has also acknowledged ‘the inherent rights of Aboriginal and Torres Strait Islander peoples to be self-determined and respected as Australia’s first peoples’ (Sherwood & Russell-Mundine, 2017, p. 134) through the Wingara Mura–Bunga Barrabugu (‘Wingara Mura’), the University’s Aboriginal and Torres Strait Islander Integrated Strategy. Wingara Mura establishes as a key student and staff capability and the ability to engage effectively, respectfully and productively in critical thinking and self-reflection regarding Aboriginal and Torres Strait Islander issues specifically, and diversity more broadly. As Sherwood and Russell-Mundine (2017, p. 134) note:

Wingara Mura places the promotion of Aboriginal and Torres Strait Islander participation, engagement, education and research as a core objective of the University. Within this policy context, the NCCC aims to provide the essential framework to embed the cultural competence qualities necessary to implement the strategy across the organisation, its staff and students.

The NCCC follows and aligns its cultural competence agenda with the recommendations of Universities Australia (2011). These recommendations position universities as agents of change, not only in improving higher education outcomes, including access and participation, for Indigenous Australians within the universities, but also by making ‘a commitment to the capacity building of Indigenous communities’ (p. 17). The NCCC locates its understanding of cultural competence very much in the context of addressing social justice issues, such as equity, access and participation in higher education, as the foundations to creating change.

Cultural competence can lay the foundations for a socially-just consciousness and culturally diverse, respectful worldviews, grown through the development and processing of core values, knowledge, behaviours and actions. Its principles and praxis have germinated through the work of Indigenous and culturally diverse scholars, health professionals, peoples and groups whose work has been based upon a human rights agenda focused on equity and justice (Cross, Bazron, Dennis, & Isaacs, 1989; Ranzin, McConnochie & Nolan, 2009; Sherwood et al., 2011; Universities Australia, 2011). The embedding of cultural competence requires deep, transformational change in behaviour, teaching and learning, a deeper institution-wide commitment to the values espoused by the University, and an investment in developing a more understanding and respectful university culture and organisational relationships. It also requires a depth of knowledge and strong, informed leadership in the field of cultural competence to support the transformational change the University wants to achieve.

The literature presents culturally competent leadership as an underlying set of attributes, skills and behaviours, as well as knowledge, that run across, and are foundational to the wide range of designations featuring in the scholarship. Culturally competent leadership is transformative/transformational, values-driven, moral, critically self-reflective (Terrell & Lindsey, 2008; Beachum, 2011) and purposeful, in its intent and scope. Leadership in this space equates to courage, advocacy for social justice and human rights, and a willingness and ability for reflexive practice and self-awareness. Culturally competent leaders in higher education settings display characteristics of ‘civility, ethical behaviour, data-driven decision-making and cultural sensitivity’ (Thompson et al., 2017, p. 79). It is the type of leadership that is required if the academy is to create more inclusive, culturally competent university communities. Leadership in this sense is seen to drive and enable transformation and change, and the notion of agency, at the institutional, organisational, as well as at the individual level, underpins culturally competent leadership. Leaders in this space work to identify, transform and question systems that generate inequity, disparity and social injustice (Horsford et al., 2011; Su & Wood, 2017; Shultz & Viczko, 2016). The moral imperative that accompanies culturally competent leadership also underpins theories of transformative leadership (Shields, 2010; Marbley et al., 2015) and foregrounds the concepts of agency (Komives & Wagner, 2016) and transformative action in leadership discourse and practice. Educational leaders in this space need to commit to critical conversations around the historical, social and material legacies of colonial practices, if they wish to enact systemic change and transformation. Shultz and Viczko (2016, p. 2) note, for example, that even higher education institutions ‘have not escaped this [colonial] legacy, and the durability of issues and intersections of race, gender, and class violence are evident in our organisations’. On the criticality of the systems that perpetuate longstanding injustices, López (2016, p. 20) asserts that ‘educational leaders who ground their work in critical perspectives seek to create social change by challenging the status quo and systems of power that dominate and subjugate'. Transformative, culturally competent leadership hence focuses on systemic change and critically illuminating, through the unpacking of systems, inadvertently held power and privilege, disparity, inequity and injustice.

In Australia, cultural competence in higher education cannot be separated from social justice, human rights, equity, equal opportunity and reconciliation discourse as it relates to Australia’s First Nations Peoples, and hence Aboriginal and Torres Strait Islander students and staff. To this end, within the Australian higher education sector, cultural competence has been defined as:

Student and staff knowledge and understanding of Indigenous Australian cultures, histories and contemporary realities and awareness of Indigenous protocols, combined with the proficiency to engage and work effectively in Indigenous contexts congruent to the expectations of Indigenous Australian peoples … [and] the ability to critically reflect on one’s own culture and professional paradigms in order to understand its cultural limitations and effect positive change. (Universities Australia, 2011, p. 3)

Developing cultural capabilities within the higher education sector hence requires leaders, in traditional leadership roles as well as individual advocates and champions who may not necessarily hold senior leadership titles, to guide a whole-of-institution approach involving the systemic, organisational, professional and individual realms (Miralles & Migliorino, 2005). This ‘includes examining individual attitudes and practice in teaching as well as management, executive, policy and strategic commitment to revise and assess capacity to implement culturally competent teaching, learning, academic, research and employment spaces’ (Taylor, Durey, Mulcock, Kickett, & Jones, 2014, p. 37).

Universities Australia’s 2011 National Best Practice Framework for Indigenous Cultural Competency in Australian Universities has been instrumental in providing guidance and direction on good and best practice strategies, approaches and methodologies to embed cultural competence across higher education institutions. Progress with embracing cultural competence in higher education in Australia has been incremental and uneven. The findings of the Review of Higher Education Access and Outcomes for Aboriginal and Torres Strait Islander People (Bradley, Noonan, Nugent, & Scales, 2008) reveal that there remains significant work to be done on embedding cultural competence at a whole-of-institution level. Leadership in all of the domains (institutional, organisational, professional and individual) is seen as fundamental to bringing about the cultural change which is being called for.

Overview of the Culturally Competent Leadership Program

The CCLP was launched in February 2017, a first-of-a-kind professional development leadership program open to both professional and academic staff across the University of Sydney. Led by the NCCC, in itself a community of leaders of cultural competence dedicated to enacting the transformational change the University wants to achieve, the program materialised the University’s strategic vision and dedication to leadership initiatives in the cultural competence space by establishing and supporting a network or community of practice of cultural competence champions, who commit to leading initiatives in their local contexts. This type of individual agency was seen to complement leadership initiatives at the organisational and systemic level, and represented a bottom-up approach of sorts, seen as vital to achieving the University’s stated vision of cultural change and transformation. The common attributes, behaviours, knowledge and skills of culturally competent leaders identified in previous sections of the literature review informed, in many ways, the vision of the CCLP.

The transformation and change envisaged by the University of Sydney requires leadership that is open and committed to resilient sense of self through a reflective cycle of thinking and developing a critical reflective praxis, intra-personally or with the self, and inter-personally or with others. Praxis:

involves the continual, dynamic interaction among knowledge acquisition, deep reflection, and action at two levels - the intrapersonal and the extra-personal - with the purpose of transformation and paradigmatic change. At the intrapersonal level, praxis involves self-knowledge, critical self-reflection, and acting to transform oneself as a leader for social justice. At the extra-personal level, praxis involves knowing and understanding systemic social justice issues, reflecting on these issues, and taking action to address them (Furman, 2012, p. 203).

Culturally competent leadership requires critical approaches to the systems and paradigms that continue to perpetuate injustices, as much as it requires critical self-reflection and a personal journey of learning and transformation.

Culturally Competent Leadership Program Aims and Themes

The aims of the CCLP were to:

  1. 1.

    Develop influential leaders who will build resilience for change through the support of the broader network of leaders;

  2. 2.

    Develop influential leaders who will engage with a new and innovative discourse of culturally competent leadership;

  3. 3.

    Create a university-wide network or a community-of-practice of empowered and engaged leaders committed to nurturing the development of successive generations of culturally competent leaders.

The CCLP consisted of four themes to address the aims of the program:

  1. 1.

    Cultural competence: the ability to participate ethically and effectively in personal and professional intercultural settings.

Cultural competence, from an Australian perspective, acknowledges that it is built on Indigenous foundation that is informed through Indigenous ways of knowing, being and doing. Universities Australia (2011, p. 171) asserts that Indigenous cultural competence requires an organisational culture which is committed to social justice, human rights and the process of reconciliation through valuing and supporting Indigenous cultures, knowledges and peoples as integral to the core business of the institution.

  1. 2.

    Leadership praxis: leadership is viewed and experienced as both reflection and practice.

To lead within a culturally competent framework requires leaders who develop a more explicit moral literacy with respect to situations in which their organisations engage in cultural change. This requires leaders who are capable of exploring their own sense of moral purpose in their work through critical reflection; appreciate the importance of culture in leading ethically; and, understand the power of moral purpose as a mobiliser of practice. This type of leadership is integral to building culturally competent leaders.

  1. 3.

    Communities of practice (or networks): a community of leaders who care about cultural competence and share in its practice.

A community of practice in the context of CCLP is one in which there is support for cultural competence; that creates a common ground and sense of common identity built on a cultural competence foundation; where there is a community of committed leaders who care about cultural competence and create the social fabric of learning around it; and, where there is shared effective practice. This requires a united structure and approach, and connection with others.

  1. 4.

    Critical reflection: challenges the learner to question assumptions, beliefs and commonly accepted knowledge and to actively participate in what they learn.

Critical reflection involves a critique of the presuppositions on which beliefs have been built. The capacity to reflect relates to how effectively individuals can learn from their personal experiences. Critical reflection therefore provides a meaningful way for leaders to gain genuine understanding. Culturally competent leaders should be highly skilled in critical reflection theory and practice.

Culturally Competent Leadership Program Phases

Phase 1: Self-nomination and selection of participants

NCCC academic staff set out in the first phase of the program to engage and consult with faculty Deans and divisional/departmental Directors. These preliminary conversations aimed to introduce senior leaders and decision-makers to the program, respond to any queries, concerns and suggestions, and encourage them to promote the program widely across their faculties and divisions/departments. Interested staff, both professional and academic, self-nominated and applied by submitting an expression of interest to their Dean or Director, who then in turn selected a maximum of five applicants based on internal selection criteria and forwarded the list to the CCLP team. Deans and Directors were required to commit to supporting nominees to be available for the entire program. The NCCC was not involved in the selection process, aside from suggesting possible criteria for consideration, including the following:

  1. 1.

    Be passionate about promoting cultural competence and actively coaching and mentoring others;

  2. 2.

    Willingness and ability to consult and collaborate with others, balancing this with a preference for making decisions;

  3. 3.

    Be comfortable dealing with other staff, forming new relationships, and at ease expressing opinions; and,

  4. 4.

    Be working towards completing the NCCC online cultural competence modules before the retreat.

Phase 2: Program launch

The NCCC hosted a program launch event, which involved an introductory workshop on cultural competence in the higher education context and the call for leadership at all levels of the university community to participate. This was followed by a networking event with participants, NCCC staff, and other relevant stakeholders from the University of Sydney that informed the CCLP education program.

Phase 3: Off-site residential program

Sixty-six CCLP participants, including both academic and professional staff from across the University’s faculties and departments, attended a three-day off-site residential program in South Durras, New South Wales (Fig. 3.1). Sessions focused on building participants’ capabilities in critical self-reflection, on building knowledge in cultural competence discourse through theory and practice, including leadership discourse, and on exploring concepts of communities of practice. Participants were also led through a future casting session, planning for how they might embed what they learned through the residential program into their local contexts, and into the business of the University. Participants also had the opportunity to spend a half day with members of the local Indigenous community.

Fig. 3.1
figure 1

Participation of staff by organisational units

Full size image

Phase 4: NCCC workshops

After the CCLP retreats, participants were invited to participate in a series of NCCC workshops that would build their cultural competence knowledge-base, as well as their confidence in practicing and championing cultural competence in their local contexts.

Phase 5: Ongoing engagement and support

Following the off-site residential program, and in conjunction with the roll-out of NCCC workshops, the CCLP team, at the suggestion of participants, organised monthly networking sessions as an opportunity for participants to reconvene and share information and ideas on embedding cultural competence in their local contexts.

Evaluation Methodology

A responsive evaluation approach (Stake, 1967; Stake & Abma, 2005; Abma, 2005) incorporating process and outcome dimensions was adopted for the evaluation. Stake (1967) argues that evaluated programs can include different stakeholders who share different views, perspectives and expectations. Therefore, it is the responsibility of evaluators to acknowledge such diversity when designing the evaluation of programs or activities. According to Abma (2005, p. 288), ‘responsive evaluation does not only deliver evidence in time, but also evidence that is context-bound. It produces local knowledge that enables practitioners to use it in their context, in a specific case’.

The evaluation of CCLP was considered a collaborative partnership in that it equitably involved participants and evaluators in the evaluation process. Participants contributed their experience of the CCLP, and the knowledge gained will be incorporated into action for the next reiteration of the CCLP. The evaluation recognised the participants as a social and cultural entity, in the sense that they are all members of the University community and are actively engaged in the evaluation process. In line with the responsive evaluation approach, processes and outcomes were identified, documented, analysed and evaluated, considering the experiences and perspectives of the participants. The evaluation was guided by the following principles:

  • Recognise the participants as a unit of identity;

  • Build on strengths and resources within the University community;

  • Facilitate collaborative, equitable involvement of all partners in the evaluation; and,

  • Integrate knowledge and action for mutual benefit of the University community.

Research and Evaluation Plan and Procedures

The following steps were undertaken in developing the research plan:

  1. 1.

    Establish the evaluation team;

  2. 2.

    Formulate the research aims; and

  3. 3.

    Identify the research design and program logic framework.

Ethics Procedure

The CCLP research team gained research ethics approval from the University’s Human Research Ethics Committee (HREC) at the onset of the program. Some minor clarifications and additional information were sought by the HREC in the first instance, and these were reviewed and approved shortly after submission, and research activities commenced thereafter. Participation in the research and evaluation of the CCLP was on a voluntary basis, and program participants were made aware of the research and evaluation project prior to the off-site residential phase. All program participants received a Participant Information Statement (outlining the breadth, scope, methodology of the evaluation) as well as a Participant Consent Form, through which they were to provide written consent to participating in the research and evaluation activities throughout the life-time of the program.

Evaluation Logical Framework

A logical framework, or log-frame was developed for the CCLP evaluation, detailing the evaluation’s goals, activities and anticipated results vis-á-vis the CCLP’s over-arching aims. The log-frame was viewed as a tool for improving the planning, implementation, management, monitoring and evaluation of the CCLP (see Appendix 1).

Evaluation Aims

As mentioned in the beginning of this chapter, the evaluation of the CCLP had three distinct aims:

  1. 1.

    To appraise the achievements of CCLP objectives;

  2. 2.

    To assess the soundness of the CCLP approach and component strategies, and analyse their respective performance in relation to achieving targeted objectives and sustainability of results achieved; and,

  3. 3.

    To analyse constraints, lessons learned, the use of evidence-based best practices and evaluate the strategic opportunity to build upon achievements to inform further CCLPs.

Evaluation Design and Data Collection

The evaluation utilised a mixed-methods design that included pre- and post-program surveys; participant focus groups; and, participant interviews (one-on-one). A total of sixty-six staff, professional and academic combined, participated in the CCLP. The two surveys were conducted online, through the Survey Monkey platform—one prior to the residential, and one at its conclusion. Key informant interviews and focus group discussions were held at the end of the program, and were audio-recorded, and subsequently transcribed. Personal names and other forms of identification were redacted to ensure anonymity, and any publications disseminating the results were de-identified.

Data Analysis

The evaluation data was analysed utilising the following five criteria:

  1. 1.

    Relevance: the extent to which the CCLP objectives are consistent with the University’s needs, priorities and policies.

  2. 2.

    Effectiveness: the extent to which the CCLP aims were achieved or are expected to be achieved.

  3. 3.

    Efficiency: the extent to which the CCLP activities were achieved within the allocated budget.

  4. 4.

    Sustainability: the extent to which the benefits from the CCLP continue after its completion.

  5. 5.

    Impact: the effects produced by the CCLP.

Online survey data, for both the pre- and post-residential phase of the program, was analysed in the first instance by two members of the research team. Raw data was initially exported from the Survey Monkey online platform and interpreted/analysed by the Chief Investigator who relayed findings in a summary document. This document was circulated amongst the research team, along with the exported raw data, for review and comment to ensure consensus in its interpretation. Focus group discussions and individual informant interviews were audio-recorded, and subsequently transcribed by professional transcription services, outsourced. Members of the research team reviewed and analysed the transcripts individually and subsequently convened to share their individual findings and reach a general consensus on these findings. Content analysis was undertaken by coding the content under the pre-determined themes of relevance, effectiveness, efficiency, sustainability and impact. The process was undertaken by two individual researchers and these findings were reviewed by a third researcher.

Discussion and Conclusion

The inaugural CCLP had a very bold and ambitious vision in creating its first generation of leaders, who would be committed to championing the cultural competence agenda and leading initiatives within their local settings—there are as many successes to be celebrated as there are lessons to be learned, as the evaluation of the CCLP has revealed. At a glance, program evaluation participants highlighted a series of program successes, particularly as they relate to the residential component, in that it:

  • engendered transformation at the individual, personal level through experiential learning opportunities and critical self-reflection activities that facilitates better understanding about Indigenous history, cultures and social issues, appropriate engagement with Indigenous peoples, as well as the importance of cultural competence;

  • facilitated relationship-building and nurturing amongst and between participants;

  • developed foundational knowledge of cultural competence for participants, and hence equipping them with the confidence they needed to champion the agenda;

  • created a desire for ongoing learning and gaining further knowledge; and

  • developed a relationship with the NCCC and its staff resulting in an enhancement of the CCLP networks.

As stated in the introduction of this chapter, the evaluation sets out to appraise the achievements of the CCLP key objectives that include the development of culturally competent leaders and the creation of a university-wide network or a community of practice of such leaders. The appraisal of these objectives has revealed that the program succeeded in creating and developing a network of cultural competence champions. The inaugural CCLP certainly engendered transformation and shifts at the individual level, facilitated relationship-building amongst and between participants, and developed the majority of participants’ foundational knowledge in culturally competence discourse and in culturally competent leadership discourse, creating a desire for ongoing learning and gaining more knowledge. The majority of respondents in the post-retreat survey (21/29 participants) felt ‘relatively’ confident that they will have an impact on embedding cultural competence on the local context post-treat. Seven of the remaining respondents felt ‘very’ confident.

The evaluation’s appraisal of NCCC staff facilitation of the CCLP has shown that the methods, approaches, processes and content have met program participants’ expectations, hence setting the benchmark for future CCLPs. What the evaluation also revealed was the CCLP facilitator team’s ability to respond to participant suggestions and feedback through progressive program revision and streamlining. Feedback to date has demonstrated that participants are now networked, engaged, and conversing on an ongoing basis, through diverse channels, sharing their local stories and practice with the broader community of champions.

Limitations

It is also important to note that the evaluation itself had limitations in terms of:

  1. 1.

    Measuring impact: in the absence of standard cultural competence measures of success in higher education contexts, it is difficult to determine the extent to which professional development programs like the CCLP have impacts on the University’s path to becoming culturally competent.

  2. 2.

    Data collection methods: it is worth highlighting that while the data collection instruments gathered sufficient data for this evaluation, further methods would have added useful data through participant observation and non-participant observation. In participant observation one member of the evaluation team could have taken an active role in the program, from the beginning phase to the end, which would provide an insider viewpoint and generate rich data. Additionally, one member of the evaluation team could take on the role of a non-participant observer, observing the CCLP activities but without taking an active part.

  3. 3.

    Potential bias of the research team: the Chief Investigator for the evaluation, who conducted interviews and focus group data collections and analysed the data, was also a staff member contributing to the CCLP retreats. This presented a level of potential bias. To mitigate this, two more levels of data analysis were undertaken by two NCCC academic staff who were independent of all of the CCLP design and delivery.

Conclusion

The responsive evaluation approach (Stake, 1967; Abma, 2005) requires evaluators to consider different perspectives and values of the program’s stakeholders. In the evaluation of the CCLP program, different measures were adopted in order to enable the evaluators to capture the diversity in terms of the participants’ perspectives and values towards cultural competence in general and the CCLP program in particular. The evaluation acknowledged the diversity of participants who shared different views, perspectives and expectations, as well as holding a variety of positions including academic and professional staff. Participants were invited to join the pre- and post-program surveys, focus groups as well as one-on-one interviews. As a result, data collected from those evaluation methods allowed the evaluators to better understand the participants’ different as well as similar perspectives and values. The use of the pre- and post-program surveys was both valuable and effective as it helped to reveal, in part, any cognitive or emotional changes that participants may have after joining the CCLP program.

Many participants acknowledged significant transformation at the individual level in terms of engaging with Indigenous peoples and views towards cultural competence. Those findings allowed the evaluators to acknowledge one significant contribution of the CCLP program as well as the achievement of the program’s objectives: to develop influential leaders who will engage with a new and innovative discourse of culturally competent leadership. CCLP participants’ transformation, as revealed via the post-program survey, focus groups and one-one-one interview, is a good starting point to developing culturally competent leaders. It is because such transformation can allow leaders to identify and question systems that generate inequity, disparity and social injustice (Horsford et al., 2011; Su & Wood, 2017).

The design used for the CCLP evaluation has shown that the questions, data sources and data collection methods were adequate to address the evaluation aims. The design involved the participants as partners in the evaluation, building on their strengths and resources and integrating their knowledge and perspectives for mutual benefit of the University community. This evaluation approach considers the full context of participants, rather than seeing them in isolation from the University environment, culture, and identity

The key advantages of the evaluation design included enhancing the relevance and use of the evaluation data by the CCLP team and improving the quality and validity of evaluation by incorporating the knowledge and experiences of the participants. The findings of this evaluation should assist the University to continue to address the professional development needs of its staff in the area of cultural competence.